University of Southern California Dissertations and Theses

In-situ quality assessment of scan data for as-built models using building-specific geometric features

(USC Thesis Other)

In-situ quality assessment of scan data for as-built models using building-specific geometric features

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

1

In-situ Quality Assessment of Scan Data for As-Built Models Using Building-
specific Geometric Features

PhD Dissertation

Pedram Oskouie

Guidance Committee Members:
Prof. Burcin Becerik-Gerber (Chair)
Prof. Lucio Soibelman, Prof. Azad Madni

December 2017
Sonny Astani Department of Civil and Environmental Engineering
Viterbi School of Engineering
University of Southern California

2
In-situ Quality Assessment of Scan Data for As-Built Models Using Building-specific
Geometric Features
ABSTRACT
With the recent advent of remote sensing technologies, terrestrial laser scanners (TLSs) have been
employed by the architecture, engineering, construction, and facilities management (AEC/FM) for
several different uses such as recording the as-built and as-is conditions of buildings. TLS systems
capture geometric information of a building in the order of minutes and with millimeter-level
accuracy, which makes them a valuable asset, especially when capturing the details of the
geometry is required. However, to date, challenges remain with accurate detection, extraction, and
modeling of building primitives from the scan data. These challenges often stem from the early
data acquisition stage. Existence of data quality issues such as missing data, noise, and low point
cloud density result in generating inaccurate building information models (BIMs) and often raises
the need for redoing the scan process. This research focuses on analyzing the scan data of
buildings, and specifically investigates the data quality assessment and quantification for 3D
geometric modeling of the architectural elements on building exteriors (i.e. walls and openings)
within the as-built modeling context. The knowledge about scan data acquisition means and
methods (e.g., scan planning, adjusting resolution settings) is integrated with the 3D modeling
techniques to realize their synergy in the context of generating accurate architectural as-built
models. The integration bridges the existing gap in accurate as-built modeling by taking into
consideration the quantitative requirements for generating an accurate 3D model in the very first
steps of data collection. The research objectives are (I) identifying the data quality requirements
for scanning small to medium size buildings’ exterior with an aim to use the data for accurate 3D
modeling, and (II) designing a framework for in-situ identification of scan data quality issues
related to extracting and modeling architectural elements of buildings’ exterior. The scan setting
parameters (e.g., angular resolution, field of view) and data characteristics (e.g., point cloud
density, noise, data completeness) that influence the scan data quality were studied through an
extensive literature review and interviews with laser scanning specialists. Next, a set of data quality
factors are derived and the quality issues that could be controlled during a scanning session are
identified. To ensure the influence of building’s geometry on the quality issues is considered, an
image processing method is introduced for creating 3D mock-up models of the buildings. The
mock-up models are then used to study the quantitative impacts of data quality issues such as point
cloud density and noise on the accuracy of generated as-built models. After the impact analysis, a
set of building component-level data quality requirements are derived and employed as guidelines
to evaluate the quality of real-life scan data. The findings of this research indicate that ground-
based images along with information such as building’s footprint, and floor-to-floor height could
be used to generate mock-up models to represent the geometry of the building. In addition, the
data quality impact analysis showed that even though the scan data of buildings with partially
similar geometric properties are impacted in a similar fashion, the quantitative degree of the impact
differs for every studied building. Lastly, the derived data quality requirements are used to
evaluate the quality of collected scan data on the job site, and the findings are used to improve the
quality of data by employing corrective data collection means and methods.

3
DEDICATION AND ACKNOWLEDGEMENT
I would like to extend my sincere appreciation to Dr. Burcin Becerik-Gerber and Dr. Lucio
Soibelman for their gracious supervision and priceless advice. It was your continuous
encouragement and enthusiasm in research and innovation that motivated me in every step of my
PhD career. I will always be grateful for every precious lesson that I learned from you throughout
this memorable five-year journey. Furthermore, I would like to thank Dr. Azad Madni for his
guidance and acceptance to join my dissertation committee.
This dissertation is dedicated to my beloved family and close friends that supported me
unconditionally throughout the course of my doctoral studies. I submit my deepest gratitude to my
dear parents Susan Khattatan, and Hassan Oskouie, as well as my brother and best friend Payam
Oskouie for their continuous support during the ups and downs of this time period. The path to
success have been paved and brightened with the torch of your love and inspiration. Words simply
fail to describe how much I owe all this to you and your emotional support. In addition, I would
like to extend my thanks to my girlfriend for surrounding me with affectionate support especially
during the last months. I would also like to thank my dear friend, Dr. Arsalan Heydarian who has
been as close and sincere as a brother to me during these years (thanks for all your supports and
invaluable advice!). Lastly, I would dedicate this thesis to anyone who is promoting education and
innovation around the world. It is only with education that we can fight for peace and equality of
rights in any sort and form in the global society.
Last but not least, I humbly extend my gratitude to the i-Lab research team in the Civil and
Environmental Engineering Department at the University of Southern California (it has been an
honor to work with you all).
Additionally, this research was partly supported by the Kiewit Corporation and Riegl USA. Any
opinions, findings, and conclusions or recommendations expressed in this document are those of
mine and do not necessarily reflect the views of the sponsors.

4

Table of Contents
Chapter 1 EXECUTIVE SUMMARY ........................................................................................ 11
Chapter 2 MOTIVATION .......................................................................................................... 14
Chapter 3 SCOPE ....................................................................................................................... 15
Chapter 4 SCIENTIFIC SIGNIFICANCE AND IMPACT ........................................................ 16
Chapter 5 LITERATURE REVIEW AND PROBLEM STATEMENT .................................... 17
5.1 Identification of Scan Data Collection Errors ................................................................... 17
5.2 Improving the Data Quality by Scan Planning.................................................................. 18
5.3 Identifying the Recent Automated 3D As-built Modeling Methods ................................. 19
5.4 Quality Assessment of Generated As-built Models .......................................................... 20
Chapter 6 RESEARCH GOAL, OBJECTIVES, AND QUESTIONS ....................................... 22
Chapter 7 IDENTIFICATION OF SCAN DATA QUALITY FACTORS ................................ 23
Chapter 8 A DATA QUALITY-DRIVEN FRAMEWORK FOR ASSET CONDITION
ASSESSMENT USING LIDAR AND IMAGE DATA ............................................................... 30
8.1 Existing Scan Planning Methods ..................................................................................... 30
8.2 Data Quality-driven Scan Planning Framework .............................................................. 31
8.3 Case Study ........................................................................................................................ 32
8.3.1 UAV Image Collection .............................................................................................. 32
8.3.2 Image-based 3D Reconstruction ................................................................................ 33
8.4 Data Processing and Results ............................................................................................ 33
8.4.1 Geometrical Features Recognition ............................................................................ 33
8.4.2 Geometrical Features Classification .......................................................................... 34
8.4.3 Laser Scanner Position Determination ...................................................................... 35
8.5 Conclusion ........................................................................................................................ 36
Chapter 9 AUTOMATED MEASUREMENT OF HIGHWAY RETAINING WALL
DISPLACEMENTS USING TERRESTRIAL LASER SCANNERS ......................................... 37
9.1 MSE Walls ........................................................................................................................ 37
9.2 Methodology ..................................................................................................................... 40
9.2.1 MSE Wall 3D Point Clouds and Basic Assumptions ................................................. 40
9.2.2 Feature Extraction Using Laser-scan Data ................................................................ 42
9.3 Evaluation Using Real-life Data ....................................................................................... 49
9.3 Evaluation Using Simulated/Synthetic Data ....................................................................... 52

5
9.4.1 Comparison of Simulated Scans with Real-life Scan Data ....................................... 54
9.4.2. Results and Discussion ................................................................................................ 56
9.5 Extended Validation ......................................................................................................... 62
9.6 Limitations and Conclusions ............................................................................................ 65
Chapter 10 AUTOMATED RECOGNITION OF BUILDING FACADES FOR CREATION
OF AS-IS MOCK-UP 3D MODELS ............................................................................................ 67
10.1 Review of Existing Work ............................................................................................... 67
10.1.1 Gradient-based methods .......................................................................................... 68
10.1.2 Bottom-up methods ................................................................................................. 68
10.1.3 Top-down methods .................................................................................................. 69
10.2 Computing Gradient Profile and Detecting Arrangement Style of Façade Elements .... 70
10.3 Predicting Locations of Façade Elements ...................................................................... 73
10.4 Detecting Shape of Façade Elements ............................................................................. 77
10.5 Classifying Façade Elements .......................................................................................... 78
10.6 Defining Split Grammar for Reconstructing a Building’s Façade ................................. 82
10.7 Performance Evaluation Metrics .................................................................................... 84
10.8 Results and Discussion ................................................................................................... 85
10.9 Limitations and Conclusions .......................................................................................... 89
Chapter 11 TOWARDS ASSESSING THE QUALITY OF SCAN DATA: IMPACTS OF
NOISE AND POINT CLOUD DENSITY ON THE ACCURACY OF BUILDING MODELS
FROM POINT CLOUDS ............................................................................................................. 90
11.1 Need for Scan Data Quality assessment .......................................................................... 90
11.2 Point Density Modeling .................................................................................................. 92
11.3 Noise Modeling ............................................................................................................... 94
11.4 3D Modeling Process ...................................................................................................... 97
11.5 Findings and Discussion ................................................................................................. 98
11.6 Real-life Data Collection ............................................................................................... 104
11.8 Limitations and Conclusions ......................................................................................... 110
Chapter 12 IN-SITU ASSESSMENT OF SCAN DATA QUALITY ...................................... 111
12.1 Identifying Missing Data .............................................................................................. 111
12.2 Aligning Mock-up Model and Scan Data .................................................................... 115
12.3 Computing Predicted Errors for Building Components ............................................... 116
12.4 Improving Data Quality ............................................................................................... 119

6
12.5 Limitations and Conclusions ........................................................................................ 124
Chapter 13 CONCLUSIONS, LIMITATIONS, AND FUTURE WORK ............................... 126
13.1 Conclusion ..................................................................................................................... 126
13.2 Limitations and Future Work ........................................................................................ 128
Relevant Publications to Date ..................................................................................................... 130
Peer-Reviewed Journal Papers ................................................................................................ 130
Peer-Reviewed Journal Papers in Preparation ........................................................................ 130
Peer-Reviewed Conference Papers ......................................................................................... 130
Appendices .................................................................................................................................. 131
Appendix 1. Interview questions ............................................................................................. 131
References ................................................................................................................................... 134

7
Figure List
Figure 1. Word frequency diagram of the identified quality factors based on
[12,19,24,28,38,48,50,58,59,67,70,73,88,97,99,114,119,120,128-130,144]................................ 24
Figure 2. The importance level of the scan data quality factors according to the interviews with
LiDAR specialists ......................................................................................................................... 25
Figure 3. Relationships of scan parameters and data quality factors (Example of four scan
parameters being related to with at least three quality factors). ................................................... 29
Figure 4. An overview of data quality-driven scan planning framework ..................................... 31
Figure 5. Case Study-Mudd Hall Courtyard ................................................................................. 33
Figure 6. Steps for Localization of Features of Interest in 3D Point Cloud ................................. 35
Figure 7. Determination of Scanner Position based on Features Orientations ............................. 36
Figure 8.Cross section profile of a typical MSE wall (Adapted from U.S. Department of
Transportation [57]) ...................................................................................................................... 38
Figure 9. An MSE wall's point cloud ............................................................................................ 41
Figure 10. Rotation of MSE wall along its main axis. The yellow dashed line shows the rotation
of the red column along the X-axis with rotational angle of φ. .................................................... 42
Figure 11. RANSAC outliers (colored as white) along with false positives (colored as yellow) 46
Figure 12. Process of extracting horizontal joints from RANSAC outliers ................................. 47
Figure 13. The extracted joints from two sets of point clouds are clustered and the clusters with
closest Euclidean distance are compared to determine the displacement of joints. ...................... 49
Figure 14. Evaluation of the proposed method using real-life MSE wall scanned data ............... 51
Figure 15. Accuracy of extracting horizontal joints from columns of panels .............................. 51
Figure 16. Average settlement and lateral displacement of joints within each column of the
studied wall ................................................................................................................................... 52
Figure 17. 3D modeled MSE walls: Top left: front view, Top right: section view, and Bottom:
top view ......................................................................................................................................... 52
Figure 18. Section views of different wall movements that were modeled for simulation. (a)
Lateral displacement, (b) Settlement, (c) Single angle rotation: All panels are rotated with the
similar angle “α”, (d) Multiple angles rotation: The two middle panels are rotated with “α” and
the others with “β” where α, β = 1° and 2° interchangeably. ....................................................... 53
Figure 19. Snapshots of real-life scan data and simulated scan .................................................... 55
Figure 20. Real-life scan vs. Simulated scan with random noise (Std. = 0.001 m) ...................... 55
Figure 21. Heat map of average displacement measurement error for different scan positions and
setups. For each scan position, the average value of measurement errors for all types of
movements is calculated. .............................................................................................................. 58
Figure 22. Average accuracy of feature extraction for different scan setups. The average
accuracy for all types of wall movements is calculated. ............................................................... 61
Figure 23. Average error of displacement measurement for different scan setups. The average
accuracy for all types of wall movements is calculated. ............................................................... 61
Figure 24. Comparison of average error values for wall displacement measurements along Y and
Z directions for conditions in which wall has rotated and where AOI has been 0°. ..................... 62
Figure 25. 3D models of two other types of MSE walls ............................................................... 64

8
Figure 26. The overlays of projected horizontal and vertical gradients (cyan color) on two façade
images with different arrangement styles. Façade type 1 (top) has separated elements and façade
type 2 (bottom) has connected elements. (Building image acquired from ©Google) .................. 72
Figure 27. (a) The distribution of the gradient differences for a set of façade images; (b) A
sample of the cumulative histogram of a façade with separated elements (façade type 1,
horizontal axis); (c) A sample cumulative histogram of a façade with connected elements (façade
type 2, horizontal axis). ................................................................................................................. 73
Figure 28. An example of the process of predicting the locations of elements for a façade with
connected elements. (a) Original gradient histogram; (b) Gradient histogram after applying of H-
minima and H-maxima filters; (c) Equivalent signal profile of the gradient histogram; (d)
Smoothed signal with red circles representing the detected valley points; (e) Horizontal division
of the façade based on the detected valley points. (Building image acquired from ©Google) .... 74
Figure 29. Process of predicting locations of elements for a façade with separated elements. (a)
Façade’s image; (b) Scattered lines representing the locations of strong gradients across vertical
and horizontal directions; (c) Lines representing the boundaries of the clusters of strong
gradients; (d) Clustered lines of strong gradients; (e) Binary image of the regions with
overlapping strong gradients; (f) Bounding boxes of the predicted elements. (Building image
acquired from ©Google) ............................................................................................................... 75
Figure 30. Diagrams of average silhouette width vs. number of clusters of the façade in Fig. 4.
According to the diagrams, the horizontal and vertical gradients have to be divided into 8 and 4
clusters respectively. ..................................................................................................................... 76
Figure 31. Overview of the shape detection process. Regions without vegetation are identified,
their HOG features are extracted, and their shape is determined by a linear SVM classifier.
(Building image acquired from ©Google) .................................................................................... 78
Figure 32. Façade elements are classified based on their feature descriptors. For this specific
façade example, different instances of each class of windows are identified through color-coded
bounding boxes. (Building image acquired from ©Google) ........................................................ 81
Figure 33. Summary of the overall steps for classifying the façade elements. (Building image
acquired from ©Google) ............................................................................................................... 81
Figure 34. Process for generating a building mock-up model: (a) Building’s footprint; (b)
Footprint is extruded based on the estimated height of the building; (c) Façade segments are split
through procedural modeling rules; (d) Façade is textured using the detected elements; (e)
Building’s mock-up model is generated. ...................................................................................... 83
Figure 35. Case-study buildings, their BIM models, and the generated mock-ups. (Top two
building images acquired from ©Google) .................................................................................... 85
Figure 36. Samples of element location prediction for two datasets (ECP database:
http://vision.mas.ecp.fr/Personnel/teboul/data.php) ..................................................................... 86
Figure 37. The frequency distribution of different point spacings, and the fitted power series
model............................................................................................................................................. 93
Figure 38. Applying different levels of noise to the building’s synthetic scan data. The noise
level increases from left to right. (Top row: top view of the building, Bottom row: front view of
the building) .................................................................................................................................. 97

9
Figure 39. Wall modeling errors at building-level for the case study buildings. Different levels of
LOD requirements defined by GSA are shown as dashed lines. ................................................ 101
Figure 40. Opening detection errors at building-level for the case study buildings. Different
levels of LOD requirements defined by GSA are shown as dashed lines. ................................. 102
Figure 41. Comparative mean errors of wall modeling and opening detection for the three case
studies. (Note: each data point represents the mean results of different noise levels) ................ 103
Figure 42. Visualization of component-level mean errors of wall modeling and opening detection
..................................................................................................................................................... 104
Figure 43. Positioning of the TLS for collecting data from the case study buildings along with
samples of the collected data ...................................................................................................... 105
Figure 44. Comparison of predicted and actual mean errors of wall modeling and opening
detection at building-level........................................................................................................... 108
Figure 45. Differences between predicted and actual errors of wall modeling at component-level
..................................................................................................................................................... 109
Figure 46. Differences between predicted and actual errors of opening detection at component-
level ............................................................................................................................................. 110
Figure 47. Identifying missing data due to occlusion: (a) Point cloud and the scan positions, (b)
Bounding boxes around the detected missing data, (c, d, and e) Ray-tracing from different scan
positions. ..................................................................................................................................... 115
Figure 48. Aligning the mock-up model and the building’s point cloud .................................... 116
Figure 49. Process of computing predicted errors for every building component ...................... 117
Figure 50. Color-coded visualization of predicted errors: (a, b) case study 1, (c) case study 2. 118
Figure 51. Snapshots of the initial (top) and improved (bottom) scan data for the three case study
buildings: (a) case study 1, (b) case study 2, and (c) case study 3.............................................. 124

10

Table List
Table 1. Scan Data Quality Factors derived from the literature ................................................... 23
Table 2. Results and LOD definition ............................................................................................ 34
Table 3. Simulated wall movements and scan setups ................................................................... 54
Table 4. Average distance of simulated scan from real-life scan data. Note that each simulation is
compared with five different real-life scan and the average is calculated and presented below. . 56
Table 5. Matrix of feature extraction accuracy for different simulation scenarios (%) ................ 59
Table 6. Matrix of displacement measurement error for different simulation scenarios (mm) .... 60
Table 7. Average feature extraction and displacement measurement error for manually moved
real-life scan data .......................................................................................................................... 63
Table 8. Average accuracy of feature extraction and error of displacement measurement for
different scan setups for wall types 1 and 2 .................................................................................. 64
Table 9. Features used at different steps of façade reconstruction ............................................... 80
Table 10. Performance results of the proposed façade structure recognition method for different
datasets (The numbers are percentages) ....................................................................................... 87
Table 11. Window and door detection accuracies for different datasets ...................................... 87
Table 12. Average error range (m) of the mock-up model as compared to the BIM model ........ 88
Table 13. Qualitative comparison of mock-up models and BIM models ..................................... 88
Table 14. The scan parameters and their expected values for determining possible point spacing
in synthetic scan data .................................................................................................................... 93
Table 15. Noise modeling parameters .......................................................................................... 95
Table 16. Real-life scan data statistics (point spacings are rounded to closest number) ............ 105
Table 17. Estimated timing for implementing the proposed in-situ quality assessment process 119
Table 18. General categories of the detected quality issues, corrective data collection actions, and
their abbreviated codes. .............................................................................................................. 120
Table 19. Improvement of scan data for different components of the case study buildings
(measurements are in millimeter) ............................................................................................... 121
Table 20. Overall improvement of data for the case study buildings (measurements are in
millimeter)................................................................................................................................... 121

11
Chapter 1 EXECUTIVE SUMMARY
Accurate documentation of building’s condition facilitates the decision-making processes in the
construction (e.g. project progress monitoring) and operation (e.g. energy retrofit analysis) of
buildings [57,60]. Laser scanners have been widely used in the AEC/FM for generating as-built/as-
is model of buildings. The rapid and accurate data collection with TLS enables continuous
documentation of buildings, resulting in an improved decision-making practice. Nonetheless, the
process of converting the data collected (3D point cloud) from TLS to 3D building models is
known to be time-consuming, and often erroneous due to deficiencies in data quality [44].
Although there have been several studies on improving the scan data collection practice, due to
lacking a quality assessment following the data collection, data quality issues such as missing data,
low point cloud density and noise still introduce several challenges to generating accurate building
models [44]. This research investigates the scan data quality assessment for generating accurate
as-built models from laser scan data. Specifically, this research aims to provide an x-ray vision to
scan specialists by proposing a quantitative insight into the quality issues of scan data. The research
objectives are organized into the following main research tasks: (1) identifying scan data quality
factors that influence the accuracy of as-built modeling algorithms; (2) Extracting the geometric
properties of buildings’ exterior by creating mock-up 3D models using ground-based images and
shape grammars; (3) generating synthetic scan data by using the geometric information from the
mock-up models; (4) studying the combinational impacts of quality issues, and building’s specific
geometry on the accuracy of the 3D models generated from the synthetic data; and (5) using
findings from the impact analysis to evaluate and improve the quality of real-life scan data.
In chapter 7, we studied scan setting parameters (e.g., angular resolution, field of view) and data
characteristics (e.g., point cloud resolution, data completeness) that influence the scan data quality
through an extensive literature review and interviews with laser scanning specialists. We identified
nine quality factors, from which the four factors of measurement accuracy, registration accuracy
consistency, resolution, and scan coverage were more frequently mentioned in different studies.
The scan parameters and data characteristics were correlated to quality factors based on the
findings from literature review and the interviews.
In chapter 8, which presents one of our earliest studies, we investigated linking of the quality
factors to scan data collection techniques. To study the possibility of improving the scan data
quality in advance of collecting data, we proposed a framework to integrate image-processing
methods with point cloud processing techniques for defining a data quality-driven approach for
scan planning. The main objective was to study the geometry of the target facility using image
data, and to incorporate the geometric properties into a model-based data quality-driven scan
planning. The framework includes the following steps: 1) capturing images of a target building
using a commercially available unmanned aerial vehicle (UAV) and generating a 3D point cloud,
2) recognizing the target’s geometrical information using the point cloud, 3) extracting the features
of interest (FOI) using the point cloud, 4) generating multiple scanning scenarios based on the data
processing requirements and the extracted features, 5) identifying the best scan plan through
iterative simulation of different scanning scenarios, and 6) validating the scan plan using real life
data.

12
A known challenge with studying scan data is that collecting sufficient data for analyzing the
robustness of data processing algorithms is time-consuming and often impossible especially in the
case of large-sized facilities due to the job-site conditions. Therefore, the process of generating
synthetic data, deriving knowledge based on studying them, and applying the knowledge to real-
life scan data is widely adopted by the researchers [106]. Chapter 9 presents one of our studies in
which we designed and tested a novel automated method for measuring the displacements of
under-construction highway retaining walls using synthetic scan data. The generation of synthetic
data allowed us to evaluate the robustness of our method using a comprehensive set of simulated
scanning scenarios, and under various data quality conditions. To examine whether the synthetic
data is a realistic representation of real-life scan data, we compared real-life scan data of highway
retaining walls with a set of synthetic data that were generated using 3D models of the walls. As
the findings confirmed the similarity between the two, we used the synthetic data to predict the
performance of our designed method for detecting the displacements of highway retaining walls
using TLS data.
Knowing that synthetic data could provide a realistic representation of real-life scan data, chapter
10 focuses on investigating an automated method for generating baseline 3D models of buildings.
These models will be used in the latter steps of this research to generate synthetic scan data of
buildings. In contrast to previous works that unrealistically used the BIM models of buildings to
study the quality of scan data, this research is based on using mock-up models of buildings. The
generation of mock-up models allows us to incorporate the specific geometric features of buildings
into the study of scan data quality issues. We proposed an automated method for reconstructing
building façade elements using rectified ground images. The objective was to generate mock-up
3D models for buildings that lacked an updated as-built model. The method starts by collecting
ground images from the building’s façade. The images get rectified to eliminate the projective
effects, and then assigned to a specific wall through a semi-automated process. Next, the gradient
profile of every image is studied to predict the location of façade elements. The blocks of pixels
with high concentrations of vertical and horizontal gradients are taken as candidate location of
façade elements. The process continues by extracting a set of local and global features to provide
color and texture information of the pixel blocks. The features are then used to classify the
elements as windows or doors, and to detect whether their shape is rectangular or arched. Once the
façade elements are recognized, the number of floors are determined based on the rows of windows
on each wall. The floor-to-floor height is then derived from reference design manuals based on the
building’s design typology. In addition, 2D footprints of the building are acquired either from
online databases such as Google Maps or from building’s drawings, if available. The footprints
along with the estimated height of the building are used to create the building’s mass extrusion.
Next, for every wall on the building, a split grammar is defined based on the corresponding
detected architectural elements. Finally, the split grammars reconstruct the building’s façade
structure using procedural modeling and. As the result, the mock-up model of the building with
representative geometry, doors, and windows is created. The results indicated an overall average
accuracy of 80.48% in classifying the façade architectural elements. In addition, the mock-up
models were created with dimension errors less than 40 cm.
In chapter 11, we generated synthetic scan data of three case study buildings with varying
architectural elements by using their 3D mock-up models. For each building, considering its
specific geometric properties, point clouds were generated with varying point densities (35
conditions) and different noise levels (10 conditions). A building component-level deviation

13
analysis was then performed to enable a quantitative study on the combined impacts of point cloud
density and noise, as well as the building-specific geometric properties on the accuracy of 3D as-
built models generated from the synthetic scan data. Finally, a set of data quality requirements (i.e.
model errors under different data quality conditions) were derived by quantifying the studied
impacts, and used as guidelines to predict the errors of 3D models generated from real-life scan
data. The findings indicate that the data quality requirements could be used to predict the errors of
3D models by an average prediction accuracy of 3 mm.
Lastly, in chapter 12, we investigated the feasibility of implementing an automated in-situ quality
assessment of scan data by using the mock-up models, as well as the model errors derived from
the proposed impact analysis. The quality assessment process starts once the data collection is
completed. An interface is provided to manually align the building’s mock-up model with the
building’s point cloud. The quality assessment is then continued by automatically detecting the
jobs-site occlusions that have caused missing data. Next, using the derived data quality
requirements and the geometric information from the mock-up model, the data quality of every
component on the building’s point cloud is evaluated. The predicted errors are color-coded to
visually map the errors on the point cloud. Finally, based on the quality assessment of scan data, a
corrective data collection was performed to improve the quality of data from three case study
buildings. The results showed that the quality of data was improved by an average of 3 mm which
upgraded the 3D as-built models to the highest level of detail (LOD) defined by the U.S. general
services administration (GSA).
The findings from this research indicated that the in-situ quantitative quality assessment of scan
data enables the early identification and reduction of quality issues that could cause modeling
errors in the downstream. Capturing the geometric properties of buildings through mock-up 3D
models assisted the quality assessment by first, enabling the incorporation of building-specific
geometric properties into the quantitative impact analysis of quality issues, and second, providing
a baseline model to locate the building components on the collected scan data which enabled a
component-level quality assessment.

14
Chapter 2 MOTIVATION

The emergence of BIM in the past decade has revolutionized the AEC/FM industry by enabling
evident improvements in engineering practices encompassing design, construction, and operation
of buildings [2-4,21]. BIM has provided a platform for integrating the information required for
variety of building studies, such as structural analysis, sustainability evaluation, and energy
efficiency assessment [21,35,46,139]. Accurate documentation of as-built conditions for
generating precise BIM models is known to have beneficial economic impacts in lowering the
costs associated with construction, retrofit, and renovation processes [59,96]. Although generating
accurate BIM models is becoming a common practice within the AEC/FM industry for new
infrastructures, 85% of the existing buildings in the United States are in their operations phase and
have limited or no accurate updated BIM model [96,140]. The lack of such access to information
could result in misleading decision-making required for activities, such as renovation practices for
reducing energy consumption [32,62,63]. Therefore, it is imperative to produce accurate as-built
models of existing buildings in order to have continuous access to updated information regarding
building condition assessment and operation.
Laser scanning technology has been recognized as an accurate data acquisition method for
condition assessment of infrastructures and is currently used as the standard technique for
collecting data [25,55,59]. Although laser scanners collect data with high precision and accuracy,
the data only provides 3D coordinates of points (i.e. 3D point cloud) on the scanned building. In
order to transform the point cloud to a 3D model, the data have to be processed and augmented
with parametric geometric information. This process is recognized as a time-consuming, labor-
intensive, and often subjective process if carried out manually [44,131,147]. Recent studies have
focused on automating the as-built modeling process, however, to date, challenges remain with
accurate detection and extraction of building primitives from the scan data [100,131,136,147].
These challenges often stem from the early data acquisition stages and have compounded effects
on the final as-built model. A poor scan plan, erroneous data collection, and presence of occlusions
could lead to poor quality scan data, which amplifies the challenges with the modeling process
[20,44,90,91,114,118].
At present, the scan data collection is performed using a limited set of quality control guidelines
that are managed manually and are rarely based on the knowledge of data quality requirements for
processing and converting the 3D data to an accurate as-built model [1,59,73]. Therefore, several
data quality issues are not identified during the data acquisition stage and often remain uncovered
until weeks after data acquisition resulting in either misrepresentation of the building’s 3D model
or the need for revisiting the site to redo the scan [118,136]. In addition, there is no quantitative
information on the magnitude of errors that could be introduced to the as-built models as the result
of the data quality issues. Critical challenges with assessing the scan data quality are: need for
recognizing the data quality requirements that are central to generating accurate as-built models
from the scan data, and lack of a framework that employs such requirements and integrates them
with 3D modeling technics to automatically evaluate scan data quality without using any prior
BIM model of the building. The overarching goal of this dissertation is to advance our knowledge
about inherent scan quality issues and to enable detection of them early in the data acquisition
process so corrective actions could take place before the scan session is finished.

15
Chapter 3 SCOPE

This scope of this dissertation is narrowed down to the scan data quality requirements for 3D
modeling of building envelopes as they are scanned frequently for energy retrofit purposes [54].
Specifically, we try to derive a set of scan data quality requirements for accurately extracting and
modeling the walls and openings on the building’s façade. Note that the 3D recognition and
modeling of windows and doors are not in the scope of this dissertation. To the best of my
knowledge, there is currently no study on providing solutions for an automated assessment of scan
data quality in the 3D as-built modeling context. Therefore, this dissertation is a preliminary step
toward developing a comprehensive framework for data quality assessment of buildings, as well
as other types of infrastructures. The findings of this research are based on the study of residential
and educational buildings. The results could potentially be applied to the scan data of other types
of buildings where modeling the exterior walls and openings for generating as-built/as-is models
are sought.

16
Chapter 4 SCIENTIFIC SIGNIFICANCE AND IMPACT

The proposed research aims to add transparency to the underlying deficiencies that exist in scan
data by mining the information that is often missed or neglected due to the inexistence of an
automated quality assessment process. The added transparency could augment the semantics that
are currently missing in as-built models due to missing or inaccurate data. This research envisions
enabling an X-ray vision that quantitatively reveals the inherent quality issues with the scan data.
This will foster a new dimension of knowledge on 3D remote sensing, and pinpoints the quality
issues that are potential risks to accurate 3D as-built modeling. The findings of this research add
the data quality dimension to the remote 3D sensing by engaging computer vision techniques in
the preliminary stages of data collection. Additionally, the findings of this research could
potentially advance the science of 3D remote sensing data collection by providing a real-time
feedback system particularly for the cases when the data is collected in an automated proactive
process by unmanned vehicles. Consequently, we can design adaptive automated data collection
systems that would continuously readjust themselves based on the data quality feedback.
Accordingly, the reliability will be reinforced by collecting more accurate data on the as-built
conditions of infrastructures.
The results will challenge the existing 3D modeling algorithms by providing a new quality
benchmarking system. Researchers could benefit from this research by using the outcomes to
improve the current automated as-built modeling algorithms by evaluating their performance
against different levels of data quality. In addition, we can potentially build on the outcome of this
research by designing new 3D modeling algorithms that are focused on new challenges, leading
towards more accurate as-built models that are more time and cost efficient. Finally, the findings
of this study could be extended to other imaging sensors, such as digital and depth cameras by
assimilating the properties of these sensors.
The AEC/FM industry could benefit from this study by revising the current definition of scan data
quality and using the quantitative analyses that will be incorporated in this work to enforce a new
set of standards, guidelines, and policies for scanning infrastructures. One of the common uses of
documentation of as-built conditions of buildings are for energy retrofitting purposes. According
to [11], retrofitting existing buildings has a potential for significant economic and employment
impacts, estimated as $1 trillion of energy savings over 10 years and creation of 3.3 million
cumulative job years of employment. The first step in such improvements is as-built
documentation of buildings, and high-quality data is central to the accuracy of energy studies on
these buildings. I argue that with higher quality data collected from the as-built conditions of
infrastructure systems such as buildings, the effective control and operation of such facilities
would be facilitated. In addition, this research will provide a new basis for higher quality
engineering decision making in existing buildings during their lifecycle. The results of this study
could potentially be expanded to interior (e.g., rooms and furniture) and above-the-ceiling (e.g.,
ducts and mechanical, electrical, and plumbing (MEP) components) scan data quality assessment.
It can also be extrapolated to other types of infrastructure systems that require continuous condition
assessment leading to more efficient operations and maintenance of facilities.

17
Chapter 5 LITERATURE REVIEW AND PROBLEM STATEMENT

The application of the state-of-the-art 3D imaging tools, especially 3D laser scanners, for
documenting the as-built conditions of buildings have been extensively studied in the literature.
The relevant existing body of work could be grouped into four main categories based on the target
objectives of our work. The first category focuses on the identification of the data collection errors
that are related to either the laser scanner sensor, such as angular accuracy, random and systematic
errors, or the errors related to scanning conditions, such as survey control network, lighting and
weather conditions. The second category investigates different scan planning methods based on
the precision and accuracy of scanning sensor, as well as the target building’s geometric properties
to ensure high quality data is collected. The third category studies different computer vision
methods and optimization algorithms for automating the process of converting point cloud data to
3D as-built models. Finally, the fourth category, which is relatively less rich as oppose to the other
categories, studies the quality of the building models generated by the 3D as-built modeling
methods.
5.1 Identification of Scan Data Collection Errors
Previous work has shown that identification of scan data quality parameters improves accuracy,
cost efficiency, and time of data collection while it reduces potential conflicts with on-going
construction processes [24,115]. The first step toward improving the scan data quality is to identify
the sources of error that would affect the performance of the laser scanner sensor. Shen, et al. [115]
conducted a set of controlled experiments and statistical analysis to prioritize the data collection
parameters that influence the scan data quality. Their study showed that scan parameters, such as
the scanning resolution, the scanner’s distance from the target, and the geometric properties of the
target have significant effects on the scan data quality. Another recent study by Bosché, et al. [29]
identified data quality issues such as occlusions and point density to have significant effects on the
accuracy of automatically generated 3D models. Dimitrov and Golparvar-Fard [44] listed the
challenges with segmentation of point clouds which is an essential step in as-built modeling as:
variation in point density, roughness and curvature of objects’ surface, noise level of data,
existence of clutter and occlusion, missing and erroneous data especially in reflective objects, user-
made decisions in modeling, and scalability when processing large datasets.
There is another line of research that focuses on quality issues with registration accuracy of scan
datasets when data is captured from multiple angles around a facility. The registration accuracy is
usually determined by studying the overlapping strips of two consecutive scan data [61,74]. As the
misalignment of the scan datasets directly and linearly contributes to errors in the generated as-
built models, the registration error needs to be as small as possible [143]. Nonetheless, there is
limited studies investigating the influences of data quality issues such as low point cloud density
and noise on accuracy of as-built models.
Occlusions are considered as one of the key challenges in automating the process of as-built
modeling as they limit the line-of-sight of laser scanners and are frequently misinterpreted as part
of the scan target [147]. In addition, they often complicate the detection of boundaries of openings
for modeling different architectural elements in the buildings [96,136]. Reducing the impacts of
occlusions using progressive scans was investigated in the study by Gao, et al. [52]. A manual

18
qualitative visibility analysis was used by the authors to verify the validity of the hypothesis.
However, manual detection of occlusions is a time-consuming process especially for scanning a
large-size facility. Adan and Huber [13] presented a method to model interior surfaces of a facility
with existence of occlusions and clutter. The method used ray-tracing algorithm to detect
occlusions between the scanner and a specific surface. Using the ray-tracing algorithm, the
surfaces in the studied point cloud were labeled as occupied, occluded, or empty. However, ray-
tracing algorithm is computationally expensive when applied to a large-size data. Meanwhile, its
application is limited to the cases where real-time processing of the point cloud is not required. In
order to improve the computational efficiency of the occlusion detection, researchers proposed
methods using data structures, such as octree to store the point cloud’s spatial information for
faster identification of occlusions [33]. However, when working with a large-size data, such as the
point cloud of a building’s exterior, the ray-tracing would remain computationally expensive even
with the use of data structures. Therefore, there is a need for an automated occlusion detection
method that could identify the sources of occlusion in the time-frame of a scanning session so the
survey team could employ corrective actions to avoid the occlusions.
In general, even though some of the scan data quality issues are identified in the literature, no
solutions have been proposed for identifying, quantifying, and reducing the quality issues during
the data collection, and prior to using the collected data for generating as-built models. To
understand how the data from laser scanners could be impacted, Tang and Alaswad [129] derived
a set of equations for modeling laser scanner’s sensor. The equations were developed based on the
relationships between the data collection errors, sensor errors, and data quality metrics. The data
quality metrics were determined based on those defined by the U.S. GSA as requirements for 3D
imaging data deliverables submitted by their contractors. To improve the data collection practice
based on the derived sensor model, they used multiple optimization algorithms to find the scan
plan that covers the largest area, and meets the user-defined data quality requirements in a
minimum amount of time. Although partial improvements were observed using their scan plan, it
was only tested for a building with simple geometry and limited occlusion. In addition, the
collected data’s quality was only evaluated for specific areas of the building. However, in order to
generate an accurate as-built model, and to verify the GSA-defined data quality requirements were
met, the quality of data and the accuracy of the generated model need to be evaluated and validated
at building component-level (e.g., wall, window, door, etc.). Meanwhile, the data quality
requirements defined by the GSA are not linked with the expected errors from 3D modeling
algorithms. Moreover, there is currently no method to evaluate the data quality based on these
quality requirements. This research will address this gap by proposing a data quality assessment
method that uses information from the specific geometry of the building to enable the automated
study of data quality at component-level, and based on the expected errors of 3D modeling
algorithms.
5.2 Improving the Data Quality by Scan Planning
In order to reduce the scan data quality issues, researchers incorporate the sources of scanning
error, and data quality requirements into scan planning methods. Scan plan is usually defined by
determining the optimal laser scanner’s configuration and positioning (i.e. location and
orientation) with respect to the data quality requirements, such as minimum point spacing,
incidence angle, overlap ratio, etc. [14,36]. There are currently two popular methods for scan
planning: view-planning and model-based planning.

19
The view-planning method which works based on visibility-analysis, customizes scan parameters,
such as scanner’s distance from the target, incidence angle, and field of view to establish a time-
efficient data collection that covers the entire building. However, view-planning methods do not
guarantee sufficient quality across the data. Recent studies [93,117,150], added the 2D and 3D
geometric information of the building into the scan planning (model-based) resulting in data with
higher quality. A 2D model-based scan plan uses the footprint (x and y coordinates) as well as
height information on the regions of interest (ROI) of the building and implements an iterative
heuristic approach to guide a complete data collection with satisfactory quality [14,36]. Zhang, et
al. [150] determined scan positions and their corresponding angular resolutions by satisfying data
quality requirements on select 3D ROIs across a building target. Their findings showed that
establishing the scan plan based on the 3D geometric information of the building as well as
incorporating the relationships of scan settings and data quality requirements resulted in data with
better quality comparing to former scan planning methods. Nonetheless, the dynamic nature of
built environments, as well as unforeseen conditions such as site occlusions, weather conditions,
and human-errors cannot be fully captured in a scan plan. Therefore, there is a need to evaluate the
quality of data after the data collection is completed.
5.3 Identifying the Recent Automated 3D As-built Modeling Methods
The main steps in a typical as-built modeling process involve using different 3D modeling, feature
extraction, and 3D object recognition methods to transform the raw scan data into a 3D as-built
model [131,147]. The 3D data processing usually starts with discretizing the point cloud and
segmenting it into cluster of points that are geometrically similar using region growing
segmentation methods [87]. The 3D modeling methods include algorithms for fitting predefined
geometric primitives such as planes, spheres, and cylinders to the scan data in order to extract basic
geometries such as walls and columns [131]. Feature extraction methods such as Hough transform
are commonly used to extract vertical and horizontal line features from 2D edge maps of
architectural elements and opening boundaries [54,147]. Lastly, various object recognition
methods are used at the final stages of the as-built modeling process to identify the architectural
element instances in the scan data [131].
Pu and Vosselman [100] proposed an automated method for reconstruction of building exterior
elements, such as walls, windows, and doors using TLS data. They used hard-coded semantics
from the topological relationship between the building elements along with a polygon-based least
squares fitting method to model the building’s exterior. Their study showed that the modeling
accuracy depends on the level of occlusions and the point density of the scan data. In order to fill-
in the occluded parts of the building’s exterior, similar to other recent studies [147], the authors
made assumptions based on the generic knowledge of architectural elements’ layout pattern.
However, if the occlusions and other data quality issues would be identified during the data
collection stage, such assumptions that lead to inaccurate as-built models could be improved by
better reasoning. There are recent studies [54,136,147] that have used context-based methods for
extracting indoor building elements including walls, ceilings, and floors. The methods involve
voxelization of point clouds and labeling patches of points using the contextual relationships
between them. Machine learning methods are then used to train classifiers for detecting openings
on the walls. Although an exhaustive training could improve the accuracy of classifiers, their

20
performance is still affected by data quality issues such as noise, missing data, and low point
density.
A further detailed modeling includes recognition of architectural elements in the buildings. The
very first step in an object recognition process is to find distinctive features (3D shape descriptors)
either on particular locations (key points) or on the entire input object. The former features are
known as local descriptors and the latter are known as global descriptors. Since local descriptors
are defined only on specific parts of a given object, they tend to perform better than global
descriptors in the cases of occlusions, viewpoint changes, and articulation. Moreover, they are
invariant to translation and rotation. In addition, vector quantization techniques could be used to
create codebooks of the descriptors for accelerating feature matching. On the other hand, object
recognition methods that use global descriptors usually perform better when there are minimal
occlusions. Aldoma, et al. [16] carried out a detailed study on evaluating the performances of
several 3D shape descriptors for object recognition. They used sets of household objects, as well
as objects from retailers such as IKEA that are publicly available. Their study showed that object
recognition methods that used SHOT (Signature of Histograms of OrienTations), FPFH (Fast Point
Feature Histogram), and USC (Unique Shape Context) local descriptors were able to recognize
objects with higher detection rates compared to other 3D shape descriptors.
Generally, the literature is rich in designing 3D modeling methods for converting point clouds to
as-built models with geometric and parametric semantics. Nonetheless, the relationship of these
methods and the scan data quality is not considered in the data collection practices.
5.4 Quality Assessment of Generated As-built Models
The quality of collected scan data is currently controlled heuristically by scanning experts. These
processes are often erroneous especially due to the fact that scan data are usually large and complex
for a naked eye to comprehend [14,111]. Recently, a physical measurement procedure was
proposed by the investigators from the National Institute of Standards and Technology (NIST) and
Carnegie Mellon University [37] for the assessment of the quality of the final deliverables
produced by 3D imaging data. The method involved manual in-situ measurement of architectural
features in a facility (e.g., doors, windows) and comparison of them with measurements derived
from 2D/3D building plans generated from 3D imaging data. The acceptance of the generated data
was based on two main criteria: (1) the absolute value of the difference between the NIST/CMU
measurements and the generated 2D/3D models measurements; and (2) the statistical uncertainty
of the NIST/CMU measurements. As manual control of data quality for every single object in the
scan data is a tedious and erroneous task, there is a need for automated assessment of scan data
quality against quality requirements for generating accurate as-built models. In addition, one
should note that the geometry and appearance of an object also influence the data quality [14].
Therefore, the impacts of quality issues on the scan data of a building that has walls with varying
heights and orientations, and openings with different types and shapes, should not be regarded the
same as a building with uniform walls and openings. To incorporate the specific geometric
properties of buildings in the scan data quality assessment, there is a need to collect the geometric
information of buildings through a timely-efficient method.
There are examples of research in the literature [20,118] that use a pre-existing baseline 3D model
to evaluate the quality of the scan data. However, they are limited in the sense of being dependent

21
on the availability of a baseline 3D model. Also, they do not provide any quantification method
for measuring the impacts of different quality issues on the accuracy of as-built models. Therefore,
there is a need to establish a link between the 3D modeling methods and data quality issues. Several
researchers have used the as-designed BIM model of buildings for evaluating the quality of the
collected 3D point clouds or the automatically generated 3D models. Feature-based matching
approaches are often used for pairing as-built point clouds with as-designed BIM models in order
to identify their discrepancies [53]. The closest study to this research is the one by Anil, et al. [20]
where the quality of the regenerated as-built model from a plant facility was evaluated by
overlaying the collected scan data and the as-built model. Discrepancies of the two models were
computed through a deviation analysis and a color-coding scheme. Deviation analysis was used to
reveal the as-built modeling errors accurately yet faster than performing manual measurements.
However, the proposed deviation analysis could not be used to evaluate the quality of the scan data
of a building that does not have a pre-existing updated BIM model. Moreover, there is a need to
extend the deviation analysis by studying the as-built modeling errors under various conditions of
scan data quality.
In general, to the best of my knowledge, there has been no study on investigating the in-situ scan
data quality assessment for assisting the scan specialists by identifying and quantifying the quality
issues and their potential impacts on the accuracy of as-built models. The current industry practice
is to evaluate the quality of data by manually comparing physical measurements from facilities
with the measurements from the data. This process is subjective, erroneous, and limited to few
manual measurements especially when dealing with large-size data. To avoid subjective manual
judgments regarding the scan quality, an automated quality assessment method is required to
quantitatively evaluate the quality of the laser scan data for every building component (i.e. walls
and openings).

22
Chapter 6 RESEARCH GOAL, OBJECTIVES, AND QUESTIONS

The overarching goal of this research is to ensure the scan data is adequate for accurate extracting
and modeling architectural elements of buildings for generating as-built/as-is models. This
research introduces a systematic approach to (1) identify factors that influence the scan data
quality, (2) quantify their impacts on generating accurate 3D models, (3) derive data quality
requirements based on the impact analysis, and (4) use the data quality requirements to predict and
analyze the quality of scan data. To tackle the defined research goal, the following research
objectives and questions are formulated and addressed in this dissertation:
Objective 1: Identification of the data quality requirements for scanning small to medium size
buildings’ exterior with an aim to use the data for generating accurate as-built models.
• Question 1-1: What are the scan setting parameters (e.g., angular resolution, field of view)
and scan data characteristics (e.g., point cloud density, noise, data completeness) that
influence scan data quality for accurately extracting and modeling architectural elements
of buildings for as-built modeling process?
• Question 1-2: How do different data quality issues impact the performance (i.e. accuracy)
of 3D modeling, and feature extraction methods that are prevalently used in the building
geometric modeling processes?
• Question 1-3: What data quality issues could be controlled during the scan session (in-
situ)?
• Question 1-4: How to define metrics for evaluation of scan data quality?

Objective 2: Design a framework for in-situ identification and quantification of scan data quality
issues related to extracting and modeling architectural elements of buildings’ exterior for as-built
modeling process.
• Question 2-1: If needed, how to integrate other data sources (e.g., image data, and mock-
up models) and the point cloud data to estimate the location of different architectural
elements on the exteriors of buildings with an aim to evaluate data quality issue for each
element?
• Question 2-2: How to identify data quality issues from scan data during the scanning
session using computational methods?
• Question 2-3: How to classify the identified issues with an aim to recognize their sources?

23
Chapter 7 IDENTIFICATION OF SCAN DATA QUALITY FACTORS

There are several number of studies that have investigated the scan data errors caused by sources
such as scanner’s sensor, scanning procedure, or properties of the object’s surface that is being
scanned [24,28,119,120]. Most of the studies have focused mainly on the accuracy of the
measurements derived from a scan data and have paid less attention to issues such as missing data
(lack of coverage), low resolution scan (low point density), etc. However, in the processes such as
3D modeling, issues such as low point density could make the as-built model inaccurate and
erroneous even if some parts of the scanned object’s dimensions are captured accurately [44]. In
addition, there is currently no study on investigating how scan data quality issues resulted from
variations in the scan parameters could affect the accuracy of the 3D models generated from the
as-built modeling process.
As the first research activity, a total of 22 references were studied to derive a set of data quality
factors that are frequently evaluated by the researchers when processing laser scan data.
Additionally, the objective was to understand which scan parameters are associated with these
quality factors. The literature review included studying the recent research studies, white papers
from the industry, and the discussions from online LiDAR forums. We narrowed down the derived
quality factors to nine main categories with respect to the number of times they were targeted in
the references. Note that in the narrow-down process, the quality factors that were studied less than
three times were omitted. The nine quality factors and their definition based on the reviewed
literature are presented in Table 1. In addition, a word frequency diagram was created based on
the number of references that have focused on each quality factor (Figure 1).

Table 1. Scan Data Quality Factors derived from the literature
Scan Data Quality Factor Definition
Resolution The point cloud density/the point spacing in the scan data
Coverage
Completeness of the scan data: no shadow or obscured
zones
Resolution Consistency Uniform resolution across the scan data
Registration Accuracy
Consistency
Uniform registration accuracy across the scan data
Clarity Visual crispness with maximum data retention
Manageability Scan data size, transfer and process speed
Color Quality Alignment of captured images with the point cloud
Artifact’s Measurement
Accuracy
Difference between actual and measured dimensions
Artifact’s Positional Uncertainty Difference between actual and measured position

24

Figure 1. Word frequency diagram of the identified quality factors based on
[12,19,24,28,38,48,50,58,59,67,70,73,88,97,99,114,119,120,128-130,144]
As seen in Figure 1, the data quality factors of 1) Measurement accuracy, 2) Registration accuracy
consistency, 3) Resolution, and 4) Coverage are remarked with a higher frequency comparing to
the other data quality factors. This higher frequency could be attributed to their importance in
terms of the final product’s quality (e.g. generating 3D as-built models, performing condition
assessment for infrastructures, etc.), as well as the feasibility of evaluation comparing to other
quality factors. For instance, evaluating the measurement accuracy could be done using the
dimensions of a ground-truth artifact whereas evaluating the color quality requires controlling
many parameters such as the calibration of camera and scanner’s sensors, registration accuracy,
etc. Nonetheless, the above quality factors could collectively define how a desired scan data is
expected to be. In addition, there are currently no sets of quantitative criteria attributed to these
quality factors particularly for using a scan data for the as-built modeling process. Note that the
importance of these factors changes depending on the purpose for which the scan data is used for.
For instance, positional uncertainty may be important for the condition assessment of highway
infrastructure’s features, however, it might be less important for creating an as-built model if the
positions of building components relative to each other remain accurate. Also, according to the
literature, there is not a consensus on whether the resolution consistency represents a high-quality
scan data. This is because a scan data might have a consistent resolution, yet not high enough for
accurate extraction and modeling of building components. On the other hand, capturing a scan
with a uniform resolution might not be optimal in terms of time as the point density requirements
might vary across the scanned project [91,99].
In addition to the literature review, a set of open-ended questions were designed (Appendix 1) for
the phone interviews to (1) investigate how LiDAR specialists define the scan data quality, and (2)
explore the effects of quality issues on the as-built modeling processes in real-world projects. The
questions were designed in a way to help understand the existing gaps and issues in the as-built

25
modeling processes that are specifically related to issues with the scan quality. Additionally, the
questions seek to investigate what procedures are currently employed in the industry to reduce the
scan quality issues. Note that the questions were designed around the quality factors derived from
the literature. A total of eleven interviews were conducted for this purpose. In addition to the
questionnaire (Appendix 1), the interviewees were asked to rate the importance of the identified
quality factors. The rating was defined based on the scale of 1 to 10, one being not important and
ten being extremely important. Note that the focus of the interviews was primarily on the scan data
of buildings. We also asked the interviewees to name the scan parameters that each quality factor
depends on. We selected interviewees who had at least three years of experience in working with
LiDAR technologies. Nine of the participators were from the industry from which three were
actively involved in the as-built modeling area. The other two participators were from the academia
with specialization in LiDAR and computer vision research. Figure 2 presents the data quality
factors’ ratings based on the discussions with the interviewees. Note that, the ratings are almost in
agreement with the word frequency diagram derived from the literature, however, the factors of
positional uncertainty and manageability are also considered as extremely important. Since most
of the interviewees were coming from the industry, the issue of scan data manageability was more
tangible to them as they had to deal with large sizes of scan data on a daily basis.

Figure 2. The importance level of the scan data quality factors according to the interviews with
LiDAR specialists
The diagrams presented below summarize the responses on (1) the scan parameters associated with
each data quality factor, and (2) the preventive/corrective actions currently employed to ensure the
corresponding quality factor meets the project requirements. Note that the percentages are
calculated based on the number of interviewees who have pointed out the corresponding items.

26

27

From the identified scan data quality factors, only a few of them could be controlled during the
scanning session. As seen in the diagrams, most of the quality factors are currently controlled
manually or by basic analysis plugins provided by off-the-shelf point cloud processing software.
The plugins can perform analysis such as determining the local point density of the scan data,
providing measurements/distances between two selected points, and the scan registration accuracy.
In addition, the coverage of the data could be analyzed based on finding missing data due to
occlusions. However, such analyses are computationally expensive as they require ray-tracing for
thousands and often millions of points in the scan data. In-situ assessment of quality factors such
as positional uncertainty and measurement accuracy is more challenging since it requires manual
measurements and is timely expensive to be performed for the entire scan data. Nonetheless, the
level of quality issues such as noise that result in inaccurate position and measurement estimations
could be estimated during the scanning session rather timely and automated.
Additionally, we used the findings from the above diagrams to identify the relationships between
different scan parameters. The objective was to identify the sets of scan parameters that contribute
to the common scan data quality factors. One should note that there is a large uncertainty with

28
identifying the quantitative level of dependency/relationship between these scan parameters as
each data quality factor is affected by some undetermined and combined level of influence from
these scan parameters. Nonetheless, to visually present the relationship between different scan
parameters, we selected those that contribute to at least three different data quality factors. Figure
3 shows four scan parameters and the data quality factors that they are related to. Note that the
bond’s strength (connector’s thickness) between these scan parameters and the data quality factors
are defined based on the findings derived from the above diagrams. The two-way arrows connect
the similar data quality factors from two scan parameters. As seen, the scan parameters “scan
procedure” and “scanner’s calibration” have three data quality factors in common. This
commonality could be weighted based on the total number of the data quality factors that each
scan parameter is related to. For instance, 3/3 of the scan procedure’s related quality factors are
common with 3/4 of those from the scanner’s calibration. This indicates a strong relationship
between these two scan parameters. Note that quantifying the relationships between the scan
parameters is out of the scope of this research. Nonetheless, by using this visualization, one can
tell that there is some level of relationship between these two scan parameters.
A future work could include increasing the number of interview participants to a reliable number
to use the data quality factors ratings along with the scan parameters percentages for creating a set
of prior probabilities to design a Bayesian network for the scan parameters. A set of posterior
probabilities could be derived using the results from a set of controlled experiments that would
study the individual and combined effects of different scan parameters on different data quality
factors. In addition, the open-ended questions that were used in the conducted interviews may lead
to design of an online questionnaire. The questionnaire could be distributed to a larger number of
LiDAR specialists to attain a deeper knowledge about the different scan parameters, as well as the
data quality factors and requirements.

29

Figure 3. Relationships of scan parameters and data quality factors (Example of four scan
parameters being related to with at least three quality factors).

30
Chapter 8 A DATA QUALITY-DRIVEN FRAMEWORK FOR ASSET
CONDITION ASSESSMENT USING LIDAR AND IMAGE DATA

This chapter presents a study which focused specifically on relating the data quality factors to scan
data collection techniques. In this study, we investigated the hypothesis that the scan data quality
could be improved if a scan plan incorporates the geometric properties of a building, and considers
the quality requirements specified by the project requirements. We propose a general data-quality
driven scan planning framework for condition assessment of infrastructures. We used
photogrammetry methods to collect 3D geometric properties of buildings using images captured
by a drone. The framework is examined by developing a scan plan for collecting data from a
historical building. Our hypothesis was validated as the proposed framework resulted in an
improved data quality. The following chapters are inspired by the findings of this chapter which
emphasizes the necessity to incorporate the geometric properties of buildings when collecting and
processing the scan data.
8.1 Existing Scan Planning Methods
In order to have an accurate 3D point cloud, and to minimize the errors in generated as-built
models, it is essential to have an accurate scan plan, which takes all parameters that could affect a
scan’s accuracy into account. Most of the scan planning studies focus on analyzing the visibility
requirements and do not consider the data quality requirements (e.g. data density and tolerance).
The use of a uniform scan plan can result in redundant LODs in parts of a point cloud and lack of
the required details in other parts. A scan plan should also consider the project specific LOD
requirements, which may vary in a project. The LOD requirements of a geometrical feature of a
project could be initially defined based on the asset condition assessment (ACA) goals. For
instance, an ACA goal for a bridge could be the inspection of concrete columns for crack detection.
In order to monitor the columns using a laser scanner, point clouds should provide detailed data
points on the columns so the data processor could analyze the severity of cracks or damages. If the
rest of the bridge is scanned with settings similar to the columns, a scan plan based solely on the
visibility requirements could potentially result in a time consuming and costly scan process.
Recently, researchers have developed new scan planning approaches focusing on scan data quality.
Pradhan and Moon [98] introduced a simulation-based framework to identify scan locations for
better capturing the critical components of a bridge such as piers and girders. Based on their
findings, capturing geometrical measurements of certain parts of a structure is more critical than
the others since different portions of structures have different levels of importance in terms of
performance/response metrics. Song, et al. [118] proposed a novel approach for scan planning,
which integrated 3D data quality analysis and clustering methods to group the geometrical features
based on their LOD requirements. Their study showed that the automatic scan planning algorithm
results in a denser point cloud without the need to increase the data collection time. Also, their
study is based on the assumption that a BIM model of the project is available, however, this is not
always the case, especially when the as-is condition of an infrastructure is different than the
archived as-built/designed models. Moreover, the line-of-sight and portability limitations of
terrestrial laser scanners have to be considered in the scan plan. For instance, a scan plan has to
provide a solution for capturing the features that are located in the blind spots of a laser scanner.
Therefore, a hybrid scan plan including different LiDAR equipment such as long-range scanners

31
and aerial imaging sensors may be required to scan infrastructure systems with accessibility
limitations.
8.2 Data Quality-driven Scan Planning Framework
To define a high-quality scan plan, there is a need for a holistic approach that takes into account
project specific characteristics and data quality requirements. The framework proposed in this
chapter was designed to enable project stakeholders to define an integrated scan plan, which is
centered on data quality requirements. (Figure 4). The input data of the proposed framework is
comprised of every project’s specific information. The main information that drives the scan plan
is realization of the condition assessment goals. The LOD requirements for different geometrical
features of the project are defined by project stakeholders based on these goals. LOD requirements
are usually not constant and may change throughout the project. The next input data to be derived
is the list of all available LiDAR sensors and their parameters. This information is directly tied to
the project’s constraints, such as time and budget, weather conditions, accessibility, etc. For
instance, in order to overcome the project’s accessibility constraints, an alternative solution would
be using long range terrestrial laser scanners, aerial LiDAR sensors, or UAV imaging. The last
input data for the framework is the 3D model of the project (if available). We propose using an
image-based 3D model for planning the scan when the updated BIM model is not available.

Figure 4. An overview of data quality-driven scan planning framework
Image-based 3D models can provide an acceptable representation of a project’s layout. However,
compared to 3D laser scanners, they might not offer an accurate data for detailed analysis required
for an ACA. Nonetheless, comparing to other remote sensing technologies and with respect to their
low cost, UAVs have less mobility constraints and therefore, they can complement the scanning

32
of a project by covering the laser scanners’ blind spots. It is noteworthy to mention that using UAV
for image collection has its own challenges. For instance, UAV’s flight trajectory and elevation
have to be determined prior to flight as they can affect the quality of collected images.
Once the image collection process is completed, a 3D model is reconstructed using the structure
from motion (SfM) technique. In order for the 3D model to be used for scan planning, it has to be
augmented with semantics about FOI’s geometrical information in the project. Hence, two
principal information are extracted from the 3D model: 1) the project’s layout that includes the
boundary coordinates which is then integrated with the project’s accessibility constraints
information, 2) FOI (points, planes, or 3D objects in general) geometrical and RGB data. The
project’s layout information assists the scanning crew to identify the feasible areas for scanner
circulation and establishing scan positions. Using the extracted information, the features are
classified based on their locations, orientations in space, and the LOD requirements. The
classification criteria are selected based on their direct relationships with scanner’s position and
data quality. The interactions of features classification criteria and sensor parameters (range,
accuracy, and angular resolution) pinpoint the best scan position. In the next step, the features are
prioritized based on their classes and a simulation-based decision-making process determines the
best scan position across different sides of the project. The output of the decision-making process
is a scan scenario, which is then evaluated by a sensor simulation software (i.e., Blensor). The
sensor simulation output enables evaluation of the proposed scan scenario by measuring the data
quality for different features. If the data quality is not satisfying, the decision-making process is
repeated with the new information using the sensor simulation output. The other module of the
proposed framework focuses on integrating LiDAR and UAV-based imaging data to generate a
single coherent 3D point cloud, in which all the blind spots of the target are covered. Due to the
fact that point clouds from different sensors might have non-equal spatial resolution, precision,
and accuracy, their registration process is challenging. In the case of having point clouds from
multiple sensors, an iterative registration process using multiple common features/control points
could improve the accuracy of final point cloud. Once the registration process is completed, if
there are missing data in the final point cloud, there will be a need to augment the 3D model by
extrapolating the missing data using their surrounding points’ coordinates and RGB values.
8.3 Case Study
In order to provide a preliminary evaluation of the proposed framework, we selected the Mudd
Hall building’s courtyard located on University of Southern California campus. The Mudd Hall
building has been named as one of the Historical Cultural Monuments by the Los Angeles City
Council and the fact that it is a feature rich building makes it a suitable case study for the purpose
of this research. Figure 5b shows some of the architectural features on the building’s courtyard.

8.3.1 UAV Image Collection
3D scene reconstruction using image sequences has been a growing field of inquiry across multiple
disciplines such as cultural heritage preservation (Stanco et al. 2011), archaeology [71], and
construction [56,62,104]. Generating 3D models from 2D images follow the SfM technique which
includes: 1) feature extraction and description using Scale-invariant Feature Transform (SIFT)
algorithm, 2) pairwise matching of images using SIFT descriptors, 3) estimation of motion and
structure using the matched images, 4) refining the estimates using Bundle Adjustment, and 5)
creating surface meshes using image-based triangulation.

33
We used a DJI Phantom 2 Vision Plus drone to take images from the case study building. The
drone comes with a 14 Megapixel built-in camera, which is mounted on a gimbal, making it stable
during its flight. The camera has a large field of view (FOV = 125°), therefore the images are
highly distorted. We selected the lowest available FOV option (85°), even then the collected
images were slightly distorted. We then calibrated the camera to circumvent the distortion effect
and to rectify the images. We installed 10 ground control points (GCP) and scanned them with a
Leica TS06-plus Total Station to be able to geo-reference the 3D reconstructed model (Figure 5a).
A total of 236 images were taken from the building’s courtyard using a drone.

8.3.2 Image-based 3D Reconstruction
We used commercially available and open source tools, such as VisualSfM [146], Autodesk Recap
360, and Agisoft Photoscan Pro to generate a dense point cloud and a 3D mesh of the courtyard.
After visual inspection of the reconstructed models, we decided to use Agisoft’s output as it
provided a denser point cloud comparing to VisualSfM and since Autodesk Recap only provides
3D mesh. Note that the selection of the software tools might have effects on the results; however,
optimum selection of the tools will be part of the future directions of this research. We then geo-
referenced the model by assigning surveyed coordinates to GCP on the point cloud. The 3D
reconstruction process was completed in 155 mins using a Microsoft© Windows workstation
laptop with Intel® Core i7 processor, 32 GB RAM memory, and NVIDIA© K2000M graphic card.

Figure 5. Case Study-Mudd Hall Courtyard
8.4 Data Processing and Results

8.4.1 Geometrical Features Recognition
Following the steps in our proposed framework, we used the images and corresponding point cloud
to detect, extract, and study the FOIs. For the purpose of this study, the west bound of the courtyard
was examined, where four large windows were selected as the FOIs (Figure 5). The authors used
Haar-like features to identify the FOIs from the images. Previous research has shown that
improved Haar-like features yields less rate of False Positives (FP) and False Negatives (FN), as
oppose to other prevalent image-based object detection methods, such as HOG and etc. (Zhang et
al. 2013). Haar-like features were introduced by Viola and Jones [138] who proposed convolving
images with Haar wavelet sequences. Haar-like features are found by calculating the difference
between sums of pixel intensities in surrounding rectangles of a location in an image. The Haar-
like classifier slides a window over an image and looks for objects of interest that were previously
used for training the classifier. Considering the fact that using sums of pixel intensities as the only
a. Large Windows (FOI) b. Architectural features
c. 3D reconstructed model

34
feature would result in a weak learner, a cascade classifier, which includes a large number of Haar-
like features is used in the Viola and Jones framework.
We ran pilot tests to determine the optimum input parameters for training Haar-like cascade
classifier. The parameters were determined based on the number of pictures and the quality of
them: number of training stages = 4, sliding window size = 128*128, false positive rate = 0.2, and
true positive rate = 0.99. We trained the cascade classifier using different numbers of positive
images (images that only include objects of interest with different poses (position/orientation)) to
evaluate the performance of the Haar-like object detector. Note that we selected a low FP rate due
to the low number of positive images. Table 1a shows the results for different number of training
sizes. The precision, recall, and accuracy are computed based on
𝑇𝑃
𝑇𝑃 +𝐹𝑃

,

𝑇𝑃
𝑇𝑃 +𝐹𝑁

,

𝑇𝑃 +𝑇𝑁
𝑇𝑃 +𝑇𝑁 +𝐹𝑃 +𝐹𝑁
,
respectively. Note that the maximum desired number of true positives in each image was four
(there are only four large windows in the courtyard), which is a low number and therefore has
resulted in large jumps in precision and recall values. Once the FOIs were detected in the images,
the corresponding points should be localized in the point cloud. A reverse engineering method
using the SfM principles could be employed to address the localization problem. Figure 6
illustrates different steps to match 2D pixels on the images to 3D points on the point cloud. The
images were previously pair-wised by matching similar SIFT features during the SfM process and
a tree data structure was made for the pair-wised images. Therefore, a tree-based search can
identify co-visible images that contain a particular FOI. Using the previously estimated camera
poses of the co-visible images, fiducial planes (G1 and G2) of the two viewpoints along with the
rotation matrix (R1,2) are computed and used to localize the detected object in the 3D model. For
this preliminary study, we manually localized the FOI in the point cloud to ensure the accuracy of
the feature classification and the following steps.

Table 2. Results and LOD definition

a) Haar-Like Results b) Scan Results c) LOD Definition

8.4.2 Geometrical Features Classification
In this step, the features (windows including their trimming) were classified based on their
location, orientation, and LOD requirements. The process of identifying FOI’s orientation begins
with estimating the feature’s surface normals. Two methods could be used for this purpose. The
first one is generating a mesh from the point cloud and estimating the triangular planes’ normal.
The second method, which is more accurate due to studying points rather than approximated
planes, is estimating the normal to a point by solving a least-square plane fitting problem. The
overall steps for the latter method are: getting the k nearest neighbors of the point P on the point
cloud, computing the surface normal, and finally determining the sign of the normal. Determining
the sign of a surface normal in point clouds that are created from multiple views is a complicated
problem, however, it can be simplified by determining the correct normal sign based on the

35
neighbor surfaces’ normals. The surface normals of the points on the FOIs were computed using
the second method and the dominating normal was used to represent the orientation of the feature
with respect to the point cloud.

Figure 6. Steps for Localization of Features of Interest in 3D Point Cloud

8.4.3 Laser Scanner Position Determination
We propose defining the new notion of Pose-Quality (PQ) vector to represent 3D objects. In order
to decrease the computational complexity to determine the best scan position, we simplify the
studying of 3D objects by representing them as a PQ vector. The PQ vector’s size represents the
LOD for the feature, the coordinates are defined based on the orientation of the feature, and are
located on the center of a feature. Our current method computes the resultant of all the PQ vectors
for every bound of the scan target to determine the best scanning position for the laser scanner.
The optimal number of scan positions for capturing the required detail for FOI is computed, using
the overlap between identified scan positions from different bounds, through an iterative
simulation process. The PQ vectors for all the features on the west bound of the courtyard (north
side in Figure 7) were computed and the resultant was derived. The west bound of the courtyard
has four large windows (the case study’s FOIs) and a door on the right corner. Standardized LOD
requirements were acquired from US-GSA [135] (Table 2.c). According to US GSA, historical
documentation of buildings requires LOD 1 or 2. We assigned LOD 1 requirement to the windows.
Also, in order to have different data quality requirements within the west bound, we assumed
LOD 2 requirement for the door. Figure 7 shows the determined scan position using the proposed
framework. Finally, we scanned the courtyard using the determined scan position with a RIEGL
VZ-400 terrestrial laser scanner. We set the vertical and horizontal angular resolutions to 0.06
(0.01 m * 0.01 m * 100 m), which is known as a medium resolution setting. The lengths of the
windows were manually measured using a Total Station and were used as the ground truth data.
We then measured the same length using the scan data to evaluate the results (Table 2b). In
addition, we took point cloud samples of (50 mm * 50 mm) from the four windows to verify the
required data density is met. The results indicated that the scan data successfully satisfied the LOD
1 requirements for all the windows. As can be seen in Table 2c, the resolution and accuracy have
decreased for the windows that are farther and have greater incidence angle.

36

Figure 7. Determination of Scanner Position based on Features Orientations

8.5 Conclusion
This chapter presented a data quality-driven scan planning framework and a pilot case study for
preliminary evaluation of the framework. We did not study all the existing geometrical features of
the case study and the laser scanner position was determined through a semi-automated approach
based on limited parameters. In addition, the reconstructed point cloud had some missing data due
to the distortion in the images. A future research direction could focus on: 1) improving the
accuracy of 3D reconstructed model using the UAV sequence images as well as exploring
alternative imaging methods such as stereo vision camera systems, 2) creating an ontology of all
parameters that have to be considered for a data quality-driven scan planning, 3) studying the
interactions between different scan planning parameters, 4) evaluating existing image-based and
3D point cloud-based feature detection techniques for extracting FOI, and 5) proposing an
optimization algorithm to define the scan plan based on the feature map of the project.

37
Chapter 9 AUTOMATED MEASUREMENT OF HIGHWAY RETAINING
WALL DISPLACEMENTS USING TERRESTRIAL LASER SCANNERS

Previous chapter showed that having information on the geometric properties of facilities could
assist the data collection in improving the scan data quality. This chapter presents a study that
focused on scan data processing of highway retaining walls. We studied a set of scan data from
highway retaining walls for detecting the walls’ displacement throughout the time. Since there was
limited access to real-life data with various geometric and quality conditions, we tested our
processing method on synthetic data that was generated from the 3D models of the walls. The
study provides a detail analysis on how the combination of geometric properties and quality issues
from different scan parameters could impact the accuracy of processing the wall scan data.
Additionally, the findings verified that the synthetic data adequately represents the real-life
conditions.
9.1 MSE Walls
Wall displacements often challenge construction and maintenance of highway retaining walls.
Excessive movements of walls could result in damages, such as cracks on a wall’s facade, cracks
on highway pavement, and breakage of facing panels. Mechanically Stabilized Earth (MSE) walls,
since their introduction in 1970s, have been prevalently used as a promising form of highway
retaining wall to stabilize steep slopes and to retain the soil below the freeway alignment where
right-of-way constraints preclude the use of a sloped embankment. Their rapid, easy, and low-cost
construction, as well as their capability in tolerating significant differential settlements compared
to conventional concrete abutments have made them popular in the United States. Figure 8 shows
the cross section of a typical MSE wall.
Wall facing panels are used to hold the soil in position at the face of a wall. These panels are
interlocked with soil reinforcements. The joints between panels are usually filled with a geotextile
filter fabric to allow excessive water to exit and to prevent erosion of the soil through the joints
[95]. The facing material influences settlement and the lateral displacement tolerance. Precast
concrete panels are the most prevalent type of facing, not only due to their low cost and ease of
installation, but also because they allow for aesthetically pleasing finishes. The panels have a
minimum thickness of 140 mm and are in forms of cruciform, square, rectangular, diamond, or
hexagonal geometry [8].

38

Figure 8.Cross section profile of a typical MSE wall (Adapted from U.S. Department of Transportation
[57])
Stability of an MSE wall structure is inspected primarily from two perspectives of internal stability
and external stability. External stability of a wall depends on the location of the project and the
soil on which the wall is constructed. Vertical displacements (settlements) of a wall are attributed
mainly to the external stability and are due to the consolidation of the soil beneath the wall.
Consolidation of the soil depends on the soil type and level of moisture. Large settlements are
expected at the beginning of a wall’s construction since the soil experiences a new load, resulting
in immediate consolidation. MSE walls can handle significantly more settlements compared to
cast-in-place concrete walls. In addition, they can usually tolerate up to 1% differential settlement
(100 mm settlement in 10 m for a square facing panel with joint width of 20 mm) [5]. MSE walls
are monitored upon installation to ensure the settlement is finalized so that the subsequent
construction activities can take place. For instance, items built on top of MSE walls, such as
moment slabs, traffic barriers, and pavements, are not tolerant of settlement. Therefore, their
construction takes place after the wall’s settlement is over.
Internal stability is influenced by the select backfill’s stability, excessive dynamic loads on the
pavement and wall’s crest, such as moving traffic or earthquakes. Horizontal movements (lateral
displacements) of the wall are attributed essentially to the internal stability and they are due to the
pullout of the soil reinforcement. Lateral displacement of soil results in sliding, over-turning, and
eccentricity of an MSE wall [9,18]. These movements are generally observed as the wall is built.
Limiting the lateral displacements to less than 50 mm prevents unacceptable settlements and
damage of the surrounding structures and transportation infrastructure, such as pavements [6].
Displacements greater than the above-mentioned thresholds could potentially result in critical
damages that are detectable by visual inspection.

39
Various geotechnical tests, instrumentations, and surveying equipment have been used to study
the stability and performance of MSE walls [66,122]. Inclinometers and total stations are the main
instruments that are currently used in the industry for monitoring reinforced soil structures and
retaining walls. Although geotechnical instrumentations are accurate in determining wall
displacements, they are usually installed in boreholes or are fixed to on-site structural elements,
requiring additional effort. Stability of an inclinometer is subject to external movements due to site
conditions and has to be verified throughout the data collection period to avoid erroneous
measurements. Moreover, geotechnical instruments are usually installed in specific locations of a
construction site that are determined based on the site conditions to avoid potential conflicts;
therefore, the collected data is limited to the selected sections of the project [45]. On the other
hand, optical survey of point targets using total stations is time-consuming and the data are limited
to the surveyed targets. Therefore, the process is not optimal for data collection of a wall with large
length and height as scanning all points on a wall is not practical. Moreover, accurate and detailed
modeling of the reinforced soil structures for studying and predicting their behavior requires data
points throughout the structure. There is a need for a high-speed and comprehensive collection of
data as well as an automated method for measuring the displacements of these structures with an
improved accuracy compared to the conventional methods.
LiDAR systems are becoming industry standards for collecting data, especially for high budget
infrastructure projects, such as transportation systems [25]. Although LiDAR equipment are more
expensive compared to the other imaging systems, such as digital cameras, their accuracy makes
them essentially desirable for large-size projects, such as highway infrastructure, where high risk
and maintenance cost are at stake. Laser scanning technology has been successfully used by
researchers in change detection for civil infrastructure projects. For example, for documenting
ground deformation during urban excavation [123], changes and deformations of a rock face cliff
[15], bridge inspection [126,127], evaluating the quality of point cloud as-is BIM models [20], and
construction quality assessment [30,72,98]. To the best of my knowledge, these studies primarily
focus on representing changes of a target as a whole and lack to distinctively assign changes to
specific features within the target; therefore, it is difficult to know what features should be used as
a benchmark to evaluate changes within the target over a period of time. Additionally, the current
change detection approaches require accurate manual inputs of parameters and there is a need for
an automated change detection technique.
In order to investigate the possibility of measuring highway retaining wall displacements using
TLS, in this chapter, we studied MSE walls, since they are the most widely used form of highway
retaining walls in the United States [17]. The proposed work takes horizontal joints between
highway retaining walls’ panels as benchmarks and provides displacement data for all the panels
on the wall through a fully automated framework. The general methodology consists of two steps.
First, an algorithm is introduced for extracting MSE wall facing panels’ horizontal joints from a
TLS generated point cloud. Next, the displacement of the wall is determined by comparing the
extracted joint displacements. In this chapter, an overview of recent literature on stability analysis
of MSE walls, as well as current change detection techniques for measuring MSE wall
displacement are provided. The proposed method for automated displacement measurement using
laser scanners is described in the methodology section. A real-life dataset is used to evaluate the
performance of the feature extraction module. Then different laser scanning settings and wall
movement scenarios are simulated to validate the method’s robustness under various conditions.
Lastly, a discussion and plan for future work in this area are presented.

40
9.2 Methodology
In order to measure MSE wall displacements, we developed a framework for an automated laser
scan-based feature extraction that uses geometric features on the wall as benchmarks to detect
movements. The framework integrates a 3D object detection algorithm with signal processing
techniques to extract horizontal joints from 3D point clouds. The framework was validated using
real-life data, which represent one of the most common forms of MSE walls. The robustness of
the proposed framework is evaluated by analyzing the accuracy in extracting horizontal joints from
the wall’s point cloud, as well as the accuracy in measuring the wall displacements using the
extracted joints. The generalizability of the framework is then evaluated using synthetic data from
other types of MSE walls. The following sections describe the MSE wall point clouds’ geometric
features; explain the proposed framework for automated wall displacement measurement; examine
the feature extraction accuracy using real-life data; and evaluate the framework’s performance
using simulation techniques.
9.2.1 MSE Wall 3D Point Clouds and Basic Assumptions
Figure 9 shows typical front, top, and section views of an MSE wall’s point cloud. The large
indentations in the top view are vertical joints and the small indentations are the panel ridges. The
section view shows how the horizontal joints are outlined in the point cloud. The penetration of
laser beams in the facing panels’ ridges creates a change of thickness for the point cloud, which is
visually evident in the section view. The vertical joints generally generate larger and wider
indentations and therefore could be simply differentiated from panel ridges across the wall.
Horizontal joints, however, are smaller and are more challenging to identify especially in noisy
point clouds. Moreover, they might not be captured completely in case of having a poor scan plan
and large angle of incidence (AOI). In addition, the mixed pixel phenomenon results in noise
around the ridges on the wall’s façade.

41

Figure 9. An MSE wall's point cloud
We built the methodology on the assumption that the MSE wall displacements are within the
standard allowable ranges as reviewed in the introduction section. A typical point cloud of an MSE
wall contains millions of points depending on the length and height of the wall and the resolution
with which the laser scanner captures the data. In order to reduce the complexity and computation
time, our method works with one column at a time. Continuous settlements and lateral
displacements result in wall’s rotation along its main axis (X-axis in Figure 10). We computed
expected rotation angle of MSE walls using the values of acceptable movements as stated in the
introduction. We studied an MSE wall with three panels similar to the case study MSE wall on
which we built our methodology. An approximate value of maximum rotation angle could be
estimated for a column of three panels with 2.8 m
2
surface area (maximum panel area commonly
used in practice) according to the following calculations:

1. Panel’s Area = 2.8 m²
2. For a square panel: 𝑃𝑎𝑛𝑒 𝑙 ′
𝑠 𝐷𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛 = ξ2.8 = 1.67 m.
3. Column’s length= Panel’s Dimension = 1.67 m.
4. Max. Allowable diff. Settlement (for each column) = 0.01(1.67) = 16.7 mm.
5. Max. Allowable Lateral Movement recommended in MSE wall design manuals = 50.8 mm.
6. For a column with 3 panels, Approx. 𝐶𝑜𝑙𝑢𝑚 𝑛 ′
𝑠 𝐻𝑒𝑖𝑔 ℎ𝑡 = 3ሺ1.67ሻ= 5010 mm.
7. Max. expected Wall Rotation (φ) = tan
−1
ቀ
50.8
5010
ቁ ≅ 0.6°

42

(*) The “5010 mm” value is an approximation; the actual value has to be smaller which will result
in a smaller rotation angle, nevertheless, we have used the approximated value to calculate the
maximum value for rotation angle.

Figure 10. Rotation of MSE wall along its main axis. The yellow dashed line shows the rotation
of the red column along the X-axis with rotational angle of φ.
Note that since the allowable lateral movements are limited to certain standard values, even for
walls with more number of panels, the rotation angle will remain close to the one calculated here.
Section 9.4 presents different possible types of MSE wall movements in more detail.
9.2.2 Feature Extraction Using Laser-scan Data
The proposed methodology consists of four steps. In the first step, a column of panels is extracted
from the point cloud and a statistical method is used to clean the noise in the data. As mentioned
in the previous section, the joints between columns of panels (vertical joints) generate large
indentations in the point cloud; therefore, they could be used as reference points for segmentation
of the point cloud to columns of panels. The indentations that have depth values greater than the
average indentations are identified as vertical joints and are used to extract the column of panels
from the point cloud. In the second step, a planar model is fitted to the cleaned data. The third step
sifts through the output from the second step and uses a step detection technique to extract
horizontal joints. Finally, the extracted horizontal joints from two sets of point clouds are
compared to determine the displacement of the wall.
9.2.2.1 Noise Removal
Laser-scan data usually contains noise that appears as fuzziness in the point cloud and usually
attributed to the random error of laser scanners. The data fluctuation characteristics cannot be
modeled with a unique distribution, however, they are usually close to normal (Gaussian) in nature,
therefore, they are modeled approximately with normal distribution [73]. A statistical outlier

43
removal technique similar to [108] was adopted to reduce the noise in the point cloud. The
algorithm traverses the point cloud twice. In the first traverse, the distances (𝑑 𝑖 ) of every point (𝑝 𝑖 )
from its 𝑘 nearest neighbors are computed. The value of 𝑘 for the purpose of noise removal
depends on the point cloud’s resolution/density and is usually determined using a trial and error
process. The resolution of point cloud refers to the average spacing between the points. The
expected spacing, for a noise-free point cloud, can be estimated knowing the angular resolution of
the laser scanner and its distance from the scan target. In order to determine an appropriate 𝑘 value,
the point cloud is projected to an X-Y plane with 10 cm × 10 cm square grids. The square size
should be proportional to the size of the data under study. The square grid size was determined
through a trial-and-error process after visually evaluating the result of the noise removal process
with different sizes of square grids on multiple point clouds. In order to have an estimation of the
point cloud’s resolution, the number of points within each square grid is calculated and the number
with highest frequency is chosen as the 𝑘 value. With the selected square grid size, the pilot data
analysis on a real-life MSE wall scan data with 50 columns of panels showed an average dominant
point density ranging from 45 to 52 points per square grid. Therefore, in this study, we used 𝑘
values ranging from 45 to 52 for the noise removal process. Note that the reported average
corresponds to the most frequent point density in every column. The standard deviation of the
average point density ranged from 1 to 3 points per square grid. The small standard deviation
points out to the homogeneity of the scan data. Once the distances are computed, the corresponding
mean (𝜇 ) and standard deviation (𝜎 ) values are calculated. In the second traverse, every point that
has distance greater or smaller than (𝜇 + 𝜎 ) is considered as outlier (noise) and is removed from
the point cloud. We used multiplier of 1 for 𝜎 as recommended by [106,108]. Visual inspection on
20 columns of panels after the noise removal process verified relative reduction in fuzziness of
data. Algorithm 1 represents the process of point cloud noise removal.

1. project the data to X-Y plane;
2. find most frequent number of points (𝑘 𝑒𝑠𝑡 ሻ within the square
grids;
3. set k = 𝑘 𝑒𝑠𝑡 ;
4. for every point 𝑝 𝑖 ∈ 𝑃
5. compute 𝑑 𝑖 =
1
𝐾 σ ሺ𝑑𝑖𝑠𝑡𝑎𝑛𝑐𝑒 𝑜𝑓 𝑝 𝑖 𝑓𝑟𝑜𝑚 𝑖𝑡𝑠 𝑘 −
𝑘 𝑗 =1
𝑛𝑒𝑖𝑔 ℎ𝑏𝑜𝑟𝑠 ሻ;
6. end for;
7. compute 𝜇 = 𝑚𝑒𝑎𝑛 ሺ𝑑 𝑖 ’s) and 𝜎 = 𝑠𝑡𝑑 ሺ𝑑 𝑖 ’sሻ;
8. for every point 𝑝 𝑖 ∈ 𝑃
9. if (𝑑 𝑖 < 𝜇 + 𝜎 || 𝑑 𝑖 > 𝜇 + 𝜎 )
10. remove 𝑃 𝑖 ;
11. end if;
12. end for ;
Algorithm 1. Removing point cloud’s
noise

44

9.2.2.2 Plane Fitting
Once the noise is removed, a planar model is estimated and fitted to the data to extract the wall’s
surface. Since there is a distance between the horizontal joints and the wall’s surface, the joints
can be assumed to be on a different plane other than the wall’s plane. In other words, the horizontal
joints can be classified as outliers relative to the points on the wall’s surface. We used random
sample consensus (RANSAC) algorithm [49] to fit the best model to the MSE wall’s point cloud.
RANSAC is widely used in the fields of computer vision and point cloud processing for robustly
fitting a model to outlier-contaminated data for object detection and feature matching purposes.
The algorithm randomly selects minimal subset of points from the data and determines parameters
of the best model that can be fitted to the data. It then identifies the points that belong to the
estimated model based on a distance threshold (𝑡 𝑅𝑎𝑛𝑠𝑎𝑐 ) and classifies them as inliers. This process
is repeated until the model with highest defined probability of having inliers (𝑃 𝑠 𝑢 𝑐𝑐𝑒𝑠𝑠 ) is obtained.
The main advantage of the RANSAC algorithm is its robustness in presence of high percentage of
outliers (> 50%) [105]. The challenge with RANSAC is to determine the best distance threshold
based on the input data. The threshold value mainly depends on the parameters, such as point cloud
resolution, ratio of noise in data, and the target geometric primitive that the algorithm intends to
extract. In a previous study, we evaluated different manually selected distance threshold values for
various sizes of the data similar to the one studied in this chapter [89]. For the purpose of this
study, we use an automated method for finding the distance threshold based on the input point
cloud. A cubic S × X × Z (S=height, X=length, and Z=depth) sliding window traverses the point
cloud in the Y (wall’s height) direction with step size of S (e.g., for a column of panels with 10
meters height, a sliding window of 0.1 m height traverses the wall in 100 steps). The value of S is
defined based on the average resolution of input point cloud (r). The values of X and Z are equal
to the point cloud’s maximum width and depth, respectively. In each traverse, the depth/thickness
(𝑇 𝑖 ) of the data points inside the window is calculated and the segment’s center point (𝐶 𝑖 ) is
determined. Once all the 𝑇 𝑖 s and 𝐶 𝑖 s were computed, the histogram of the point cloud’s thickness
is established by assigning frequencies (𝑓 𝑖 ) to every 𝑇 𝑖 . The thickness with highest frequency is
defined as the dominant thickness (𝑇 𝑑𝑜𝑚 ) of the point cloud, which is in fact the thickness of the
panels’ texture. Next, for the data segments that have thickness value equal to 𝑇 𝑑𝑜𝑚 , the values of
𝐶 𝑚𝑎𝑥
and 𝐶 𝑚𝑖𝑛 are extracted. Due to the fact that the MSE walls are not necessarily straight
(because of displacements), using 𝑇 𝑑𝑜𝑚 as the distance threshold might not retain all horizontal
joints since the center-line of the wall is not straight. Note that the objective of fitting plane on the
wall’s point cloud is to isolate the horizontal joints from the points on the wall. In order to take
into account the non-straight shape of the wall, and to make sure that the horizontal joints would
not fall in the inliers’ range, the distance threshold is defined as follows:
𝑡 𝑅𝑎𝑛𝑠𝑎𝑐 =
1
2
ሺ𝑇 𝑑𝑜𝑚 ሻ − ሺ𝐶 𝑚𝑎𝑥
− 𝐶 𝑚𝑖𝑛 ሻ (1)
Although the above choice of distance threshold results in inaccurate model fitting, it decreases
the possibility of detecting horizontal joints as inliers. An initial evaluation was done to test the
accuracy of the threshold estimation. RANSAC algorithm was used to fit planar model to 50
different datasets (column of panels) using automated and manually selected distance thresholds.

45
The manual method comprised of evaluating different distance thresholds using a trial and error
process. The error percentage in threshold estimation was defined as:
𝐸𝑟𝑟𝑜𝑟 𝑃𝑒𝑟𝑐𝑒𝑛𝑡𝑎𝑔𝑒 = ቀ
𝑎𝑢𝑡𝑜𝑚𝑎𝑡𝑒𝑑 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑖𝑜𝑛 − 𝑚𝑎𝑛𝑢𝑎𝑙 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑖𝑜𝑛 𝑚𝑎𝑛𝑢𝑎𝑙 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑖𝑜𝑛 ቁ × 100 (2)
Note that the manual distance threshold estimation is based on obtaining a planar model for the
data while the horizontal joints are preserved. An average error percentage of 5.65% was observed
after testing the algorithm on 50 different data sets, which indicates the proposed automated
method can provide an accurate estimation for the distance threshold of RANSAC algorithm.
Algorithm 2 summarizes the process of fitting plane to MSE wall’s point cloud for extraction of
horizontal joints as RANSAC outliers.

9.2.2.3 Extracting Horizontal Joints
The RANSAC outliers are comprised of horizontal joints along with scattered points from the
wall’s surface (false positives). The conservative choice of RANSAC’s distance threshold together
with the fact that the walls are not straight results in presence of false positives in the data (Figure
11). The false positives represent themselves as fuzziness and scattered data points. They cannot
be removed with noise removal techniques similar to the one explained previously because the
1. set S = r and 𝑦 1
= 𝑦 𝑚𝑖𝑛 and 𝑦 2
= 𝑦 1
+ 𝑆 ;
2. while ሺ𝑦 1
< 𝑦 𝑚𝑎𝑥
ሻ
3. set 𝑇 𝑖 = 𝑍 𝑖 ሺ𝑚𝑎𝑥 ሻ − 𝑍 𝑖 ሺ𝑚𝑖𝑛 ሻ ;
4. compute 𝐶 𝑖 ;
5. set 𝑦 1
= 𝑦 2
and 𝑦 2
= 𝑦 2
+ S ;
6. end while;
7. compute histogram of T for 𝑃 𝑖𝑛𝑝𝑢𝑡 by assigning frequency (𝑓 𝑖 ) to all
𝑇 𝑖 ′
𝑠 ;
8. find corresponding T with 𝑓 𝑚𝑎𝑥
;
9. set = 𝑇 𝑑 𝑜 𝑚 = 𝑇 ;
10. find 𝐶 𝑚𝑖𝑛 and 𝐶 𝑚𝑎𝑥
for T with 𝑓 𝑚𝑎𝑥
;
11. set 𝑡 𝑅𝑎𝑛𝑠𝑎𝑐 =
1
2
ሺ𝑇 𝑑𝑜𝑚 ሻ − ሺ𝐶 𝑚𝑎𝑥
− 𝐶 𝑚𝑖𝑛 ሻ and 𝑃 𝑠𝑢𝑐𝑐𝑒𝑠𝑠 = 0.99;
12. compute RANSAC planar model and fit the model to 𝑃 𝑖𝑛𝑝𝑢𝑡 ;
13. return 𝑃 𝑅𝑎𝑛𝑠𝑎𝑐 𝑜𝑢𝑡𝑙𝑖𝑒𝑟 ;
Algorithm 2. Fitting plane to point cloud

46
point cloud does not have a solid shape and therefore some of the points on horizontal joints would
be removed as well.

Figure 11. RANSAC outliers (colored as white) along with false positives (colored as yellow)
In order to isolate and extract horizontal joints from the RANSAC outlier results, we designed a
point cloud processing method that is based on edge detection principles. The horizontal joints can
be perceived as edges since they are in fact sudden changes in the point cloud. The joints can be
detected if the point cloud is processed across the wall’s height (Y direction). In order to simplify
the problem and to prepare the data for an edge detection process, the data can be projected to an
X-Y plane. Rectangular stripes across the Y direction can be replacements for pixels in images.
The stripes that contain significantly more points can be identified as edges. In other words, the
histogram of Y values can be used for edge detection. We used Freedman-Diaconis [51] method
to determine the number of bins for calculating the corresponding histogram. Freedman-Diaconis’s
method has been reported to be more accurate compared to other methods that are used for
histogram’s bin number estimation [116]. The method is less sensitive to the outliers in data than
common standard deviation-based methods. The bin sizes are defined based on the following
formula:
Bin Size =
2
ξ𝑛 3
ሺ𝐼𝑄𝑅 ሻ (3)
Interquartile range (IQR) is a measure of statistical dispersion and is defined as the difference
between the upper (third) and lower (first) quartiles for a dataset with n points. The histogram of
Y values is computed using the above method. Next, an empirical cumulative distribution function
(CDF) is calculated, which is in fact the cumulative histogram of Y values. The height of each
point on CDF is the frequency (𝑓 𝑖 ) of the corresponding histogram’s bin. As seen in Figure 12, the
CDF diagram is step-wise and has sudden jumps in joints’ locations. As a result, the first derivative
of CDF has local maxima and could be used to localize the horizontal joints. The first derivative
(𝐺 𝑖 ) of CDF (𝐶 𝑌 ) is defined as follows:
𝐺 𝑖 =
𝜕 ሺ𝐶 𝑌 ሻ
𝜕𝑓
= 𝐶 𝑌 ሺ𝑓 𝑖 +1
ሻ − 𝐶 𝑌 ሺ𝑓 𝑖 ሻ (4)

47
Once the first derivative operator is applied to the empirical CDF, the edges (horizontal joints) are
detected by comparing the calculated gradients to a predefined threshold (t). Since the jumps are
near-step in nature (see Figure 12), any gradient with approximate value of 1 can be identified as
an edge. We set the threshold to 0.9 as a conservative number to ensure all the jumps are detected.
Algorithm 3 summarizes the joint extraction process.

Figure 12. Process of extracting horizontal joints from RANSAC outliers

1. set 𝑃 𝑌 = Pሺ𝑦 1
, … , 𝑦 𝑛 ሻ;
2. compute 𝑄 1
= 25
𝑡 ℎ
𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑖𝑙𝑒 and 𝑄 3
= 75
𝑡 ℎ
𝑝𝑒𝑟𝑐𝑒𝑛𝑡𝑖𝑙𝑒 ;
3. compute 𝐼𝑄𝑅 = 𝑄 3
− 𝑄 1
;
4. compute 𝐵𝑖𝑛 _𝑆𝑖𝑧𝑒 =
2
ξ𝑛 3
ሺ𝐼𝑄𝑅 ሻ;
5. compute histogram of 𝑃 𝑌 using 𝐵𝑖𝑛 _𝑆𝑖𝑧 𝑒 ;
6. compute cumulative histogram of 𝑃 𝑌 (empirical CDF) = 𝐶 𝑌 ;
7. for every bin 𝑏 𝑖 ∈ 𝐶 𝑌
8. 𝐺 𝑖 =
𝜕 ሺ𝐶 𝑌 ሻ
𝜕𝑓
= 𝐶 𝑌 ሺ𝑓 𝑖 +1
ሻ − 𝐶 𝑌 ሺ𝑓 𝑖 ሻ;
9. end for;
10. for every bin 𝐺 𝑖 ∈ 𝐺
11. if 𝐺 𝑖 >t
12. find corresponding 𝑃 𝑖 , 𝑃 𝑖 → 𝐽 𝐻
13. end if;
14. end for;
15. return 𝐽 𝐻 ;
Algorithm 3. Extracting horizontal joints

48

9.2.2.4 Measuring Horizontal Joint Displacements
As a result of the preceding steps, the horizontal joints on the wall faces are extracted and can be
used as benchmarks for measuring the wall displacements. For every two sets of laser-scan data,
one is taken as the baseline and the other one is compared to it to identify the changes. MSE wall
displacements are generally monitored on a weekly basis near the end of and right after their
construction to determine if the settlement is complete. Point clouds from two scan dates are
derived and columns of panels are studied and compared one by one. The comparison is based on
the assumption that the surveying team uses a unified coordinate system for scanning the walls.
The unified coordinate system is project specific and it is defined using a network of ground control
points (GCP) surrounding the wall. The points on the GCP network are selected by the surveying
team prior to the scan of the wall and their coordinates are collected using a total station. For each
scan session, reflective targets are placed on every point within the GCP. The reflective targets are
scanned along with the wall and corresponding coordinates from the GCP network are assigned to
them. The point clouds from different dates are registered and georeferenced using the unified
project coordinate system. Using a unified project coordinate system is a practice commonly used
by surveying teams as it decreases registration errors especially during construction when the
project is experiencing several changes. The real-life data has been registered and georeferenced
using a similar methodology. The point clouds from different dates are aligned using the unified
project coordinate system and therefore a detailed comparison of points from two scan dates gives
the actual displacement of the wall between the two dates. Nonetheless, in the case when there is
no access to a valid GCP network, the point clouds have to be registered using geometric features
close to the wall that have remained static throughout the time. Common coordinates have to be
assigned to these features so that the point clouds from different dates would follow a similar
coordinate system.
The points within each point cloud of horizontal joints are clustered using K-means clustering
technique and based on their Euclidean distance from each other (Figure 13). The advantages of
using K-means clustering technique for this problem are (1) it is an unsupervised technique and
(2) the convergence is fast since the most suitable number of clusters is already known. The
number of clusters (K) is defined based on the actual number of horizontal joints that are
determined previously by finding the number of jumps in the point cloud’s CDF diagram. The
clusters’ centroids from two point clouds are compared and matched/paired based on their
Euclidean distance. Once the corresponding horizontal joints were matched, the data points that
have the closest distance to each other (i.e. have minimum Euclidean distance) are compared
individually to determine displacement of each point. For each paired joints, the average
displacements of the points across Y and Z directions show the settlement and lateral displacement
of that particular joint. The proposed displacement measurement method only compares the
clusters with the closest distance, therefore, if a point cloud from newer scan date have more
horizontal joints due to progress in construction, the extra joint will not be mixed with older ones.
Algorithm 4 summarizes the process of measuring horizontal joints’ displacements.

49

Figure 13. The extracted joints from two sets of point clouds are clustered and the clusters with
closest Euclidean distance are compared to determine the displacement of joints.
9.3 Evaluation Using Real-life Data
A 150 m section of the MSE walls on California Interstate I-405 Sepulveda Pass Widening project
was used to evaluate the accuracy of the proposed method (Figure 14a). The section includes 100
columns of 1.5 m * 1.5 m precast concrete panels. The height of wall varies from approximately
4 to 7 meters throughout the freeway. The data were collected by the project’s surveying group
using a RIEGL VZ-400 TLS with AR set to 0.027°. The detailed specifications of RIEGL VZ-400
could be found at [10]. The laser scanner’s average distance from the wall was 10 m for 3 scan
positions with average spacing of 50 m. The registration accuracy of the scan data was determined

1. set 𝐾 1
= 𝑁𝑜 . 𝑜𝑓 𝑗𝑢𝑚𝑝𝑠 𝑖𝑛 𝑃 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒 and 𝐾 2
=
𝑁𝑜 . 𝑜𝑓 𝑗𝑢𝑚𝑝𝑠 𝑖𝑛 𝑃 𝑐𝑜𝑚𝑝𝑎𝑟𝑒𝑑 ;
2. cluster 𝑃 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒 using 𝐾 1
clusters and 𝑃 𝑐𝑜𝑚𝑝𝑎𝑟𝑒𝑑 using 𝐾 2
clusters;
3. find corresponding clusters in 𝑃 𝑏𝑎𝑠𝑒𝑙𝑖𝑛𝑒 and 𝑃 𝑐𝑜𝑚𝑝𝑎𝑟𝑒𝑑 using minimum
Euclidean distance of clusters’ centroids;
4. compute average displacement between the points of corresponding
clusters in Y and Z directions;
Algorithm 4. Measuring horizontal joints’ displacements

50
by evaluating the alignment of overlapping scanned areas on the surface of the wall panels. The
accuracy was evaluated by fitting a planar model to the surface and computing the average standard
deviation of the points from the plane. The average standard was computed as 2 mm which is
reasonable since the surface is textured. The project team uses TopoDOT (Certainty 3D)
commercial software to process the laser-scan data for measuring wall displacements. The
software requires manual alignment of cleaned laser-scan data with the wall’s CAD (Computer
Aided Design) model as well as input of vertical joints coordinates to recognize the geometry of
the wall.
We studied the wall displacements during the two weeks of construction by using the laser scan
data from the beginning (baseline/prior date) and end of the two-week period (later date). The
laser-scan data was cleaned by manually cropping out the wall’s surrounding environment and by
removing construction materials that occlude the wall’s surface using the automated method used
by [89]. We used vertical joints to segment the point cloud to columns of panels. The point cloud
was segmented with 100% accuracy. An alternative segmentation solution would be to simply split
the wall’s point cloud to smaller portions along the direction of length. The panels’ width could be
used as the spacing between the segments. The main objective for the segmentation is to reduce
the computational complexity in different steps. Therefore, the spacing used for the segmentation
has to be small and preferably in the order of panels’ width. For walls with curvature, the segments
have to be small enough so a planar model could be fitted to the data [20]. Therefore, a spacing
size similar to the width of panels would be suitable.
Once the point cloud was segmented, the columns of panels were processed following the proposed
methodology (see Figure 14b and c). Figure 15 represents the results for extracting horizontal
joints from the two sets of scan data (prior and later date). The feature extraction accuracy was
defined based on the ratio of the horizontal joints that were extracted by the proposed methodology.
Given the fact that horizontal joints are similar to lines, the extracted joints were projected to a
grid-lined screen and the accuracy of feature extraction was determined based on the number of
squares that were filled by the joints. The number was then rounded to the closest quarter in the
scale of 0 to 1 (0= the joint is not detected, 1= the joint is detected completely). The average
accuracy of feature extraction for the two scans was 94.26 %. The parts of the wall that did not
have enough data points mainly caused the inaccuracy in extraction of horizontal joints. There
were some missing data due to low point density in the areas that were a large AOI between the
wall’s surface and the scanner.
In the next step, the extracted joints from two scans were aligned and compared with each other to
measure settlement and lateral displacements values. Figure 16 presents the average values of
settlement and lateral displacements for each column of panels. The results indicated a maximum
settlement of 46 mm as well as a lateral displacement of 24 mm for the studied wall. Unfortunately,
the data for actual displacements of the wall were not available as no field measurement was
collected by either geotechnical instruments or total stations therefore, we were not able to quantify
the error of automated displacement measurements and compare the proposed method’s results to
other conventional methods. However, we carried out further studies to evaluate the validity of
displacement measurements for real-life data (Section 9.5).

51

(a) Top view of the studied MSE wall section showing the wall’s layout

(b) Extracted horizontal joints

(c) The horizontal joints from two scan dates are matched and compared. The white and red
points belong to prior and later dates respectively.
Figure 14. Evaluation of the proposed method using real-life MSE wall scanned data

Figure 15. Accuracy of extracting horizontal joints from columns of panels

52

Figure 16. Average settlement and lateral displacement of joints within each column of the
studied wall

9.3 Evaluation Using Simulated/Synthetic Data
In order to evaluate the accuracy and robustness of the proposed methodology, we simulated
different scanning scenarios by using the 3D model of the case-study MSE wall. The simulation
method was used since we did not have long-term access to the case-study MSE wall to evaluate
the proposed method’s accuracy using different scan setups. Moreover, the case-study wall did not
necessarily experience all types of movements and therefore, there was a need to manually model
different types of movements. The geometric parameters and dimensions of the studied real-life
MSE wall were extracted and the wall was 3D-modeled in Rhinoceros 3D software (Figure 17).
BlenSor was used for modeling the laser scanner’s sensor. The physical properties of scan target
and laser scanner can be modeled and represented using Blender as the underlying environment of
BlenSor.

Figure 17. 3D modeled MSE walls: Top left: front view, Top right: section view, and Bottom: top
view

53
Different possible movements of MSE walls were identified based on the information from
available U.S. highway retaining walls design and construction manuals [7,26,109]. For every
movement scenario, the wall was manually shifted from the baseline (wall with no movement) in
the modeling software to represent a new condition. Settlements and lateral displacements of 10,
50, and 100 millimeters were modeled based on the expected range of movements as discussed in
Section 9.1. Note that for settlement, the wall only moves in Y (height) direction and for lateral
displacement it only moves in Z (depth) direction. In addition, two types of rotational movements
were modeled with rotation angles that a MSE wall can normally tolerate before getting visually
evident damages (discussed in section 9.1). In rotational movements, the wall moves in both Y
and Z directions. The first rotational movement assumes that the entire column of panels has
rotated along its principal axis. The clockwise and counterclockwise rotations are denoted with
positive and negative angles, respectively. The second rotational movement assumes unequal
rotation of panels on the wall. The rotation angles are either 1 or 2 degrees due to the fact that
larger rotation angles will usually result in breakage of panels. The reason for the selection of the
rotation angle value is explained next. In order to simplify the process of creating MSE wall’s 3D
model for the simulation, we rounded the 0.6° rotation angle (calculated in Section 9.2) to 1°.
Additionally, in order to test the method’s performance for extreme conditions, we also considered
scenarios in which the wall would rotate twice the expected maximum rotation (2°). Figure 18
shows different wall movements that were modeled for simulation.

Figure 18. Section views of different wall movements that were modeled for simulation. (a) Lateral
displacement, (b) Settlement, (c) Single angle rotation: All panels are rotated with the similar angle
“α”, (d) Multiple angles rotation: The two middle panels are rotated with “α” and the others with
“β” where α, β = 1° and 2° interchangeably.
In addition to modeling different wall movements, we also modeled various scan setups for each
movement. All of the movements were virtually scanned in the simulation environment with
different scan setups including changes in the AOI, distance from the laser scanner, and angular
resolution (AR) of the laser scanner. The angular resolution of 0.06° (equivalent to 0.01 m * 0.01
m point spacing in scan data when the laser scanner is in 10 m range) was defined as medium
resolution considering that it is commonly used for outdoor scanning [92,118]. Low resolution and
high resolution were set to half and twice the medium resolution, respectively. The range and
performance specifications of RIEGL VZ-400 TLS were derived from the manufacturer’s

54
brochure and were entered in the BlenSor to model the sensor appropriately. The default laser
scanner’s height was set to 1.75 meters. Table 3 represents different wall movements and scan
setups that were considered in the simulation. For every simulation, a Gaussian random noise with
mean value of 0 and standard deviation of 0.001 was added to make the simulation more realistic.
In addition, since the MSE wall panels are normally made from concrete, the reflectivity of panel
surfaces was set to average reflectivity of concrete-made materials, which is equivalent to ρ= 40%.
A total of 540 simulations were conducted for the validation purposes of this study. It is noteworthy
to mention that several simulation scenarios were randomly selected and for each of them the
simulation was repeated for three times.

Table 3. Simulated wall movements and scan setups

Wall Movements Scan Setups
Settlement
Lateral
Displacement
Single
Angle
Rotation
Multiple
Angle
Rotation
Incidenc
e angle
Distanc
e
Angular
Resolution
0 mm
(Baseline)
0 mm
(Baseline)
1 deg. 1 deg. 0 deg. 5 m Low= 0.12°
10 mm 10 mm -1 deg. -1 deg. 10 deg. 10 m
Medium=
0.06°
50 mm 50 mm 2 deg. 2 deg. 20 deg. 30 m High= 0.03°
100 mm 100 mm -2 deg. -2 deg. 30 deg.
* Total Number of Simulations: 3 (Distances) * 4(Angles) * 3(Resolutions) * 15 (Wall
Movements) = 540

9.4.1 Comparison of Simulated Scans with Real-life Scan Data
In order to verify the validity of the simulations, we evaluated the similarity of real-life scan data
with simulated scans. As mentioned previously, Blensor allows adding a fixed random Gaussian
noise to the simulated scans. We generated five simulated scans with different levels of random
noise using a 3D model of the MSE wall with five columns of panels. Different levels of random
noise were modeled by changing the standard deviation of Gaussian noise from 0.001 to 0.005
meters with 0.001 intervals. Note that the mean value of the noise was set to zero for all of the
cases. Other simulation parameters were selected based on the settings representing the condition
in which real-life MSE walls were scanned. In order to perform a fair comparison, we manually
selected sections of the real-life MSE wall point clouds (the dataset that was used in Section 9.3)
that seemed to be straight and without curvature or rotations. Figures 19a and 19b show the real-
life scan data and a sample simulated scan respectively.

55

(a) Real-life scan data (b) Simulated scan with random noise (Std. =0.001 m)
Figure 19. Snapshots of real-life scan data and simulated scan
Once the simulated scans were generated, they were registered to the real-life scan data using the
commercially available software CloudCompare. The software uses the ICP method to register the
two point clouds. The registration results were manually controlled to ensure the two point clouds
were aligned. After the registration process was completed, the two point clouds were compared
using the software. The process is based on a point-to-point comparison method that computes the
distance between nearest neighbors within a voxel search. Figure 20 presents the heat map of the
distances between the points of the real-life scan and a simulated scan.

Figure 20. Real-life scan vs. Simulated scan with random noise (Std. = 0.001 m)
The color histogram in Figure 20 shows that most of the points have distances less than five
millimeters, which verifies that the simulated data is closely similar to real-life scans. Table 4
presents the results for comparison of five simulated scans with real-life scan data. The results
show how different levels of random noise affect the similarity of simulated point clouds with the
real-life data. Based on the results, the simulation with 0.001 level of noise is the most similar to

56
the real-life scan, which is in compliance with the settings that we used for the simulations in this
study.

Table 4. Average distance of simulated scan from real-life scan data. Note that each simulation is
compared with five different real-life scan and the average is calculated and presented below.
Std. Deviation of
Simulated Scan’s
Random Noise
(m)
Average Distance of
Simulated Scan from Real-
life Scan (m)
Average Std. Deviation of
Simulated Scan’s Distance
from Real-life Scan (m)
0.001 0.0062 0.0091
0.002 0.0069 0.0093
0.003 0.0075 0.0098
0.004 0.0084 0.0104
0.005 0.0102 0.0107

9.4.2. Results and Discussion
Once the simulation process for all wall movement scenarios was completed, the 3D point clouds
were processed using the algorithms that were explained in Section 9.2. The entire process of
extracting horizontal joints and measuring wall displacements took less than 15 seconds in average
for each column of panel using a Microsoft© Windows workstation laptop with Intel® Core i7
processor, and 32 GB RAM memory. Tables 5 and 6 present the feature extraction accuracy and
displacement measurement error respectively.
The feature extraction accuracy ranged from 66.67% to 100%. For the simulation scenarios, where
the laser scanner has low AR and is placed 20 meters from the wall, and has a large AOI (20 or 30
degrees), the feature extraction accuracy has largely decreased especially for rotational movements
of 2 degrees. This low accuracy is due to the fact that the AR of the laser scanner is set to low.
Moreover, the large AOI would result in less number of laser beams penetrating the horizontal
joints. Also, the accuracy decreases as the wall’s AR increases particularly in the cases that the
panels have unequal rotation angles (multiple angles). In fact, for such walls, a planar model that
is fitted to the column of panels does not necessarily follow the wall’s sectional centerline.
However, this inaccuracy can be reduced to some extent by defining a suitable distance threshold
when estimating the planar model for the wall. Indeed, the reason that the center points of the
wall’s scan data were studied for the distance threshold definition was to take into account the
wall’s rotation effect. The displacement measurement’s error for different simulation scenarios
ranged from 0.082 mm to 2.428 mm. The absolute difference between the actual displacement of
a horizontal joint and the automatically measured displacement is defined as the measurement
error. Note that the values in Table 6 represent the average displacement measurement error of all
horizontal joints within a column of panels. The measurement errors have the same pattern as
feature extraction accuracy in the sense that the error increases for lower ARs and higher AOIs.
However, the increase in displacement measurement error is not proportional to the decrease in

57
the feature extraction accuracy because the displacement measurement algorithm computes the
displacement even if some of the points on the horizontal joints are not detected. Note that this is
true based on the assumption that the horizontal joints do not move along the horizontal axis (X).
Nevertheless, there would be still some error in the displacement measurement if some of the
points on similar horizontal joints from two different scans that are compared with each other do
not actually correspond to each other.
In general, the results indicate that the proposed automated displacement measurement error is
mainly associated with the laser scanning error and not with the method itself. Figure 21 illustrates
a heat map of average displacement measurement errors for different scan positions and setups for
the simulated data. Note that the error values for points between the simulated scan positions are
interpolated to present a continuous and smooth heat map. In addition, according to
Soudarissanane, et al. [120], the scan accuracy is identical for similar AOIs with opposite signs
(e.g. 10° and -10°), therefore, the heat map is symmetrical. The heat map provides a visualization
of the displacement measurement error pattern and it confirms that the error is caused by typical
laser scanning inaccuracy sources, such as long range, low AR, and large AOI. Figures 22 and 23
present the average feature extraction accuracy and the displacement measurement error for
different simulated scenarios. According to the results, the range of variation in average feature
extraction accuracy and displacement measurement error values decrease especially as the AR is
reduced. In addition, there is a large increase in displacement measurement error when the AOI
changes from 20° to 30°. Note that the scan setup that was used for real-life data is similar to the
simulation scenario where the AR is set to high (AR=0.027< 0.03) and the scanner is placed 10
meters from the target. As can be seen in Figure 22, the expected feature extraction accuracy falls
between 95% and 100% depending on the AOI. For the real-life data, the scan data is a
combination of registered point clouds from three scan positions, therefore, the AOI factor is
different for overlapping areas. The average feature extraction accuracy is smaller than the
simulation results, which is due to the systematic and random errors caused by instruments,
operators, and conditions on site. Figure 24 represents a comparison for wall displacement
measurement error along Y and Z directions for conditions in which wall has rotated (single and
multiple angles rotations). The differences are less than 0.1 mm, which verifies the proposed
methodology is robust in measuring displacements along both Y and Z directions.

58

Figure 21. Heat map of average displacement measurement error for different scan positions and
setups. For each scan position, the average value of measurement errors for all types of
movements is calculated.

59
Table 5. Matrix of feature extraction accuracy for different simulation scenarios (%)

60
Table 6. Matrix of displacement measurement error for different simulation scenarios (mm)

61

Figure 22. Average accuracy of feature extraction for different scan setups. The average
accuracy for all types of wall movements is calculated.

Figure 23. Average error of displacement measurement for different scan setups. The average
accuracy for all types of wall movements is calculated.

62

Figure 24. Comparison of average error values for wall displacement measurements along Y and
Z directions for conditions in which wall has rotated and where AOI has been 0°.

9.5 Extended Validation
In order to verify the robustness of the proposed methodology for the real-life scan data, we
generated multiple synthetic displacement scenarios by manually moving the point clouds from a
scan date. A set of 45 columns of panels was selected randomly from the studied real-life scan
data. Then the 15 different movement scenarios, discussed in Section 9.4, were manually applied
to the selected columns of panels. Each type of movement was applied to three different columns.
For instance, in order to model a lateral displacement of 50 mm, the panels on the selected column
were manually moved 50 mm across the lateral direction. This process was repeated three times
for every movement scenario in order to have three instances of every movement. The manually
moved point clouds were considered as the “later date” scan and were processed using the
proposed methodology. Feature extraction accuracy and measurement errors were determined
similar to the process mentioned in Sections 9.3 and 9.4. The main objectives of this process were
to compare the results from the simulated data with real-life scan data, as well as to determine
whether any additional error would be generated when processing the real-life data. Note that, as
mentioned in Section 9.4.1, the Blensor software that was used for simulating the scanner’s sensor
adds a fixed random Gaussian noise which is helpful to make the scan data realistic, however, it
does not necessarily result in replicating the real-life conditions accurately. Moreover, real-life
data’s quality is subject to scanning procedure errors, scan registration inaccuracies, and
environmental conditions such as lighting and weather clarity. These factors are not included in
the simulation so it is possible that there might be additional/other potential errors when working

63
with the real-life data as opposed to the simulation data. Table 7 presents the results of the proposed
method for manually moved real-life scan data. Note that the feature extraction accuracies and
displacement measurement errors for different columns are determined and their average values
are calculated and presented in Table 7. Similar to the simulation results from Section 9.4, the
accuracies of feature extraction and displacement measurement decrease slightly when the wall is
rotated. However, based on the results, the error margin is still less than one millimeter. Note that
as explained in Section 9.3, the real-life data was captured with angular resolution equal to 0.027
(equivalent to high resolution in the simulations) and from 10 meters distance. The comparison of
the data from Table 7 to the corresponding cells in Tables 5 and 6 verifies that the error ranges in
both cases are the same, however, not identical. It is noteworthy to consider that these walls were
moved manually; therefore, they have the scan data quality of a straight wall. However, in reality,
if a wall rotates, scan parameters such as the AOI and AR will have direct effects on the accuracy
of the scan since some of the laser beams might not penetrate in the panel joints. Therefore, one
should consider that the presented results do not necessarily reflect the real-life condition exactly,
however, they are reasonable approximations to evaluate the proposed methodology.
Table 7. Average feature extraction and displacement measurement error for manually moved
real-life scan data

In order to verify the generalizability of the proposed methodology, we studied two more types of
MSE walls that are commonly used in highways. Wall type 1 (Figure 25a) is made of polygon
panels with jagged texture and wall type 2 (Figure 25b) is made of rectangular panels with bricked
texture. The walls were 3D-modeled and all the movement types were simulated. The AOI was
fixed to 10° for all simulation scenarios with the purpose to reduce the number of simulations.
Table 8 presents the average accuracy of feature extraction as well as average displacement
measurement error for the two walls. The results indicate that the accuracies are very close for
both walls except that for wall type 2, the accuracy drops in the distance of 10 m and AR of low.
This sudden accuracy drop occurred in the rotational movements in which the panels had different
angles of rotation (multiple angles 2). Since the brick face wall (wall type 2) has minor joints on
its surface, in the rotational movements, the minor joints are occasionally mixed with main
horizontal joints. Note that even for cases with lower accuracy in feature extraction, the error of
wall displacements measurement is less than 2 millimeters.

64

(a) Wall type 1

(b) Wall type 2
Figure 25. 3D models of two other types of MSE walls

Table 8. Average accuracy of feature extraction and error of displacement measurement for
different scan setups for wall types 1 and 2
Distance = 5 m
AR= Low AR= Medium AR= High
Wall type 1 Wall type 2 Wall type 1 Wall type 2 Wall type 1 Wall type 2
95.83%
0.73 mm
94.17%
1.18 mm
97.92%
0.82 mm
95.25%
1.03 mm
98.75%
0.2 mm
97.5%
0.39 mm
Distance = 10 m
AR= Low AR= Medium AR= High
Wall type 1 Wall type 2 Wall type 1 Wall type 2 Wall type 1 Wall type 2
90.00%
1.16 mm
84.25%
1.33 mm
91.25%
1.13 mm
87.33%
1.28 mm
96.97%
0.27 mm
95.67%
0.45 mm
Distance = 20 m
AR= Low AR= Medium AR= High
Wall type 1 Wall type 2 Wall type 1 Wall type 2 Wall type 1 Wall type 2
77.92%
1.35 mm
76.42%
1.85 mm
88.00%
1.22 mm
84.67%
1.34 mm
94.25%
0.42 mm
91.58%
0.64 mm

65
9.6 Limitations and Conclusions
Condition assessment of highway retaining walls requires regular and accurate measurement of
wall displacements particularly during construction when the displacements are larger and could
potentially affect the construction quality. 3D laser scanners provide as-built data with millimeter
level accuracy, which makes them valuable especially for identifying small-scale changes. A fully
automated method for extracting and comparing geometric features from 3D point clouds was
presented in this chapter. The method consists of two steps: (1) extracting horizontal joints on the
facing panels of a highway retaining wall; and (2) measuring the displacements of the
corresponding horizontal joints from two sets of scan data. The method’s accuracy was evaluated
using simulations as well as real-life data. An average accuracy of 94.72% was achieved in
extracting horizontal joints for 540 simulated data sets. The consistency of results for different sets
of data verifies the robustness of the proposed methodology for extracting horizontal joints from
3D point clouds. Condition assessment criteria for highway retaining walls require millimeter level
accuracy, which is attainable by the proposed method. There was a maximum error of 2.428 mm
for automated wall displacement measurement, which is still less than the average error of
currently used methods (3-5 mm). The method was applied to three types of MSE walls and the
results indicate that the method is promising; therefore, the method could be employed for other
types of highway retaining walls unless the geometric features could not be extracted from the
scanned data. The work presented in this chapter distinguishes itself from similar studies by
proposing a fully automated framework for feature extraction from point clouds as well as
millimeter-level displacement measurement using the extracted features as benchmarks. In
addition, the proposed method provides displacement data for all the panels on a wall and it is not
limited to selected regions. It is noteworthy to mention that the scan data along with detailed
analysis of the wall displacements can be used for finite modeling of the wall in order to analyze
and predict its behavior.
For the cases where the geometric features, such as panel joints are not fully detected by the laser
scanner, or for cases with irregular and non-uniform surface features (e.g. walls with rocky
façades), the method cannot give an accurate result. In addition, the accuracy of the method is
directly related to the accuracy of the scanning and registration process of two scans, therefore,
this method cannot be used for scans with poor planning or with erroneous registration. Quality
issues, such as data inhomogeneity (data not being uniform across the project) is very critical and
could affect the accuracy of any sort of data analysis. Also, one should be considerate in choosing
features of interest on the wall or any other target in the sense that the features should not get
diminished or occluded during the time. For instance, if the selected features disappear due to
retrofits or get corroded due to natural events, the proposed method would not work as it takes the
features as benchmarks for change detection.
Regarding the type of MSE walls, the methodology is based on the assumption that the wall has
parallel or uniformly distributed joints. In addition, the method assumes that the wall joints are
deep enough for the laser scanner’s beams to penetrate and make them distinguishable in the point
cloud. Moreover, the method is designed for walls that have aesthetic features on their façade
(texture-rich surfaces). For the walls with flat and featureless facades, regular edge detection
techniques that are based on sudden changes in the geometry may be used to extract wall joints.
Additionally, generating realistic systematic and random noises in the simulation environment is

66
not possible and the laser-scan data simulation is based on a fixed Gaussian noise. Therefore, the
real-life data is expected to be noisier and the average feature extraction is slightly lower than
synthetic data (less than 1%).

67
Chapter 10 AUTOMATED RECOGNITION OF BUILDING FACADES
FOR CREATION OF AS-IS MOCK-UP 3D MODELS

This chapter builds on the findings of the previous chapters by focusing on generating mock-up
3D models of buildings so the geometric properties of buildings could be captured and used for
studying the impacts of quality issues on the accuracy of as-built models. Identification of façade
elements, such as windows and doors and their layout is critical to enable the creation of realistic
façades for mock-up models [82]. However, accurate reconstruction of building facades remains
to be a great challenge. The challenge is primarily attributed to the vast diversity of building
architecture styles, the variety in the types of different façade elements, and the existence of
complex geometries. Thus, there is a need for an automated, rapid, and low-cost building modeling
method that could address the aforementioned challenges. This chapter presents a novel façade
reconstruction process and its validation. To recognize building façade elements, in our approach,
we use ground images that could be collected by a human operator with a smartphone or obtained
from urban image databases, such as Google Maps. The proposed façade reconstruction process
starts by predicting the locations of façade elements by studying the projection of image gradient
profile. The candidate locations that indicate strong gradient responses are clustered across the
width and height of the image and the overlaps of the clustered locations are extracted from the
image to be further processed. The extracted regions go through a vegetation detection process
followed by a classification step to determine whether they have arched or rectangular shapes.
Next, the extracted regions are classified into different types of façade elements by getting
clustered based on using a set of local and global features. Finally, once the location, shape, and
type of façade elements are determined, the corresponding split grammar of the façade is defined
to generate the building’s mock-up model. The proposed method is neither dependent on large-
scale labeled ground-truth data nor requires hours for training classifiers. The mock-up models
will be used in the next chapters to represent the geometric information of the buildings when there
is no updated BIM model available. Specifically, the geometric information will be used to enable
the quality assessment of scan data with respect to building-specific data quality requirements.
10.1 Review of Existing Work
Ground-based, aerial, and satellite images are the prevalent types of data that have been used for
façade reconstruction and 3D modeling of buildings. Aerial imaging is suitable for retrieving
building roofs, however, it is limited in capturing sufficient information on façades [86,141].
Therefore, ground images are commonly used for reconstructing façades. Existence of occlusions,
mostly from vegetation, as well as the reflections from windows along with the illumination effects
introduce major challenges to façade reconstruction from the image data [75,77,79].
The process of façade parsing usually includes segmentation of the rectified images into regions,
such as windows, doors, and balconies [75]. Placements of doors and windows on façades usually
follow a logical structure that leads to a repetitive pattern. For instance, windows are usually placed
in an ortholinear order with alignment across the horizontal and vertical axes of a façade; similarly,
doors are normally placed on the street-level. Existing façade reconstruction methods generally
fall into three categories of gradient-based, bottom-up, and top-down approaches. Gradient-based

68
methods detect façade elements by studying the distribution of gradients in the rectified façade
images. Bottom-up methods use pixel-level classifiers while top-down approaches use shape
grammars for parsing a façade. The parsing tree in top-down approaches is designed to divide a
façade into a set of semantic segments. Therefore, different shape grammars are fitted to a façade
and the one that is the closest in terms of pixel-level classification is chosen as the best match.
However, these methods often tend to be inefficient in terms of time, and occasionally produce
segmentations that are not as accurate as bottom-up approaches at the pixel-level [69]. On the other
hand, bottom-up approaches, if applied standalone, do not necessarily provide sufficient semantics
since they lack information about a façade’s type and layout. Recently, the use of dynamic
programming methods along with probabilistic graphical models, such as Markov Random Fields
(MRF) and Conditional Random Fields (CRF), have become popular for boosting the speed of
classifiers in top-down and bottom-up approaches as well as various image parsing frameworks
[23,103,134,151]. These probabilistic models, originally designed for natural language processing,
are used to model the patterns resulted from the relationship between the adjacent pixels within
the grids of an image.
10.1.1 Gradient-based methods
Image gradients are one of the primary features that are commonly used for detecting objects from
images. Lee and Nevatia [77] identified windows from rectified ground images by using a gradient
profile projection method. An arch detection algorithm was used to choose the closest fit between
the image gradients and a series of hypothesized arches in order to classify the windows. However,
a minor amount of noise or dislocated gradient response could result in misclassification of façade
elements. Therefore, there is a need to involve more image features in the classification stage to
make the classification process more robust to local noise and missing gradients. Recky and Leberl
[101] extended the gradient profile projection method by adding color features. They used k-means
clustering to label the super-pixels in a façade’s images based on the color space. The results
showed that adding color features to the traditional gradient-based method could result in improved
distinction of wall and window areas. Nonetheless, for the façades with near-uniform color, it was
challenging to perform segmentation based on color features. Other studies have suggested the use
of morphological filters to enhance the robustness of gradient-based methods against textured
façades and noise in images [112]. However, using the morphology as a standalone solution is
challenging in the sense that it requires manual fine-tuning of the filter parameters.
10.1.2 Bottom-up methods
Bottom-up approaches became popular since deep learning methods were introduced in computer
vision. These methods are capable of classifying pixel-level features at multiple layers of learning,
which significantly improves the detection accuracy especially for objects, such as windows and
doors that do not always have bold features. Architectural rules are usually applied in the last step
of these type of façade reconstruction methods to reinforce the detection process especially in case
of having occluded objects. Martinović, et al. [82] proposed a multi-step detection process for
parsing façade images. The method consisted of a three-layered segmentation process that used
Recursive Neural Networks (RNN) and MRFs to perform pixel-level detections. A set of
architectural rules was added as the third layer to improve the detections from the prior layers. The
third layer assumed a specific architectural style as the baseline knowledge for identifying a

69
façade’s layout. Although this baseline knowledge reinforces the detection process, diversity is
inevitable even among similar architecture styles (e.g., Haussmann). Moreover, human expertise
is often required to develop appropriate and representative architectural rules for every specific
architecture style. In a similar study, Cohen, et al. [39] introduced a sequential optimization
method that determined the label of each pixel by scoring them according to their compliance with
a set of architectural constraints. These constraints were defined as regularity in a façade’s
structure as well as the presumptive locations of different façade elements and their positioning
with respect to each other. Overall, bottom-up approaches offer accurate pixel-wise detection,
however, their robustness is challenged if employed without using a reliable set of architectural
constraints or rules that follow the façade structure. Therefore, several studies used the shape
grammar of the façades in order to guide the parsing process. In addition, these methods are often
challenged by the excessive time that is required in the training and testing stages.
10.1.3 Top-down methods
Priori models of exterior structure of buildings, as well as hard-coded architectural rules provide a
set of baseline information that could be used to improve and facilitate the façade segmentation
process. For instance, Pu and Vosselman [100] used hard-coded architectural principles to
facilitate the extraction of building elements, such as windows, doors, and roofs, from partially
occluded 3D point clouds. Shape grammars, originally introduced by [121], are prevalently used
to represent hierarchical spatial relationships, which make them suitable for encoding knowledge
about façade types and layout. Shape grammars were evolved by the introduction of split grammars
[145] and eventually maturated into the popular system of Computer Generated Architecture
(CGA) rulesets [85]. The early adopters of shape grammars used them along with the rectified
ground images of buildings to parse façade images with repetitive and symmetrical window grids
[84,85].
Finding the closest shape grammar to an underlying structure of a building’s façade is considered
to be computationally expensive since all grammar rules have to be checked through a pixel-by-
pixel study of a façade’s image. Optimization methods, such as Markov Chain Monte Carlo, are
used to improve the efficiency of parsing façades [103]. Martinovic and Van Gool [83] proposed
a top-down stochastic approach for learning the shape grammar that fits best to a specific building
style. A set of candidate grammars were produced by using a set of pixel-labeled ground truth data
and a Bayesian model was used to identify the closest fit candidate. A Reversible Jump Monte
Carlo approach was then used for grammar parsing of the images at the test stage. Loch-Dehbi and
Plümer [80] proposed a top-down approach by using Gaussian mixture models to develop
statistical reasoning for predicting the type and layout of building façades. Priori reasoning was
derived from a database of building façades and was integrated into a Bayesian network to produce
hypotheses for façade image segmentation. The results showed that such statistical reasoning could
be used in top-down approaches to improve the façade segmentation if the façade layout is similar
to those that were used for training the stochastic models. However, robustness of the probabilistic
models is dependent on the accurate definition of prior probabilities and the stochastic model
parameters [82].

70
Although the top-down approaches are often accurate in parsing façades, they cannot be applied
to buildings with different architectural styles if their grammar is not provided. In addition, pixel-
level accuracy of top-down façade reconstruction methods might not work for all cases since these
methods are limited to the priori structure that a grammar defines. Therefore, robustness of top-
down approaches is dependent on the knowledge derived from a bottom-up study of building
images. As a result, recent studies have incorporated the knowledge derived from bottom-up
approaches to improve the performance of top-down parsing. For instance, some of the parsing
processes employ common regularity and organization of façade layout as a reinforcement to
object detection methods [75]. Ok, et al. [87] used object detectors and façade layout information,
such as repetitive patterns, and line segments to guide a shape grammar-based parser. The results
showed that the façade layout information improved the robustness of the parser. In a recent study,
a user-defined shape prior was used to encode the alignment and placement patterns of pairs of
adjacent pixels. The patterns were transformed into a MRF structure in order to parse rectified
façade images [75]. Overall, two challenges still remain with the studies that integrate top-down
and bottom-up methods. First, they are domain-dependent due to using specific grammars for
training the façade parser. Second, they are time-consuming especially in the training stage due to
pixel-level definition of grammars [69,83].
Overall, the existing approaches are mainly suitable for buildings that have an architecture style
similar to the hand-labeled ground-truth data, with which these methods are initially trained.
Therefore, there is a need for a façade reconstruction method that is independent of architecture
styles and uses the general features of façade elements rather than those that are specific to a
particular architectural style. In addition, recent works have focused on the segmentation of
façades for detecting the general category of façade elements (i.e., window, door, wall) and are
less concerned with recognizing different types (e.g., windows with different sizes or framing) that
exist within each class of an element. Therefore, there is a need for adding more semantics to the
façade reconstruction process by recognizing different instances of the elements within each
element class.

10.2 Computing Gradient Profile and Detecting Arrangement Style of Façade
Elements
Our proposed façade reconstruction process starts by collecting ground images from the exterior
of a target building. The images have to cover the entire façade area of the building and then get
rectified to eliminate the projective and affine distortions. For façade segments that are not covered
in a single image, multiple images are collected, rectified, and merged using a median method to
represent the entire façade. Each image is assigned to a building’s wall through a semi-automated
process by estimating the camera’s position following a standard Structure from Motion procedure.
In our method, instead of using a priori model, such as a baseline shape grammar, we simply take
the gradient profile of façade images as the basis for recognizing the underlying structure of a
façade. The gradient profile of the façade’s image provides critical information about the location
and layout of the façade elements. Façade images are pre-processed prior to computing the gradient

71
profile in order to mitigate the effects of potential lighting bias. We use the color constancy
approach of gray world assumption to remove the lighting bias from images, followed by a
Gaussian filter to smooth the images in order to suppress the edge responses resulting from
textured walls. We used a window size of 5x5 pixels with the standard deviation of five for the
smoothing filter (these parameters were set empirically based on our pilot experiments).
Once the image is pre-processed, the Sobel operator is used to extract horizontal and vertical
gradients across an image. A set of binary morphological operators are then applied to remove the
isolated gradients and to join the unconnected pixels on a binary gradient profile [68]. Given a
binary matrix of pixels, the values of pixels are switched automatically from zero to one if they
are surrounded by nonzero pixels. In addition, if there is an individual nonzero pixel surrounded
by zero-valued pixels, the pixel value is switched to zero. The gradient values are then projected
to the horizontal and vertical axes of the image to generate two accumulated gradient histograms
(similar to the gradient profile visualization introduced by [77]). For the purpose of the study in
chapter, we normalized the histograms to have values between 0 and 50. We chose this scale in
order to be able to overlay the gradient histograms on façade images with different sizes (see Figure
26).
Oscillations of histograms, as well as their overall pattern are influenced by the façade elements’
layouts. In contrast to previous studies [65,77,148] that have considered a single type of façade, in
our approach, we perform a statistical analysis on the normalized histograms to determine whether
they represent separated (e.g., façade type 1 in Figure 26) or connected (e.g., façade type 2 in
Figure 26) façade elements. The analysis is performed by computing the difference between the
1
st
and 3
rd
quartiles of gradient magnitudes in the normalized gradient histograms across horizontal
and vertical directions. If the computed differences are close to each other, it is concluded that the
histogram does not have large oscillations; therefore, the gradients are evenly distributed on the
histogram. As a pilot experiment, we examined the normalized histograms of 175 façade images
with various styles of element layouts in order to define a threshold for distinguishing façades with
connected elements from those with separated elements. These images were acquired from the
Ecole Centrale Paris (ECP) façade parsing database [132], as well as manually collected pictures
from different buildings at the University of Southern California (USC) campus with various
façade layouts. The ECP database consists of 104 rectified façade images of Paris buildings with
Haussmannian architecture style. According to our selected scale for normalizing the histograms
(0-50), we observed that façades with an average gradient difference of less than 10, have
connected elements (see the borderline between the blue and red dots in Figure 27). Note that the
choice of threshold is empirical, however, it is not limited to a specific façade and are/could be
applied to various façade images. Figure 27a presents an example of the distribution of gradient
differences for the studied façade images. Note that the arrangement style of elements could be
different across horizontal and vertical axes. For instance, the windows in façade type 2 in Figure
26 are connected across the horizontal axis but separated across the vertical axis.

72

Figure 26. The overlays of projected horizontal and vertical gradients (cyan color) on two façade
images with different arrangement styles. Façade type 1 (top) has separated elements and façade
type 2 (bottom) has connected elements. (Building image acquired from ©Google)

73

Figure 27. (a) The distribution of the gradient differences for a set of façade images; (b) A sample
of the cumulative histogram of a façade with separated elements (façade type 1, horizontal axis);
(c) A sample cumulative histogram of a façade with connected elements (façade type 2, horizontal
axis).
10.3 Predicting Locations of Façade Elements
We employ two different façade parsing methods to predict the locations of elements depending
on a façade’s detected arrangement style. As seen in Figure 26, a façade with connected elements
generates a gradient histogram that has oscillations with small magnitudes of difference. On the
other hand, in façades with separated elements, the gradient histogram declines in the pixels
between every two consecutive elements (see façade type 1 in Figure 26).
For façades with connected elements, the valley points on the histogram represent the narrow
dividers between elements. In order to identify these valley points, there is a need to sift through
the gradient histogram. For this purpose, at the start, we use two adaptive H-minima and H-maxima
transform filters to suppress the gradient responses that are more than one standard deviation away
from the mean. The filters eliminate the local minima and suppress the strong gradients that are
greater than one standard deviation from the mean (see transition from Figure 28a to Figure 28b).
In the next step, the remaining isolated bumps in the histogram are removed and a moving average
filter is applied in order to smooth the histogram while preserving its original pattern. In the last
step, a search window traverses over the smoothed histogram and extracts the minimum within
each interval. Note that the extracted minima are the valley points that represent the dividers
between the façade elements (Figure 28d). Finally, the façade is parsed based on the detected
valley points and the area between the dividers are considered as predicted locations of the façade
elements (Figure 28e).

74

Figure 28. An example of the process of predicting the locations of elements for a façade with
connected elements. (a) Original gradient histogram; (b) Gradient histogram after applying of H-
minima and H-maxima filters; (c) Equivalent signal profile of the gradient histogram; (d)
Smoothed signal with red circles representing the detected valley points; (e) Horizontal division
of the façade based on the detected valley points. (Building image acquired from ©Google)
For the façades with separated elements, we use a different mechanism to predict the locations of
façade elements. The process starts by extracting strong gradient responses across one of the axes
and looking for the corresponding gradients on the orthogonal axis. Framing and trimming of
façade elements, such as windows, normally create horizontal and vertical edge responses on the
gradient profile. Therefore, we make an assumption that if a region on the image contains
corresponding strong horizontal and vertical gradients, that region belongs to a façade element.
This method is robust for textured façades because these textures usually appear as strong gradients
across only one axis. Since the gradient magnitudes of every image are impacted by different
properties, such as illumination, color, camera angles, and rectification effects, one cannot use the
similar threshold for every image. Therefore, we use an adaptive thresholding technique, which is
a tailored version of Otsu’s thresholding algorithm [94] for extracting only the strong gradients
from the histogram. Threshold is defined based on studying the distribution of gradient response
histogram, particularly its skewness value. Specifically, in our normalized scale of gradients (0-

75
50), we compute the ratio of the number of pixels with gradient values larger than 25 to the total
number of pixels. This ratio indicates what portion of the image pixels hold large gradients. We
use this ratio (hereafter referred as γ) to guide Otsu’s algorithm for finding the optimal threshold
for isolating strong gradients. This guidance is further supported by limiting the search window of
Otsu’s algorithm to gradient magnitudes that are between the 50
th
and (50×(1+ γ))
th
percentiles of
the gradient histogram. We also limit the upper-bound of the search window to 90
th
percentile in
the rare cases of having large gradient magnitudes all over the image. The boundary values of the
search window were set to help Otsu’s algorithm find the proper threshold for the purpose of this
study.
Next, for every extracted strong gradient, a corresponding gradient on the orthogonal axis is
searched. If a corresponding strong gradient is found, the gradient’s location is retained. For
demonstrating the process, we implemented the method on a façade with separated elements. The
vertical and horizontal lines in Figure 29b represent the locations of strong gradients that remained
after the thresholding process.

Figure 29. Process of predicting locations of elements for a façade with separated elements. (a)
Façade’s image; (b) Scattered lines representing the locations of strong gradients across vertical
and horizontal directions; (c) Lines representing the boundaries of the clusters of strong gradients;
(d) Clustered lines of strong gradients; (e) Binary image of the regions with overlapping strong
gradients; (f) Bounding boxes of the predicted elements. (Building image acquired from ©Google)

76
The next step is to group these scattered lines into areas that include façade elements. We use k-
means clustering to automatically identify a proper grouping for the lines. We chose k-means over
other clustering methods since the gradient lines do not follow any specific distribution or density
function that could be used for clustering. In order to determine the best number of clusters, lines
are clustered using k values ranging from 2 to 10. The assumption is that, for every image, the
number of columns/rows of the façade elements is within this range. This range could be increased
in case of having images with more sets of elements. We use the Silhouette method to determine
the optimal number of clusters in each direction. The Silhouette method uses a similarity measure
that is defined based on the distance (we use Euclidean) of the data points from their own clusters
(intra-cluster variance) and the comparative distance to other clusters (inter-cluster variance).
Similarity measures range from -1 to 1. Similarity values that are close to 1 indicate a well-matched
cluster. Therefore, the k value that produces the highest average Silhouette value represents the
most optimal clustering configuration. Due to incorporating two factors in the similarity measure,
Silhouette method is less prone to error comparing to other cluster analysis techniques, such as
elbow method that uses only sum of squared errors as the comparison factor. For the façade that
is shown in Figure 29, the optimal number of clusters for the horizontal and vertical directions are
automatically derived based on the analysis of the Silhouette diagrams as presented in Figure 30
Note that the number of clusters across the vertical axis determines the number of building stories
as well. Once the lines are clustered, cluster boundaries are used to develop a dense representation
of gradient overlaps (Figure 29d). The overlapping squared regions are then extracted from the
dense clustered lines (Figure 29e) and their dimensions are expanded by 25% to create bounding
boxes around the predicted elements (Figure 29f). The expansion is in fact a safety measure to
ensure the bounding box covers the element by making up for possible inaccuracies in the process
of predicting an element’s location.

Figure 30. Diagrams of average silhouette width vs. number of clusters of the façade in Fig. 4.
According to the diagrams, the horizontal and vertical gradients have to be divided into 8 and 4
clusters respectively.

77
10.4 Detecting Shape of Façade Elements
Following the prediction of elements’ locations, identified regions are studied in order to detect
the shape of the elements. Window shapes are assumed to be rectangular and/or arched as they are
predominantly seen on buildings façades [77]. Accurate detection of window shape could be
challenging if it is occluded by another object. Vegetation (especially trees) are considered as one
of the main sources of occlusion in a façade reconstruction process [75,79]. Therefore, our method
incorporates a vegetation detection module in the shape detection process.
For every region of predicted façade elements, we use color and texture features to distinguish
vegetation from façade elements. Our proposed vegetation detection consists of a training and a
testing stage. For the training stage, we selected 26 images of different vegetation from the ECP
database and the collected images from the USC buildings that were used in the pilot experiment.
For every training image, the RGB color space is converted to CIE-Lab. The CIE-Lab color space
provides a better distribution of colors for clustering purposes and is affected less by changes in
illumination [101]. The values in “a” and “b” color channels of every training image are collected,
concatenated, and assigned to a 20 bin histogram and then normalized to have a value in the range
of [0-1]. In addition to the color feature, we derive the Maximum Response 8 (MR8) filters from
Root Filter Set (RFS). The RFS filter bank is comprised of 38 first and second derivatives filters
that are applied at multiple scales and orientations. The MR8 filters [78,110,137] are the maximum
responses of RFS across all orientations at every scale, which produces 8 filter responses. These
filters are commonly used to capture texture patterns and surface contours for material recognition
purposes. We convolve the training images with the MR8 filters and then concatenate them to
create a dictionary of the vegetation texture class using 20 clusters (equivalent to 20 textons). The
two normalized histograms of “a” and “b” color channels are then added to the vegetation texture
dictionary. Note that the numbers of bins and clusters for generating feature vectors were
determined based on a trial and error process on the experimental data. However, if a façade is
occluded by a vegetation type that has no similar instance in the training dataset, then the
vegetation dictionary has to get updated with the new information. We selected the smallest
number of bins/clusters that resulted in a distinction between different classes of textures. The
objective for selecting a small number of bins/clusters was to keep the dimensionality of feature
vectors low for a faster classification.
The testing stage in automated vegetation detection includes going over every region of predicted
façade elements, and comparing their color and texture features with those derived from the
training data. However, every region has to be segmented into sub-regions prior to the feature
extraction process in order to separate the vegetation from other areas. Based on the visual
inspection of the façade images in the pilot experiment, we made the assumption that on average,
every region, if occluded, is comprised of an element, wall area, and vegetation. This assumption
could be challenged if there are more than three types of objects in the predicted element region.
However, it enables simple detection of vegetation pixels in the image. Therefore, as the first step
in the vegetation detection process, every region is segmented into three clusters based on its CIE-
lab color features by k-means clustering. Next, the color and texture features of every cluster are
extracted to be compared with the training data. We compute the Euclidean distance between the
feature vectors of test and training data in order to determine whether the identified regions contain

78
vegetation. If the distance is less than a certain threshold (we set the threshold to 0.25 empirically
based on our observation), the region is detected as vegetation. The regions that contain vegetation
are marked and excluded from the subsequent shape detection process since they could cause
erroneous results. At the end of our façade reconstruction method, the regions with vegetation are
revisited and matched with the element class to which their feature descriptor has the least
Euclidean distance.
In order to characterize the shape (rectangular/arched) of windows and doors, we use Histograms
of Oriented Gradients (HOG) feature descriptors that were originally described by [41] for
pedestrian detection. HOG features have been prevalently used for various object detection
purposes, specifically where extracting the characteristics of the object’s shape format is sought
[22]. Similar to the configuration that was used by [41], every predicted region of an element is
divided into blocks with 50% overlaps. Each block consists of [2×2] cells with [8×8] pixels. The
gradient magnitudes are then computed and quantized into a 9 bin (0-180°) histogram based on
their orientations. Finally, the histograms of all the blocks are concatenated to produce the HOG
feature descriptor. This process is repeated for a set of façade elements with known shapes (training
data) to train a linear Support Vector Machine (SVM) classifier. The HOG feature descriptors of
every region of element is then compared with the learned model to detect the shape of the element.
Figure 31 summarizes the steps that are taken to detect the shape of façade elements.

Figure 31. Overview of the shape detection process. Regions without vegetation are identified,
their HOG features are extracted, and their shape is determined by a linear SVM classifier.
(Building image acquired from ©Google)
10.5 Classifying Façade Elements
There is usually more than one type from every class of element on building façades. For instance,
there could be multiple windows that are rectangular but have different types of framings and/or
dimensions on a single façade of a building. Therefore, in order to add more semantics to the mock-
up models, the detected façade elements are classified by using a set of local and global features
that characterize elements’ color, shape, texture, and dimension. Extracting and integrating color,
texture, and other image features have been proven to be effective for recognition of shape and
material of objects on construction sites [34]. The list of features that are used for element

79
classification in our work is presented in Table 9. The reason for aggregating these features is that
elements, such as windows usually do not have features that could sufficiently characterize and
distinguish them from each other. Previous research showed that using a single feature type as a
standalone solution is ineffective for detection and classification purposes [151]. In addition to the
features, such as color histogram, HOG, and MR8 filters that are used for the shape detection
process, we add grayscale Gist and Scale Invariant Feature Transform (SIFT) features along with
the dimensions of the element’s bounding box to better distinguish the façade elements. Gist
descriptors characterize the gradient information of an image and are widely used for context-
based scene and object recognition [133]. Computation of Gist descriptors starts by convolving
every region of an element with 32 Gabor filters at 4 scales and 8 orientations, resulting in 32
feature maps. Feature maps are then divided into 16 blocks and for each block, the average feature
value is computed. Averaged values are then concatenated to generate a Gist descriptor
(dimension: 16×32 =512). We also extract SIFT features [81] from every region to better
characterize the textural attributes of the elements. The fact that SIFT features are robust against
changes in illumination, rotation, and viewpoint makes them suitable for façade images that have
variation in lighting.
Once the aggregated feature descriptor of every region is generated, they are concatenated to create
a matrix of features for all elements in a façade’s image. Next, the principal components of the
feature matrix are derived using the corresponding covariance matrix to reduce the dimension of
the vectors for a more efficient clustering process. We use k-means clustering to quantize the
reduced feature matrix. The integration of image features, quantization method, and the
development of feature dictionary at different stages of the methodology are inspired from the bag
of features method proposed by [40,76]. Similar to the element’s location prediction process,
different numbers of clusters are evaluated to determine the optimal clustering configuration using
the Silhouette method. Feature descriptors of the elements that were identified to be occluded by
vegetation are then computed likewise and their distance to different clusters’ centroids is
computed to determine their class. As the result, each façade element is labeled with a class
number. Next, the elements are compared with a database of window and door images (sampled
from ECP database) by using an SVM classifier to determine whether they represent a window or
a door. Since there are only two possible outcomes in detecting whether an element is a window
or a door, the hyperplane in SVM classifier is suitable for binary classification as compared to
other types of classifiers designed for identifying multiple classes. As the result of the above
classification stages, the elements are augmented with semantics of (1) whether the element has
rectangular or arched shape, (2) whether the element represents a door or a window, and (3) the
class number (of window/door) that the element belongs to (in case of having multiple types of
windows or doors).
In the final step of the classification, we use Hough transform features of rectangular elements and
gradient orientations of arched elements in order to detect the approximate boundaries of the
detected elements. Figure 32 shows the classification results for a façade with separated elements
and Figure 33 summarizes the overall steps of the elements classification process.

80

Table 9. Features used at different steps of façade reconstruction
Purpose Feature Dimension
Predicting the location of the elements
Vertical Sobel Operator Image Height
Horizontal Sobel Operator Image Width
Detecting vegetation
Color Histogram (CIE 1976 Lab) 20 × 2
MR8 Filters 20
Detecting the shape of the element HOG 360
Classifying the elements
Color Histogram (CIE 1976 Lab) 20 × 2
Grayscale Gist 512
HOG 360
MR8 Filters 20
SIFT 128
Bounding Box 2
Detecting the element’s framing
Hough (Rectangular) N/A
Gradient Orientation (Arched) N/A

81

Figure 32. Façade elements are classified based on their feature descriptors. For this specific façade
example, different instances of each class of windows are identified through color-coded bounding
boxes. (Building image acquired from ©Google)

Figure 33. Summary of the overall steps for classifying the façade elements. (Building image
acquired from ©Google)

82
10.6 Defining Split Grammar for Reconstructing a Building’s Façade
Once a building’s façade structure is determined, the parametric information of the detected
elements could be coupled with the procedural modeling methods to generate the 3D mock-up
model of the building. Procedural modeling encodes the structure of the façade as a set of
parametric rules and is known to be an efficient method for rapid creation of 3D models. Each rule
transforms a basic shape to another through a sequential procedure. A simple example of this for
a building’s façade is starting off with a rectangle and splitting it along the vertical and horizontal
directions to different floors and tiles of windows and doors [132]. We use a target building’s
footprint (referred to as building lot in the CGA language) as the starting node for creating the
mock-up model. Building footprints could be acquired either from online sources such as Google
Maps and Mapbox or from building’s 2D drawings, if available. The height of the building is
approximated by having the knowledge of the number of floors and the average floor-to-floor
height. The number of floors is determined based on the number of rows of the detected windows
and the average floor height is derived according to the building’s design typology information
that is available in [42].
Building’s mass extrusion is created based on the estimated height while a split grammar is defined
for each façade segment to reconstruct the building’s façade structure. We use the boundary lines
of the detected façade elements for splitting the façade across the height and width directions. The
split grammar is encoded in the CGA language and implemented in ESRI CityEngine to generate
the building’s mock-up model. Figure 34 summarizes the overall process of creating the mock-up
model using procedural modeling. The pseudocode for generating the building’s mock-up model
is shown in Algorithm 5. Note that the parameters of a building’s roof model are usually extracted
from aerial images of the building. Therefore, accurate roof modeling process is out of scope of
this study. We manually defined extrusions in ESRI CityEngine to model buildings’ roofs.
Algorithm 5. Procedural modeling for creating the mock-up model
1. Derive the footprint (lot) of the building
2. Estimated building’s height → attr. BuildingHeight
3. Lot → extrude (BuildingHeight)
4. for (every façade segment (𝑓 𝑖 )) do
5. Derive the boundary lines of the detected elements
6. face →
7. split (y) (y
0
r, ሺy
1
− y
0
ሻr, ሺy
2
− y
1
ሻr, … , ሺy
n
− y
n−1
ሻr, ሺh − y
n
ሻrሻ
{wall
y
.index(0) |Floor.index(0)| wall
y
.index(1) |Floor.index(1)| … |wall
y
.index(n)
|Floor.index(n)}
8. split (x) (𝑥 0
𝑟 , ሺ𝑥 1
− 𝑥 0
ሻ𝑟 , ሺ𝑥 2
− 𝑥 1
ሻ𝑟 , … , ሺ𝑥 𝑚 − 𝑥 𝑚 −1
ሻ𝑟 , ሺ𝑤 − 𝑥 𝑚 ሻ𝑟 ሻ

83
{𝑤𝑎𝑙𝑙 𝑥 .index(0) |Tile.index(0)| 𝑤𝑎𝑙𝑙 𝑥 .index(1) |Tile.index(1)| … |𝑤𝑎𝑙𝑙 𝑥 .index(m)
|Tile.index(m)}
9. for (i= 0; i= n; i++)
10. for (j= 0; j= m; j++) do
11. if (Floor(i).element == True && Tile(j).element == True) do
12. FloorTileIntersection.texture (𝑑𝑒𝑡𝑒𝑐𝑡𝑒𝑑 _𝑒𝑙𝑒𝑚𝑒𝑛𝑡 𝑖 ,𝑗 ሻ
13. end if
14. end for
15. end for
16. end for

Figure 34. Process for generating a building mock-up model: (a) Building’s footprint; (b) Footprint
is extruded based on the estimated height of the building; (c) Façade segments are split through
procedural modeling rules; (d) Façade is textured using the detected elements; (e) Building’s
mock-up model is generated.

84
10.7 Performance Evaluation Metrics
The proposed method’s performance was evaluated against a public façade image database, as
well as three case-study buildings. We used the ECP façade parsing database to examine the façade
element detection performance and benchmarked it against other related studies [39,83]. In
addition, we used the data of three case-study buildings (see Figure 35) on the USC’s main campus
to further validate our proposed method for generating mock-up models. We employed four
different performance metrics (true positives, false positives, true negatives, and false negatives)
to quantify the accuracy of the proposed method at different levels. We computed the precision
and recall values to evaluate the performance of elements’ location prediction and vegetation
detection steps.
For location prediction, true positives (TP) refer to the areas (areas are represented by red dashed
bounding boxes in Figures 29 and 36) that are correctly detected to contain elements (i.e.,
windows/doors). False positives (FP) are the areas that do not contain elements but are wrongly
detected to contain elements. False negatives (FN) refer to the locations that contained elements
but are overlooked by the algorithm. For the vegetation detection step, these performance
parameters are calculated using pixel by pixel comparison of vegetation detection results with
hand-labeled ground-truth. Performance of the element classification step is evaluated by
determining the percentages of images with correct segmentation of elements as well as images
with under and over segmentation. For instance, the façade in Figure 32 has three types of windows
(two types of rectangular and one type of arched) and the classification algorithm has segmented
the elements into three types. For the same façade, under-segmentation and over-segmentation
refer to the situations where the elements are wrongly segmented into less than three and more
than three types, respectively. The accuracies of window and door detections were also determined
by dividing the number of correct detections by the total number of doors/windows. The detection
accuracies were then compared with the findings of other studies. We have also provided
quantitative and qualitative comparisons between the generated mock-ups and the ground-truth
BIM models of the case-study buildings. The quantitative comparison focuses on quantifying the
discrepancies of the mock-up model in terms of positioning and measurements of the elements as
well as the dimensions of the exterior walls. For instance, the height and width of a window of a
mock-up model are compared with those of the ground-truth BIM model. On the other hand, the
qualitative comparison (visual inspection) identifies the main elements from the BIM model that
are missing in the mock-up model.

85

Figure 35. Case-study buildings, their BIM models, and the generated mock-ups. (Top two
building images acquired from ©Google)
10.8 Results and Discussion
The proposed façade reconstruction method was implemented in Matlab and the performance was
evaluated on a 64-bit workstation with Intel Core i7 CPU at 3.00 GHz, 32.0 GB RAM, and
NVIDIA Quadro K2000M graphics card. A total of 596 door and window images was randomly
acquired from the ECP database (equivalent to 20% of the total elements in the database) for
training the SVM classifiers in the shape detection and element classification steps. The data from
the case-study buildings were left out for the testing stage. Computational time for the training
stage was 39 min and the average testing time was measured as 8 secs per image. The process of
extracting split grammars for façades and developing of mock-up models took an average time of
2 hrs per building. Sample results for detecting the façade elements’ locations are shown in Figure
36. Table 10 presents the detailed performance of the proposed façade structure recognition
method at different steps. Note that the case-study (1) building was not occluded by any vegetation,
therefore the corresponding cells are shown as “N/A”. As seen in Table 10, the element location
prediction step has performed with 100% precision for case-study (2), which has connected
elements. In general, buildings with connected elements usually have less diversity in elements
layout, which makes the process of predicting the elements’ locations easier. In the façades with
separated elements, the precision of location prediction drops by 5-20% where there is an
irregularity in the façade’s layout that is filled with misleading features. For instance, when there
is a missing window on a row of windows and its vacant area contains strong horizontal and
vertical gradients, the area is falsely detected as a potential window.
The vegetation detection step has performed with over 85% precision in two case studies, however,
the performance has dropped in cases, where the illumination and poor resolution effects made the

86
vegetation area in the images featureless. In addition, the method was inaccurate in classifying the
elements that had more than 50% occlusion (i.e., more than 50% of the pixels of an element’s
bounding box belonged to vegetation) due to having limited features from the elements. In general,
we tried to reduce the impacts of vegetation and wall texture in two stages: (1) the image pre-
processing step by convolving the image with a Gaussian filter; and (2) the element location
prediction step by extracting only strong gradient responses from the image gradient profile. On
the other hand, the element shape detector performed with more than 95% precision and recall in
most cases verifying the fact that HOG features are suitable in distinguishing arched and
rectangular shapes.

Figure 36. Samples of element location prediction for two datasets (ECP database:
http://vision.mas.ecp.fr/Personnel/teboul/data.php)

Our findings show that classification accuracy dropped in images that have areas with direct
reflection of light. This is due to the fact that high light reflection results in weak gradient traces.
In addition, the elements were over-segmented in the cases where some windows from a similar
class had an awning that entirely obstructed their framing. The obstruction results in producing
different feature descriptors for instances of a single class, therefore, the elements were not
grouped in the same cluster. Moreover, our method resulted in inaccurate detections in facades
from the ECP database that had shopping stores with multiple windows on the ground floor. The
store windows did not have the same pattern as the rest of the building and since our method does
not use a prior grammar, it was not able to detect them accurately.
Table 11 shows the results for window and door detection accuracies. The detection accuracies
were generally higher for the case-study buildings as oppose to the ECP database due to multiple
reasons: (1) the images were collected with a better angle, (2) the images had better quality in

87
terms of resolution, color, and lighting, and (3) the façades were less diverse and congested in
terms of elements types and layout. For the ECP database, the detections are 13% more accurate
compared to the results reported by [83], however, they are 6% lower in average compared to the
results reported by [39]. Nonetheless, our method does not require a multi-stage training and more
importantly, it does not require using of a priori grammar. In addition, our method adds more
semantics to the window and door detection by classifying them into different types with respect
to their shapes and dimensions.

Table 10. Performance results of the proposed façade structure recognition method for different
datasets (The numbers are percentages)

Location
Prediction

Vegetation
Detection
Shape Detection Element Classification
Dataset Precision Recall Precision Recall Rect. Arch
Under
Seg.
Corr.
Seg.
Over
Seg.
ECP database 76.98 80.43

80.00 75.00

98.10 97.08

15.38 66.35 18.27
Case-study (1) 90.38 92.16

N/A N/A

100.00 85.71

22.22 77.78 0.00
Case-study (2) 100.00 98.38

85.29 80.56

100.00 100.00

0.00 87.50 12.50
Case-study (3) 95.92 92.89

88.24 83.33

97.94 100.00

6.45 90.32 3.23
Total Average 90.82 90.96 84.51 79.63 99.01 95.70 11.01 80.48 8.50

Table 11. Window and door detection accuracies for different datasets

Detection Accuracy (%)
Dataset Window Door
ECP database 79.47 77.66
Case-study (1) 88.89 84.62
Case-study (2) 98.34 81.82
Case-study (3) 95.26 100.00
Total Average 90.49 86.02

88
Table 12 shows the results for the quantitative comparison of the mock-up models with the ground-
truth BIM models that were obtained from the USC’s Facilities Management. For each case-study,
the discrepancies between the two models were manually measured and reported as average error
range values. As seen on Table 12, the models have dimension error as low as 3 cm and a maximum
of 40 cm at worst case with an average error of 14 cm. The large errors are mainly attributed to
the dimensions of the walls especially their height because they were estimated based on the
number of floors. However, the relative positioning of the elements has been estimated with a
better accuracy. Generally, the mock-up model for case-study (2) has more accuracy in comparison
to the other two models. This is due to the fact that case-study (2) has a façade with connected
elements that are less diverse in comparison to the other two case-studies. The mock-up model for
case-study (2) satisfies LOD 1 at the element position level as defined by U.S. GSA, which allows
dimensional deviation of lower than ±51 mm [59]. In addition, semantics (i.e. elements’ types,
shapes, etc.) in the mock-up model could potentially improve the quality of as-built 3D models
created in a model-assisted SFM [64] or be employed in a data quality-driven scan planning
framework [91,150].

Table 12. Average error range (m) of the mock-up model as compared to the BIM model
Item Case Study (1) Case Study (2) Case Study (3)
Element's position ± (0.05 - 0.15) ± (0.03 - 0.12) ± (0.08 - 0.18)
Element's measurements ± (0.08 - 0.18) ± (0.05 - 0.10) ± (0.07 - 0.16)
Wall's dimensions ± (0.12 - 0.40) ± (0.08 - 0.22) ± (0.10 - 0.36)

The qualitative comparison of mock-up and BIM models are presented in Table 13. Overall
geometric layout of the building and the façade properties, such as number of floors and the
elements positioning, were modeled accurately. Nonetheless, ornaments and embellishments on
the façade were not detected and their modeling is left out as future work. In addition, exposed
structures, such as exterior stairs or the bracings in case-study (1), were not modeled, however,
ESRI CityEngine, in which procedural modeling was implemented, allows manual and customize
adding of complex structures. Lastly, the roofs were not modeled accurately since ground images
do not provide sufficient coverage for accurate detecting of their type.

Table 13. Qualitative comparison of mock-up models and BIM models
Inconsistency Subject Case Study (1) Case Study (2) Case Study (3)
Modeling setbacks Accurate Accurate Inaccurate in balconies

89
Modeling roof Inaccurate slope and form Accurate Inaccurate slope
Number of floors Accurate Accurate Accurate
Placement of ornaments Not modeled N/A Not modeled
Modeling exposed structure N/A Braces missed N/A

10.9 Limitations and Conclusions
This chapter presented an automated method for recognition of building façade elements using
rectified ground images. The method first detects the arrangement style (e.g., connected or
separated) of façade elements. Elements’ locations are then predicted based on their arrangement
on a façade and using the gradient profile of the façade’s image. Windows and doors are detected
and classified by extracting a set of local and global feature descriptors. The façade’s split grammar
is derived based on the layout of the elements on the building’s façade. Procedural modeling was
used to generate the 3D mock-up model of a building using the split grammar. The method’s
performance was evaluated using a public façade image database as well as three case-study
buildings. An average accuracy of 88.25% was achieved in detecting windows and doors from the
testing datasets. The generated mock-up models are accurate in representing the number of floors,
exterior walls, and the layout of façade elements. The average dimension error of the mock-up
models was measured as 14 cm. In addition, the training stage of our method was completed in
less than an hour and the testing of each façade image took less than 10 seconds in average.
Moreover, in this chapter, we introduced a method that detects the vegetation prior to element
detection in order to avoid element misclassification. As a result, the detected area, if occluded
with vegetation, is excluded from the subsequent classification stages and is studied separately. To
the best of my knowledge, current studies do not incorporate the arrangement style of façade
elements (i.e. connected/separated windows) into the façade parsing process. Our method detects
a façade’s arrangement style in the beginning and parses the façade accordingly. Lastly, the
presented method enables the production of grammar rulesets that could be used to rapidly
generate façade structures for the test data and other buildings with similar façade layouts.

90
Chapter 11 TOWARDS ASSESSING THE QUALITY OF SCAN DATA:
IMPACTS OF NOISE AND POINT CLOUD DENSITY ON THE
ACCURACY OF BUILDING MODELS FROM POINT CLOUDS

In this chapter, we use the mock-up models of buildings to generate synthetic scan data with quality
issues. The quality issues are modeled by replicating varying values of point density, and adding
different levels of noise to synthetic data. The objective is to understand the synergies between the
as-built modeling accuracy, building’s geometry, and the data characteristics (noise and point
density) that represent quality issues. A novel method is introduced for quantifying the impacts of
point density and noise using synthetic scan datasets that are generated based on the geometric
properties of the buildings. The synthetic data generation method starts by sampling points on the
surfaces of mock-up models to generate 3D point clouds. Next, a set of 3D features are extracted
to represent the geometric properties of the buildings. A classifier is then used to label every point
on the building’s point cloud as wall or edge. Given the labeled point cloud, appropriate types of
noise are applied at every point. The noisy point clouds with varying point densities are then
processed to regenerate the exterior of mock-up models. Next, each wall and opening component
of the generated model (referred as “mock-up as-built” hereafter) is compared to its corresponding
component in the original mock-up models. The comparisons were then studied to quantify the
impacts of noise and point density on the accuracy of the mock-up as-builts. The same process was
repeated to generate 3D architectural models of buildings’ exterior (referred as “as-built”
hereafter) from the real-life scan data of the case study buildings. The as-built models were then
compared to the buildings’ exteriors using their ground-truth BIM models. Finally, the results from
studying the real-life data were compared to those from the synthetic data to evaluate the validity
of the proposed impact analysis, and the potential of using such method for assessing the quality
of scan data.
11.1 Need for Scan Data Quality assessment
Traditional surveying methods such as using measuring tapes and total-stations are proven to be
time-consuming, subjective, and often inaccurate particularly on large-scale facilities [131]. Laser
scanning surpasses photogrammetry-based techniques in terms of its performance especially when
project requirements enforce documenting with millimeter-level accuracy [36]. However,
benefitting from the precision of laser scanners is not assured unless high-quality data are collected
through an accurate scan planning and quality assurance process [14]. The required precision of
scan data collection in terms of accurate capturing of the geometric characteristics of facilities
could vary depending on task and project owner’s requirements. Tasks such as documenting
artifacts for historic preservation of ancient building facades may necessitate high precision
geometric capture (less than ±6 mm of measurement error tolerance), whereas other tasks such as
urban design and planning usually require lower precision (±51 mm of measurement error
tolerance) [59]. Other applications such as generating BIM models for controlling construction
progress, and documenting as-built condition of buildings usually requires precision in the range
of 6 to 51 millimeters [60].

91
The process of generating an as-built BIM model is commonly broken down to three main steps
of data collection, data pre-processing, and geometric modeling [96,131]. The errors in the
generated as-built models could stem from the data collection practice, geometric modeling
process, or both [20]. Geometric modeling consists of automated recognition and modeling of
geometric primitives such as planar and cylindrical surfaces, as well as manual semantic modeling
of objects such as windows, doors, structural components by identifying their categories and
specific material properties [20]. Geometric modeling process can take as long as multiple weeks
or even months [44]. In addition, scanning specialists often realize during the geometric modeling
process, that there is a need to revisit project site for further data collection due to data quality
deficiencies. Therefore, implementing a data quality assessment method during data collection
could potentially save time and cost by assisting the surveying team with the identification of
quality issues early-on in the process.
Previous studies have made efforts to improve the scan data collection methods and automate the
geometric modeling processes. Yet, data quality issues, such as missing data, low point cloud
density, and noise remain as challenges to generate accurate as-built models [44,131]. In order to
ensure the project-specific requirements for generated as-built models are satisfied, there is a need
for identifying the relationships of scan setting parameters (e.g. angular resolution, field of view)
and data characteristics (e.g. point cloud density, data completeness) either at the planning stage
or in-situ [14]. Anil, et al. [20] emphasized the necessity to design a scan data quality assessment
method that: (1) provides a comprehensive evaluation of accuracy and completeness of data based
on project requirements and desired level of detail (LOD); (2) unveils errors and quality issues as
the project progresses; and (3) demonstrates the potential issues in a comprehensible and timely
fashion. The challenges with designing such scan data quality assessment method are: first, there
is usually no access to an updated BIM model to be used as a reference for evaluating the data
quality; second, since there is a large variety in buildings’ geometry, using similar data quality
requirements for every building does not result in an accurate quality assessment; and third, the
impacts of data quality issues on the accuracy of as-built models have not been studied.
As a preliminary step toward designing a scan data quality assessment method, we focus on
quantifying the influences of point density and noise on the geometric modeling of walls and
openings on the buildings’ exterior. The quality issues resulting from missing data were not
investigated since the level and type of them are not deterministic or replicable especially when
large-size scanned objects such as buildings are studied. The process of scanning a building’s
exterior is limited to capturing surfaces rather than volumetric information. As a result,
information, such as the thickness of walls and depths of architectural elements on the façade,
could not be derived accurately. Since such information is necessary for generating a BIM model,
we limit the scope of this study to 3D modeling and leave the enhancement of 3D models with
BIM semantics for future work where the entire scan data of buildings are studied. Also, we
selected to study buildings’ exterior because their 3D modeling involves processes for which
existing studies have proposed automated methods. We surveyed the existing automated modeling
algorithms and employed the most commonly-used ones in our impact analysis to avoid deriving
results that are biased by proprietary or manual methods. Additionally, this allows the future
studies to use the findings of this chapter as a point of comparison when examining the influences

92
of scan data quality issues on similar types of built environments. Studying the impacts of quality
issues on modeling specific types and categories of objects such as windows, doors, as well as
structural and utility systems are reserved for a future study when the required geometric modeling
processes become fully automated.
The process of replicating and modeling different levels of point density and noise using real-life
data comes with several challenges. First, maintaining a precisely uniform point density across a
building’s exterior is not feasible especially across the building’s elevation. Second, scanning the
building’s exterior with multiple settings (i.e. scanner’s resolution and distance) is highly time-
consuming especially when studying a wide range of point density is targeted. Third, covering the
entire surfaces of a building is often impossible due to the job-site conditions, existence of
occlusions, and accessibility limitations. Fourth, in order to apply different levels of noise to a scan
data, there is a need for a baseline scan data of the building’s exterior with zero noise to be used
as ground-truth data. Nonetheless, it is not feasible to retrieve a scan data that covers the entire
building’s exterior without containing any noise. Therefore, in this chapter, in order to replicate
and study different conditions of point density and noise levels, we used synthetic datasets that are
generated based on the geometric properties of buildings’ 3D mock-up models. Previous research
has suggested that synthetic data could provide a viable alternative to real-life scan data, if they
represent the primitive surfaces of objects, and become contaminated with noise levels similar to
real-world datasets [106]. Thus, we hypothesized that the synthetic data generated from the 3D
mock-up models of buildings could be used to study the potential impacts of noise and point
density on the 3D modeling of building’s exterior in real-life data. This hypothesis is evaluated
and quantified by comparing the errors of models regenerated from synthetic data with the errors
of models regenerated from real-life scan data. The advantage of using mock-up models in a scan
data quality assessment method is that the impacts of noise and point density are integrated with
the specific geometric features of buildings.
11.2 Point Density Modeling
Point density is usually considered as a primary feature by which the quality of a scan data is
described [60,74]. Using geometry and trigonometry, Ahn and Wohn [14] derived an equation for
studying the changes in incidence angle at candidate scan positions produced by a scan planning
algorithm. Equation (1) is a different representation that relates the parameters of scanning
distance, scanner’s angular resolution, and incidence angle to point density:
𝐷 =
𝑙 𝜑 𝑟𝑒𝑠 𝐶𝑜𝑠 2
Ѳ
Eq. (1)
where D stands for point density, l denotes the scanner’s distance from target, 𝜑 𝑟𝑒𝑠
represents the
scanner’s angular resolution, and Ѳ refers to the incidence angle. Point density could also be
described as the distance between two adjacent points on the point cloud. Therefore, hereafter in
this chapter, point density will be referred to as point spacing.
In order to generate synthetic scan data, first a realistic range for possible values of point spacing
needs to be determined. The range could be derived based on the values that the parameters on the
right side of Eq. (1) could take. We assume that the scanner is placed in positions with the distance
range of 5 to 150 meters from the building. In addition, similar to the previous studies, the

93
maximum incidence angle of the laser was set to 70° [14,27]. The incidence angle is limited to a
specific threshold since the current scanning practice uses overlapping and sequential data
collection. Lastly, the angular resolution of 0.06° (equivalent to 0.01 m × 0.01 m point spacing in
scan data when the laser scanner is in 10 m range) was selected as the medium resolution
considering that it is commonly adopted for outdoor scanning [117]. Four other resolutions were
selected proportional to 0.06° to allow for data with sparser and denser qualities. The scan
parameters and the values that were used for them are listed in Table 14.
Table 14. The scan parameters and their expected values for determining possible point spacing
in synthetic scan data
Scan parameter Expected values
Scanner's distance (m) 5, 10, 15, 20, 30, 40, 50, 70, 90, 100, 120, 150
Incidence angle (°) 0, 10, 20, 30, 40, 50, 60, 70
Scanner's angular resolution (°) 0.01, 0.03, 0.06, 0.09, 0.12

The integration of the parameters in Table 14 results in 480 different point spacings ranging from
0 to 2672 mm. Nonetheless, the point spacings are not evenly distributed across this range. In order
to identify the distribution model, we computed the histogram of the data using 100 bins. A two-
term power series model was fitted to the histogram data to draw an equation (see Figure 37) for
relating point spacing to its frequency of occurrence. The coefficients of the equation were
determined based on a 95% confidence interval, resulting in a R-squared value of 0.98.

Figure 37. The frequency distribution of different point spacings, and the fitted power series model.
As seen in Figure 37, the smaller point spacings have a relatively higher frequency as oppose to
large point spacings. Since generating synthetic scan data with all possible point spacings is a time-
consuming task, we chose to study the point spacing values that fall under the 90
th
percentile of
the computed frequencies. The value of point spacing at the 90
th
percentile is approximately 250
mm. The number of point spacings that were used for this study was then narrowed down to 35,

94
according to the following selection rationale: for the range of 0-100 mm, a point spacing was
selected with 5 mm increments excluding zero (19 point spacings); for the range of 100-250 mm,
a point spacing was selected with 10 mm (14 point spacings). Note that the selection intervals were
inspired by the fact that point spacings smaller than 100 mm have larger frequency.
After the selection of point spacing values, the mock-up models of the case study buildings were
imported into the ©CloudCompare software for generating synthetic scan data. The point sampling
tool in the software was guided by the point spacing values to sample points on the mesh models.
As a result, for each case study building, 35 synthetic scan data with different point spacing were
generated.

11.3 Noise Modeling
The uncertainty in laser scanner’s measurements is commonly referred to as noise. Noise is
generally decomposed into random and systematic components. The former impacts the data by
causing fuzziness/thickness, and the latter results in repetitive offsets in the data. Systematic noise
in data usually originates from improper scanner calibration or a dysfunction in the scanner’s
internal components [74]. On the other hand, random noise is the result of fluctuations in the
strength of the signal returning to the scanner. The signal’s strength is impacted by several factors
such as the scanner’s distance, incidence angle, weather conditions, ambient lighting conditions,
and surface properties of the object being scanned [28]. The object surface properties are
comprised of geometric features (e.g. orientation), and material features (e.g. reflectivity, color,
roughness). As the result of analyzing the influences of aforementioned factors on the scan data,
previous studies approximated the behavior of random noise as Gaussian, and independently
distributed at every data point [43,106,124,149]. Nonetheless, the magnitude and behavior of noise
change around sharp edges and transparent surfaces (e.g. glass). For instance, in sharp edges, the
average magnitude of the noise could become one to two times larger than planar surfaces with
typical reflectivity (20-80% reflectance) [113]. In addition, the noise magnitude distribution
becomes close to a Laplace distribution. Findings from a recent study by El Houari and El Ouafdi
[47] showed that noise behavior could be characterized more accurately by the General Gaussian
Distribution (GGD).
We used GGD for noise modeling since its shape parameter (β) allows conversion to both Gaussian
and Laplace distributions. The proposed noise modeling method is based on the following set of
assumptions: (1) There are no environmental impacts (i.e. weather and lighting conditions); (2) the
scanner is calibrated (i.e. no systematic noise); and (3) the material properties on the inner surface
of the building’s wall only generate Gaussian noise. A detailed noise modeling that takes the entire
sources of noise into account is out of the scope of this research. Nonetheless, the impacts of the
wall edge and window trimming are incorporated in the noise modeling. Therefore, the exterior of
the building is divided into two classes of wall and edge. The Gaussian distribution (equivalent to
GGD with β = 2) was used for modeling noise on the wall area, whereas for the edge points, the
GGD model was used to reflect the noise more realistically. The ranges of the noise model
parameters (Table 15) were selected based on the experimental findings of previous studies
[28,113,125]. Note that the mock-up models do not provide information on the blinds. Therefore,

95
we assumed that the windows do not have blinds and that the glazing area does not add any non-
Gaussian noise. Also, note that the state-of-the-art TLSs have measurement precision of 5 mm and
higher; therefore, the range of the noise parameters in our noise modeling method will cover
additional measurement errors up to six times more than the scanner’s measurement precision.

Table 15. Noise modeling parameters
Class-Noise Model Model Parameter Value
Wall-Gaussian Standard Deviation (SD) 0.1, 0.5, 1, 2, 4, 6, 8, 10, 20, 30 (mm)
Edge-GGD Shape Parameter (β) 2, 1.75, 1.5, 1.25, 1, 0.75, 0.5, 0.25, 0.1, 0.05

We used an eigenvalue-based feature extraction and classification method for distinguishing wall
and edge areas on the mock-up point clouds with different point spacings. The eigenvalue-based
features capture the local geometric properties of a point through studying the covariance matrix
of its neighborhood [142]. If a small value is chosen for the number (K) of neighboring points, the
locality of the features is increased and they would represent the properties of a limited region. On
the other hand, a large K value will overlook the local properties. After a thresholding process, we
empirically determined that studying 20 neighbors of every point sufficiently represents the point’s
local geometric properties for noise modeling as the points all fall on a local plane.
For every point in the point cloud, the covariance matrix is computed and the three eigen-values
𝜆 1
, 𝜆 2
, and 𝜆 3
(𝜆 1
> 𝜆 2
> 𝜆 3
) of each region are derived. The concatenation of features was
adopted from Wang, et al. [142]. In addition to the eigenvalue-based features, we integrated the
depth difference ratio to the feature space. The depth difference ratio was derived specifically to
support distinguishing wall from wall edges by bolding the local depth fluctuations in the
surrounding of edges. Algorithm 6 describes the types and computing processes of the features
that were used for the proposed noise modeling.
Algorithm 6. Computing eigenvalue-based features
1. For every point (𝑃 𝑖 ) in the 3D point cloud
2. Find k nearest neighbors
3. Compute covariance matrix: 𝐶 𝑝 =
1
|𝑁 𝑝 |
σ ሺ𝑞 − 𝑝 ̅ሻሺ𝑞 − 𝑝 ̅ሻ
𝑇 𝑞 ∈ 𝑁 𝑝
4. Compute eigenvalues of 𝐶 𝑝 and normalize them: 𝜆 𝑖 =
𝜆 𝑖 σ 𝜆 𝑖 𝑖 i=1,2,3
5. Compute following features:
6. Structure tensor omnivariance: √∏ 𝜆 𝑖 3
𝑖 =1
3

96
7. Structure tensor anisotropy:
𝜆 1
−𝜆 3
𝜆 1

8. Structure tensor planarity:
𝜆 2
−𝜆 3
𝜆 1

9. Structure tensor curvature:
𝜆 3
𝜆 1
+𝜆 2
+𝜆 3

10. Structure tensor linearity:
𝜆 1
−𝜆 2
𝜆 1

11. Depth difference ratio:
𝐷 𝑚𝑖𝑛 𝐷 𝑚𝑎𝑥

12. Concatenate computed features
13. end for

The feature classification starts with a training stage that uses features from a set of manually
labeled point clouds. The training dataset contained 150 examples of façade elements from the
synthetic data of the case study buildings. We tested the performance of k-nearest-neighbor (KNN)
and linear support vector machine (SVM) classifiers as they are commonly used in point cloud
classification. A 5-fold cross-validation scheme was adopted to train and evaluate the classification
performances. The average classification accuracies of KNN and SVM were 92.5% and 91.1%,
respectively. Thus, we used the KNN classifier to automatically label the synthetic dataset of the
case study buildings. Next, different levels of noise were added to the synthetic data based on the
label of each point. As a result, 350 synthetic data with varying point spacing and noise level were
generated. Figure 38 shows the synthetic data of the case study building (3) receiving noise at three
magnitudes.

97

Figure 38. Applying different levels of noise to the building’s synthetic scan data. The noise level
increases from left to right. (Top row: top view of the building, Bottom row: front view of the
building)

11.4 3D Modeling Process
We employed a 3D modeling process that is defined based on surveying the existing point cloud
processing algorithms [96,106,107,131,147]. The process consists of five steps: noise filtering,
data discretization, data segmentation, wall modeling, and opening detection. The 3D modeling
process starts by removing random noise from the scan data. For every point, the Euclidean
distances of the K-nearest local points are studied. With the assumption that the distances follow
a Gaussian distribution, the points that are more than one standard deviation (SD) farther from the
mean are removed from the point cloud. The noise removal algorithm, as well as the selection of
appropriate parameters were adopted from [106]. Next, the sizes of the point clouds are reduced
by discretizing them through a voxelization process. Theoretically, a denser point cloud represents
a higher quality, however, current 3D modeling algorithms are not designed in a capacity to benefit
from such point density. Note that even with high-speed processors, the data processing time for
3D modeling can take from several hours to days. Therefore, current 3D modeling algorithms
discretize point clouds with voxel sizes that have average dimensions in the range of 15 to 30 mm
[44,147]. Thus, we chose the voxel size of 15 mm to discretize only the scan data with smaller
point spacing. Nevertheless, one should note that in point cloud discretization, the centroid of each
voxel replaces the points in the voxel. Therefore, as the point cloud gets denser, its discretized
version still provides a more accurate representation of the geometric information.
The 3D modeling process continues by segmenting the discretized point clouds into patches of
points through a region growing segmentation. We used the open-source region growing
segmentation algorithm available on the Point Cloud Library [107]. Next, in order to extract the

98
building wall elements, a planar model is fitted to the large segments using the RANSAC method.
The normal vectors and the parameters of the planes, along with the outline contours of the
segments are used to generate mesh models of the walls. For the opening detection step, we adapted
the algorithm proposed by [147]. The detection consists of two stages. In the first stage, the eigen-
value based features of the wall segments are computed to detect the edges on the wall interiors.
A bounding box is created around the connected edges. The data points in the bounding box are
considered to create an opening candidate. Next, the area and aspect ratio of the bounding box are
computed and added to the eigen-value feature space of the points inside the bounding box. The
second step uses a KNN classification followed by a template matching method to complete the
detection process. We created a training dataset with 80 examples of openings. The classification
compares the features of the opening candidate with the training dataset and selects the three
closest opening examples. Next, the opening candidate’s center of gravity is aligned with the three
selected examples to enable a template matching process. Finally, the example with the minimum
Euclidean distance is selected as the actual opening.

11.5 Findings and Discussion
The scan data quality criteria established by the U.S. GSA are used to study the impacts of noise
and point density on the accuracy of the generated 3D models. GSA has defined two criteria for
describing the quality of scan data: level of accuracy (LOA) and LOD. The former refers to the
maximum tolerance of deviation from the ground-truth positioning and dimension (i.e.
measurement error), and the latter represents the smallest detectable artifact (i.e. point spacing) in
the scan data. In this study, the LOA criterion is controlled at two scales of building-level and
component-level. At the component-level, we study the errors from wall modeling and opening
detection across the entire building. The errors are computed for each individual wall and opening
on the three case study buildings. Since the 3D models are generated based on the point clouds
with 35 different point spacings and 10 levels of noise, for every component on each building, 350
different errors are computed. The mean error values from all the components on each building
are then computed to represent the building-level LOA.
In order to perform a component-to-component analysis, first the generated 3D models were
aligned with the mock-up models. Since the 3D models were generated based on point sampling
from the mock-up models, they were already located on the similar coordinate system. The
corresponding components were then matched based on their distance and similarity in orientation.
The components that were wrongly matched were manually linked with their true
correspondences. We computed the Euclidean distance between corresponding nearest neighbors
to determine the discrepancies between the generated 3D model and the mock-up model. Previous
research has shown that Euclidean distance offers sufficient accuracy, and is computationally
efficient for comparing two correspondent models [20,31,106].
The computed errors of wall modeling at the building-level scale are presented in Figure 39. The
horizontal axis represents the point spacings of the synthetic point clouds, while the corresponding
noise levels are demonstrated as stacked colored points. The vertical axis represents the mean
Euclidean distance (i.e. error) between the walls on the generated 3D model and the actual mock-

99
up model. The allowable errors and the minimum artifact sizes defined by GSA for different LOD
levels are shown as well. Note that GSA has considered similar minimum artifact sizes for LOD
levels 3 and 4. Therefore, only three different LOD levels are presented on the horizontal axis.
As seen in Figure 39, the mean error of the wall modeling for case studies (1) and (3) starts by a
declining trend. The decline trend could be perceived as an abnormal incident since the error is
expected to increase as the point spacings become larger. However, the decrease is in fact an
indication of the combined impact of building’s geometry and data’s point spacing on the wall
modeling accuracy. We found that the large errors at the small point spacings were originated from
the modeling of adjacent walls with different heights. We traced back the error to the segmentation
step of 3D modeling. The region growing segmentation algorithm groups adjacent walls together
and detects them as one wall when the point clouds are too dense to distinguish between the
boundary of the walls. Note that the same error did not happen in case study (2) where the walls
have similar heights. This modeling error could potentially be avoided by using a segmentation
algorithm that adapts its parameters based on the point spacing of the point cloud. However, this
anomaly shows that the geometric properties of the buildings should not be disregarded when the
quality of scan data is evaluated. Also, note that the decrease in the error continued up to 30 mm
in case study (1) and 20 mm in case study (3). Therefore, even with adjusting the parameters of
3D modeling algorithms based on the point cloud’s point spacing, there are still possibilities for
generating inaccurate 3D models.
Overall, after point spacing 30 mm, the wall modeling error follows a similar pattern in all three
case studies (see Figure 41). Nonetheless, the error magnitudes are larger in the case study (3).
This is due to the fact that the building has several step-backs and variations in geometry which
have highly impacted the wall modeling accuracy in larger point spacings. Another observation is
that the error level has experienced less than 10 mm variation under different levels of noise. This
is reasonable because first, the noise level is reduced at the beginning of the modeling process, and
second, the RANSAC method that was used for plane fitting is known to be robust against the
noise in data.
Figure 40 presents the computed errors of opening detection at the building-level. The results show
that the case studies follow a similar error pattern. The maximum errors of the case studies (2) and
(3) are less than 5 mm different. However, the maximum error at case study (1) is approximately
15 mm larger than the other two. The difference stems from the variety in the opening shapes of
the building. The building has four different types of arched openings, whereas the other two case
studies have either one or zero type of arched opening. For point clouds with large point spacing
and high levels of noise, the openings were either misclassified or positioned at a wrong location.
The misclassification was worse for arched openings as the extracted features from noisy data
misguided the classifier. As oppose to the wall modeling, the impacts of noise on opening detection
is evident. In general, the feature extraction methods are highly sensitive to noise [106]. Therefore,
in addition to implementing a robust noise removal method, the noise level of the collected scan
data should be evaluated before the data processing stage starts. Also, as seen in Figure 40, for all
three case studies, there is a sudden jump in the error magnitude at 20 mm point spacing. After
this point, only the scan data with point spacings less than 30 mm and noise level with SD less

100
than 1 mm would qualify for opening detection with GSA’s LOD 3. However, based on Figure 39,
for wall modeling, the point spacing could go as high as 45 mm. This discrepancy in data quality
requirement is in agreement with the recent scan planning studies that customize the scan settings
based on the type of object being scanned.
Finally, for every component in the case studies, we computed the mean error (i.e. distance from
ground-truth) under the entire point spacing and noise level conditions. Figure 42 visualizes the
mean errors at the component-level. The visualization shows that similar elements do not
necessarily follow the same error pattern, which is another indication that the building’s geometry
influences the data quality requirements. The overall error patterns in the visualization reveal
specific geometry-related impacts including: (1) Adjacent walls with different heights have
experienced higher levels of modeling error; (2) Walls with narrower dimensions such as setbacks
on case study (3) are more impacted by variations in point density and noise level; (3) Arched
openings have experienced larger errors than rectangular openings; (4) Openings that are located
close to each other (e.g. case study (2)) are negatively impacted which is also amplified in the
presence of noise. However, the magnitudes of impacts for different case studies are different and
dependent on the specific geometry of them. Therefore, in order to have an accurate quantitative
analysis of the geometry impacts, the mock-up models of the buildings have to go through the
proposed impact analysis prior to the data collection.

101

Figure 39. Wall modeling errors at building-level for the case study buildings. Different levels of
LOD requirements defined by GSA are shown as dashed lines.

102

Figure 40. Opening detection errors at building-level for the case study buildings. Different levels
of LOD requirements defined by GSA are shown as dashed lines.

103

Figure 41. Comparative mean errors of wall modeling and opening detection for the three case
studies. (Note: each data point represents the mean results of different noise levels)

104

Figure 42. Visualization of component-level mean errors of wall modeling and opening detection

11.6 Real-life Data Collection
To evaluate the findings from the impact analysis on the synthetic data, we collected real-life data
from the same three case study buildings. We used a RIEGL VZ-400 TLS for the data collection.
The detailed specifications of the scanner could be found at [102]. Scan positions were selected to
have an average of 10 to 20 m spacing, and data were collected with 20% overlap (Figure 43).
However, as the buildings were partially occluded by vegetation and parked cars, quality issues
such as missing data and low point cloud density were still present in the data.
For each scan position, data were collected with four different angular resolutions of 0.03, 0.06,
0.09, and 0.12 degrees. The selected angular resolutions were similar as the assumptions in
generating the synthetic data. However, we did not collect data with 0.01° angular resolution since
it would have increased the data collection time exponentially. The scan datasets of every building
were registered using ©Autodesk Recap generating building point clouds with average 4 mm of
registration accuracy. As a result, for every building, four sets of point clouds with different point
density were produced.

105

Figure 43. Positioning of the TLS for collecting data from the case study buildings along with
samples of the collected data
11.7 Real-life Data Analysis
We compared the modeling errors from the synthetic scan data with the modeling errors from real-
life scan data of the case study buildings. The objective was to evaluate whether the study of the
impacts of point density and noise on synthetic scan data could be used as a guideline for assessing
the quality of real-life scan data.
For every scan data, the mean point spacing was determined by computing the Euclidean distances
of neighboring points in the local search. Also, the mean noise levels were determined using the
similar local search and computing the SD of neighboring point distances based on the assumption
of Gaussian noise. The mean point spacings and noise SDs of the collected scan data are presented
in Table 16.
Table 16. Real-life scan data statistics (point spacings are rounded to closest number)

Mean Point Spacing
(mm)
Point Spacing Range
(mm) Mean Noise SD (mm)
Scanner's
Angular
Res. (°)
Case-
study
(1)
Case-
study
(2)
Case-
study
(3)
Case-
study
(1)
Case-
study
(2)
Case-
study
(3)
Case-
study
(1)
Case-
study
(2)
Case-
study
(3)
0.03 3 5 4 (2,5) (4,8) (3,7) 0.090 0.114 0.108
0.06 6 11 8 (4,11) (7,14) (6,13) 0.145 0.341 0.247
0.09 13 15 14 (9,15) (12,18) (10,17) 0.362 0.422 0.407

106
0.12 17 22 19 (14,21) (19,25) (16,21) 0.427 0.479 0.452

The real-life scan data went through the same 3D modeling process that was used for the synthetic
data. However, prior to processing the data, the point clouds were manually cleaned by removing
the data points of unwanted objects around the buildings. Also, in contrast to the synthetic data,
the real-life data were affected by external occlusions. Since the impact of occlusions was not
studied for the synthetic data, the real-life data analysis did not incorporate the building
components that were partially or completely occluded. Nonetheless, building components that
had missing data due to reasons such as self-occlusion or oblique incidence angles were included
in the analysis.
Once the modeling process for the real-life data was completed, the models were manually aligned
with the ground-truth as-built BIM models of the buildings and the corresponding components
were matched through a semi-manual process. The errors in the modeled walls and openings were
computed through the same procedure that was used in analyzing the synthetic data. Next, the
mean values of point spacing and noise SD were computed for every component of the real-life
scan data so they could be matched with the synthetic data that have similar conditions. Once a
matching synthetic data was found, the computed model error from the real-life data (hereafter
referred to as actual error) was compared to the model errors from the matching synthetic data
(hereafter referred to as predicted error). Note that this comparison inherently introduces an error
in the calculations as for a large building component, the point spacing and noise could vary in
different positions. We reduced such an error by using scan positions that were located close to
each other (i.e. less than 15m apart) and single angular resolutions when scanning the buildings.
Our observation from the collected data was that the point spacing variation in the elements such
as walls did not exceed 3 mm. In addition, since the point spacings in the synthetic data were
multipliers of five and ten, we used linear interpolation and extrapolation for cases where the point
spacing of real-life data fell in between the ranges from the synthetic data. However, as seen in
Figures 39, 40, and 41, for point spacing in the range of 5 to 25 mm (mean point spacing range in
the real-life data), the model errors from two consecutive data (e.g. point spacing 5 and 10 mm),
are less than 1 mm different. Therefore, interpolation and extrapolation produce a maximum of ±1
mm error.
Figure 44 presents the predicted and actual errors at the building-level for the three case studies.
As seen, for wall modeling, the patterns of predicted and actual error are similar except for the
case study (3) at the angular resolution of 0.12°. Based on the information on Table 16, the mean
point spacing for the case study (3) at the angular resolution of 0.12° is equal to 19 mm. The
findings from the synthetic data suggest that the error would continue to decline up to point spacing
of 20 mm. Conversely, the actual error increased at this point. Therefore, the appropriate angular
resolution to scan the building is 0.09° which produces a point cloud with a mean point spacing of
15 mm (see Table 16). Nonetheless, for wall modeling, the mean difference between the predicted
and actual errors are less than 2 mm.

107
For opening detection, the patterns of predicted and actual errors for case studies (1) and (3) are
very similar. However, the actual error magnitudes are 4 mm larger in average. This difference is
as high as 5 mm for case study (2). Based on our observations, some of the openings on the higher
levels of case study (2) were not fully captured due to oblique incidence angle and self-occlusions
of the building. These quality issues were not modeled in the synthetic data; therefore, the
predicted errors were relatively optimistic. Overall, the results show that except for one case, the
patterns of the predicted and actual errors agree. Also, the actual errors are higher than predicted
errors, except for the wall modeling of case study (1).
Figures 45 and 46 summarize the comparison of the predicted and actual errors at component-level
for the three case studies. We have measured the mean, minimum, maximum, and SD of the
differences between the predicted and actual errors for all components. For the wall modeling, the
results show that the difference between the predicted and actual errors are as small as 0.1 mm.
Also, for case studies (1) and (2), the maximum difference between the predicted and actual errors
is less than 4 mm. Note that the range of SD fluctuations is less than 1.5 mm which shows that the
prediction of errors has performed consistent for all components. In contrast, for the opening
detection, the results indicate larger differences between the predicted and actual errors. In case
study (2), the challenge in detecting openings with missing data has deviated the error prediction
by 20 mm. Another reason for the difference is that, in some cases, the building openings were not
captured accurately in the mock-up models. Also, the SD of noise in real-life data was estimated
based on the assumption that the noise distribution is Gaussian for all areas of the building. In
order to detect the noise level more accurately, there is a need to assume a GGD model in edge
points. However, estimating GGD parameters requires iterative numerical methods such as
maximum likelihood which is computationally expensive especially for dense point clouds.
Despite the mentioned challenges, as seen in Figure 46, the minimum differences in predicted and
actual errors go as small as 0.1 mm. Note that the SD of the error differences is less than 5 mm.
Overall, for the wall modeling, the mean discrepancies of errors are measured to be in the range
of (1.67 ± 0.96) mm with 95% confidence interval. On the other hand, for opening detection, the
mean discrepancies of errors are measured to be in the range of (3.97 ± 3.74) mm with 95%
confidence interval. If one uses the GSA-defined LOD requirements, based on the prediction
accuracy of the proposed method, the maximum deviation tolerance of the data could be predicted
accurately if the predicted error is not within the ±7.71 mm (3.97 + 3.74 = 7.71) range of the two
successive LOD levels. For instance, the maximum deviation tolerance for LOD level 2 is equal
to 13 mm. If the predicted error of the data is estimated to be 5 mm (< (13 - 7.71 = 5.29)), then
using our method, one can conclude that the data meets the LOD level 2 requirements. Note that
for wall modeling, the mean error for angular resolution of 0.09° and higher is less than 2 mm (see
Figure 45). Therefore, the proposed method could predict the errors even more accurately (±2 mm
range of the two successive LOD levels for this case) for higher angular resolutions. Also, based
on the numbers in Figure 46, given that the largest mean difference is 5 mm, the point with 20 mm
maximum difference is more than two SDs away from the mean. Therefore, occurrences of large
differences between the predicted and actual error is rare. Generally, the findings indicate that the
predicted errors are within the sub-centimeter range of the actual errors. Therefore, the proposed
quantifications for the impacts of data quality issues using the synthetic data could be potentially

108
used to assess the quality of real-life scan data by determining whether the point density and noise
level of the scan data will result in modeling errors that are acceptable based on project-specific
requirements.

Figure 44. Comparison of predicted and actual mean errors of wall modeling and opening detection
at building-level

109

Figure 45. Differences between predicted and actual errors of wall modeling at component-level

110
Figure 46. Differences between predicted and actual errors of opening detection at component-
level

11.8 Limitations and Conclusions
This chapter presented a study on assessing the quality of scan data by pre-analyzing the influences
of point density and noise on the accuracy of 3D models generated from synthetic scan data as
representatives of real-life scan data. We used 3D mock-up models of three case study buildings
as the baseline model for generating synthetic scan data. The mock-up models were sampled with
35 different point spacings, and later contaminated with 10 levels of noise magnitude. A novel
method for adding noise based on the geometric properties of the building was proposed. The
method classifies the building’s scan data into wall and edge areas by extracting a set of local
geometric features. Two different noise models are then assigned to each class to provide a realistic
representation of noise in scan data. The synthetic scan data were then used to regenerate the mock-
up models. Individual wall and opening components of the generated model were compared to
their correspondences in the original mock up model to compute the model errors at a component-
level granularity. The comparison of results showed that under similar point spacing and noise
level conditions, the model errors were different for each case study. Further observations
indicated that the geometric properties of the buildings increased/decreased the impacts of noise
and point spacings on model errors. However, the impact magnitudes were measured to be
different for each case study. The derived model errors were then compared with the errors of the
3D models generated from real-life scan data. The comparisons showed that the difference
between the two errors were in the range of 0.1 to 20 mm with an average value of 3 mm. As
oppose to the current data quality guidelines that do not consider building’s geometry, the proposed
impact analysis could be used to develop a framework for scan data quality control in which the
data quality issues are quantified based on considering the geometric features of buildings. Prior
to the job-site scan data collection, the framework will use the mock-up model of the target
building to quantify the potential impacts of point density and noise on the geometric modeling of
point clouds. The derived impact analysis, could be then used on the job-site to assess the quality
of the collected scan data, and predict the potential as-built modeling errors. Chapter 12 will
implement and evaluate the proposed quality assessment framework. The presented study could
benefit from several improvements through further research. The line-of-sight limitations of the
TLS were not considered in the data quality issue modeling. As the result, the impacts of site-
occlusions or building’s self-occlusions were not reflected in the synthetic data. The challenge
with modeling missing data is that their extent and location are dependent on several parameters
such as the type and position of the site-occlusions, scanning conditions, scanner’s position, etc.
The future work will include studying the impacts of site-occlusions by modeling and projecting
them to the scan data.

111
Chapter 12 IN-SITU ASSESSMENT OF SCAN DATA QUALITY

The results of chapter 11 showed that the as-built modeling errors from real-life scan data could
be predicted using the derived model errors with an average 3 mm prediction accuracy. In this
chapter, we investigate the feasibility of implementing an automated in-situ quality assessment of
scan data by using the mock-up models, and the model errors derived from the data quality impact
analysis introduced in chapter 11. In addition to evaluating the point density and noise level of
data, we propose a novel method for detecting missing data resulted from occlusions. The objective
is to identify the data quality issues that could potentially lead to as-built model errors immediately
after the data is collected. The predicted errors of as-built models, as well as the missing data are
presented as color-coded visualizations at a component-level granularity. Findings from the quality
assessment are then used to employ corrective data collection measures to improve the data quality.
12.1 Identifying Missing Data
The existence of occlusion and clutter, such as vegetation and construction equipment result in
missing regions in the scan data. Recent as-built modeling methods have introduced methods such
as machine learning, tracing, and inpainting to fill in the missing regions [13,147]. Nonetheless,
the performance of these methods is significantly reduced by the increase in the occlusion level
(i.e. percentage of the building objects’ surface that is not visible due to the occlusion). In addition,
it is yet unclear what level of partial occlusion is tolerable by these methods. Therefore, we assume
any missing data due to occlusion as a quality issue. Current point cloud processing methods detect
occlusions by ray-tracing from every scan position through the entire scan data which is a time
consuming process [147]. To reduce the required time for performing ray-tracing, we propose a
novel method in which the ray-tracing is limited to the predicted missing regions of scan data. We
assume that the collected scan data are registered by the TLS’s software and the scan positions are
known (see Figure 47a). Note that these assumptions are made based on the features that are
available in all state-of-the-art scanning software. Our ray-tracing method starts by providing a
user interface to specify the perimeter of the building. The specified perimeter will be used
throughout the computing processes to identify the approximate premises of the building’s point
cloud. The process continues by detecting the edges on the point cloud using the same process
explained in chapter 11. Once the edges are detected, we map them to a 2D binary surface and
draw a bounding box around the regions that include connected edges (see Figure 47b). The
bounding boxes will represent the locations of potential missing data. Next, the bounding boxes
are studied by performing ray-tracing at different scan positions from which they are visible (see
Figure 47c, d, and e). If the rays intersect any objects between the building boundaries and the
scanner, the object is recognized as occlusion and the corresponding region on the scan data is
labeled as missing data.
As seen in Figure 47, our proposed ray-tracing is limited to the square pyramid regions created
between each scanning position and the bounding box on the building’s point cloud. The square
base of the pyramid is created by the detected bounding box, and the apex is located at the scanner’s
sensor position. The proposed ray-tracing was programmed in Unity 3D© game engine. The point
cloud was imported into the Unity’s environment, and the scanner positions were modeled by a

112
camera object. The coordinates of the bounding boxes were then imported and used to specify the
square base of the pyramid. We programmed Unity to create a pyramid mesh between the bounding
box and the scanner. A collider was attached to the pyramid mesh to detect collisions between the
mesh and the point cloud. Note that the collider feature in the Unity allows instant detection of
collisions. If a region is confirmed to be occluded from co-visible scanning positions, it is detected
as missing data and the collision points are identified as the occlusion source.

113

114

115

Figure 47. Identifying missing data due to occlusion: (a) Point cloud and the scan positions, (b)
Bounding boxes around the detected missing data, (c, d, and e) Ray-tracing from different scan
positions.
12.2 Aligning Mock-up Model and Scan Data
To enable a component-level data quality assessment, there is a need to identify the building
components on the point cloud. We use the building mock-up models to assist with the process of
identifying the components. The mock-up model’s mesh object (.FBX format) is imported to the
Unity and an interface is provided for manual alignment of the mock-up model and the point cloud.
The interface allows for transformation and scaling of the mock-up model. As explained in chapter
10, the geometry of the mock-up model has an average inconsistency between 3 to 40 cm.
Nonetheless, the degree of freedom from the scaling feature allows the inconsistency to be reduced.
Once the models were approximately aligned, an iterative closest point (ICP) algorithm is used to
finely register the two models. Next, the components on the mock-up model are registered with
the regions on the point cloud with minimum Euclidean distance.

116

Figure 48. Aligning the mock-up model and the building’s point cloud
12.3 Computing Predicted Errors for Building Components
Once every component on the mock-up model is matched with its correspondence on the point
cloud, the designated areas on the point cloud are studied for mean point spacing and noise level.
The mean values are computed per the process explained in chapter 11 section 7. We created a
relational database from the model errors that were derived based on the impact analysis in chapter
11. Each case study building has a table of components that are called using a specific
identification number. Additionally, every component has a table which relates the model errors
to their corresponding point spacing and noise level. Therefore, the predicted error of each
component in the point cloud is automatically queried from the database with respect to the
computed mean point spacing and noise level. The process of computing predicted errors is
summarized in Figure 49.

117

Figure 49. Process of computing predicted errors for every building component
Once the predicted errors are queried, the values are color-coded and mapped to the mock-up
models (see Figure 50). Additionally, the detected missing data are mapped to the models with
respect to their locations on the scan data. Note that the quality assessment is merely performed
for wall modeling and opening detections. The color-coded windows and doors in Figure 50
represent the predicted errors for the corresponding openings. Finally, we measured the required
timing for implementing the proposed quality assessment method on the three case study buildings.
Table 17 presents the estimated time for conducting every step of the quality assessment method.
The proposed method was implemented using a 64-bit Acer Predator with Intel Core i7 CPU at
4.00 GHz, 32.0 GB RAM, and NVIDIA Geforce GTX1080 graphics card. Note that the analyses
of error prediction accuracies are the same as those presented in chapter 11, section 7. The use of
mock-up models, the pre-computed model errors, as well as the proposed ray-tracing method
resulted the average estimated time for the quality assessment to be as low as 31.33 minutes. The
required time is significantly lower than the point cloud deviation analysis method proposed by
Anil, et al. [20] which was estimated to be 185 minutes. In addition, our method does not require
manual measurements, nor it is dependent on having access to a ground-truth BIM model.
Moreover, it identifies the missing data and the corresponding occlusion source. Considering that
a scan data collection normally takes several hours to complete [59,73], and that data quality
deficiencies would require the survey team to perform rescans [118], the proposed quality
assessment method could potentially assist the survey team to identify the data quality issues
within the time-frame of a scan session.

118

Figure 50. Color-coded visualization of predicted errors: (a, b) case study 1, (c) case study 2.

119

Table 17. Estimated timing for implementing the proposed in-situ quality assessment process
Processing Step

Case
Study (1)

Case
Study (2)

Case
Study (3)

Average Estimated
Time (mins)

Identifying missing data

6

9

11

8.67

Aligning mock-up model and the scan data

3

3

5

3.67

Matching corresponding components

6

6

8

6.67

Computing point spacing and noise

5

5

6

5.33

Finding predicted errors

2

2

3

2.33

Preparing color-maps of predicted errors

4

4

6

4.67

Total Time

26

29

39

31.33

12.4 Improving Data Quality
The component-level granularity of the proposed data quality assessment, as well as the color-
coded reporting enables the rapid identification of building components that lack the required data
quality based on the project requirements. In order to investigate the possibility of improving the
data quality using the information from the proposed quality assessment method, we studied select
components from the three case study buildings. We set the assumption that the scan data of the
buildings requires to meet the highest LOD requirement (i.e. LOD = 4) defined by the U.S. GSA.
Note that this LOD level defines a maximum error tolerance of ±3 mm. For every building, we
studied the scan data that was collected with angular resolution of 0.09° as they provided several
instances of building components with predicted errors of larger than ±3 mm.
The data quality issues were grouped into two main categories of missing data, and low point
density. The former refers to the sections of data that were missing either due to site occlusions,
or self-occlusions (i.e. parts of building being occluded by the building itself); and the latter refers
to parts of data that were sparse due to being distant from the scanner or creating a large oblique
incidence angle with the scanner. We used the proposed in-situ occlusion detection, as well as the
findings from chapter 7 and Eq. (1) in chapter 11 to pinpoint the sources of the quality issues, and
to employ appropriate corrective action for improving the data quality. For the components that
were partially occluded by site occlusions such as trees and parked cars, the sources of occlusion
were detected, and new scan positions were determined accordingly to circumvent the occlusions.
For building components with low point density, based on the error tolerance asserted by LOD 4,
the corresponding model error’s database was queried to retrieve the minimum scan data
characteristics (i.e. point density/spacing and noise) that meet the quality requirements.
According to the scan data’s statistics that were presented in Table 16 of section 11.7, the mean
noise of data was measured to be lower than 0.422 which is relatively lower than the error tolerance
for LOD 4. Therefore, we disregarded the effect of noise, and only focused on improving the point
density of data by taking appropriate corrective actions. Using Eq. (1), as well as the maximum
value of point density that meets the quality requirements, we employed an iterative process of

120
selecting different parameters for a scanner’s setting and location to determine candidate scan
positions that provide the required point density. Since the data was collected with an average
angular resolution of 0.09°, we only investigated higher angular resolutions of 0.06° and 0.03°.
Also, we set the allowable maximum distance of the scanner based on the initial position from
which the data was originally captured. Lastly, the range of incidence angle was limited to 0° to
70° based on the discussion in section 11.2. Table 18 summarizes the general categories of the
quality issues, and different corrective data collections that were used to improve the data quality.
Table 19 presents quantitative information on the initial quality conditions of select components
on the three case study buildings, and their quality conditions after the corrective data collection
was performed. Note that the abbreviated codes in Table 18 are used for conciseness of Table 19.
The findings indicate that the error levels of the studied components were reduced as a result of
the corrective data collection guided by the quality reports of the proposed data quality assessment
method. Also, the target LOD 4 data quality requirement has been successfully met by all the
studied components. Table 20 presents the overall mean quality improvement of data for the three
case study buildings. The results show that the data quality has been improved with an average of
3.085 mm upgrading the as-built models from prior LOD levels of 2 and 3, to the LOD level 4.
Figure 51 illustrates the general conditions of scan data for the case study buildings prior and after
the quality improvements. The top openings in Figure 51a are the sample openings listed in Table
19 where the low point density has resulted in model errors. These errors are reduced by decreasing
the incidence angle and increasing the angular resolution of the scanner using a new scan position.
For instance, for this case an angular resolution of 0.06° was sufficient to meet the LOD 4 quality
requirements. Note that the component-level quantitative values provided by the proposed quality
assessment guided the corrective data collection by preventing the selection of a higher angular
resolution such as 0.03° which only results in redundant quality by doubling the scanning time.
Also, as discussed in chapter 11, a higher point density does not always result in less model errors.
Figures 51b and c, show sample scan data of walls and openings that were impacted by low point
density and occlusion. Corrective actions such as moving around the scanner positions and moving
them closer to the target components were employed to improve the data quality.
Table 18. General categories of the detected quality issues, corrective data collection actions, and
their abbreviated codes.
Subject

Code

Quality Issue Description
Missing Data MD

Low Point Density LD

Corrective Actions for Quality Improvement

Avoid occlusion using a new scan position AO

Reducing incidence angle using a new scan position RA

Reducing scanner’s distance using a new scan position RD
Increasing angular resolution in a new scan position IN

Increasing angular resolution at the original scan position IR

121

Table 19. Improvement of scan data for different components of the case study buildings
(measurements are in millimeter)

Component
ID*

Initial Condition

Improved Condition

Point
Density

Noise SD

Predicted
Error

Actual
Error

LOD

Quality Issue
Description

Corrective
Action to
Improve
Quality

Point
Density

Noise SD

Predicted
Error

Actual
Error

LOD

Case Study (1)

CS1_WL_1

9.909

0.668

9.115

9.832

3

MD, LD

AO, RD

3.441

0.643

3.103

2.866

4

CS1_WL_2

10.602

0.542

8.311

8.751

3

MD, LD

AO, RD

2.851

0.518

2.589

2.203

4

CS1_WL_3

9.300

0.545

8.093

8.291

3

LD

RD

2.688

0.587

1.043

1.544

4

CS1_OP_1

11.103

0.640

8.911

10.082

3

MD, LD

AO, RD

3.570

0.635

2.253

2.072

4

CS1_OP_2

10.670

0.582

8.537

8.901

3

LD

RA, IN

3.746

0.564

1.066

1.483

4

CS1_OP_3

13.970

0.533

10.665

9.610

3

LD

RA, IN

3.732

0.604

2.713

2.339

4

CS1_OP_4

12.264

0.728

9.053

8.425

3

LD

RA, IN

4.372

0.498

2.156

1.876

4

CS1_OP_5

12.864

0.444

9.808

10.178

3

LD

RA, IN

3.576

0.573

3.040

2.443

4

CS1_OP_6

14.482

0.670

10.783

10.399

3

LD

RA, IN

3.153

0.652

2.808

2.677

4

CS1_OP_7

13.414

0.553

10.391

11.663

3

LD

RA, IN

3.536

0.585

2.626

2.813

4

Case Study (2)

CS2_WL_1

12.222

0.454

6.509

7.731

3

MD, LD

AO, RD

3.586

0.462

2.137

2.549

4

CS2_WL_2

13.726

0.636

7.559

8.767

3

MD, LD

AO, RD

3.727

0.710

1.774

2.883

4

CS2_WL_3

13.314

0.466

6.879

6.775

3

LD

IR

2.757

0.526

2.235

1.891

4

CS2_WL_4

14.570

0.773

8.333

10.827

3

MD, LD

AO, RD

2.827

0.661

1.493

2.944

4

CS2_OP_1

13.284

0.598

7.986

10.733

3

MD, LD

AO, RD

3.694

0.459

1.400

2.685

4

CS2_OP_2

13.968

0.576

8.103

8.022

3

LD

AO, RD

2.697

0.487

1.694

1.838

4

CS2_OP_3

16.657

0.641

9.386

13.315

2

MD, LD

AO, RD

4.104

0.556

2.855

2.969

4

CS2_OP_4

13.260

0.528

7.951

9.246

3

LD

RA, IN

3.119

0.517

1.986

9

4

CS2_OP_5

15.566

0.619

8.025

10.956

3

MD, LD

RA, IN

3.757

0.601

1.562

2.729

4

CS2_OP_6

17.118

0.586

9.967

13.955

2

MD, LD

RA, IN

4.154

0.524

2.928

2.979

4

Case Study (3)

CS3_WL_1

10.823

0.416

8.593

10.323

3

MD, LD

AO, RD

3.329

0.411

2.145

2.494

4

CS3_WL_2

10.199

0.474

7.786

8.954

3

LD

RD

3.004

0.457

1.105

1.862

4

CS3_WL_3

11.131

0.549

9.333

10.994

3

MD, LD

AO, RD

3.450

0.531

2.272

2.579

4

CS3_WL_4

10.781

0.504

8.443

9.926

3

LD

RD

3.221

0.477

1.856

2.346

4

CS3_WL_5

12.189

0.497

10.353

11.079

3

MD, LD

AO, RD

3.535

0.443

2.472

2.713

4

CS3_WL_6

12.818

0.575

11.256

13.677

2

MD, LD

AO, RD

3.717

0.528

3.070

2.937

4

CS3_OP_1

11.160

0.530

7.027

7.895

3

LD

RA, IN

2.943

0.511

1.334

2.190

4

CS3_OP_2

10.543

0.440

6.547

6.452

3

LD

IR

2.849

0.420

1.138

1.931

4

CS3_OP_3

14.695

0.556

10.553

9.475

3

MD, LD

RA, IN

3.990

0.554

2.052

2.821

4

CS3_OP_4

13.574

0.464

9.822

8.599

3

MD, LD

RA, IN

3.929

0.441

1.506

2.670

4

*CS: Case Study, WL: Wall, OP: Opening

Table 20. Overall improvement of data for the case study buildings (measurements are in
millimeter)
Building
Name

Initial Condition

Improved Condition

Reduced
Mean
Error

Point
Density

Noise
SD

Mean
Model
Error

Overall
LOD

Point
Density

Noise
SD

Mean
Model
Error

Overall
LOD

Case Study (1)

12.799

0.362

4.651

3

3.441

0.643

1.737

4

2.914

122
Case Study (2)

14.869

0.422

5.102

3

2.851

0.518

2.391

4

2.711

Case Study (3)

13.772

0.407

6.444

2

2.688

0.587

2.815

4

3.629

123

124

Figure 51. Snapshots of the initial (top) and improved (bottom) scan data for the three case study
buildings: (a) case study 1, (b) case study 2, and (c) case study 3.

12.5 Limitations and Conclusions
An in-situ scan data quality assessment method was introduced in this chapter. The mock-up
models of the buildings were used to guide the quality assessment method for determining the
locations of building components. The missing data on the point clouds, as well as their
corresponding sources of occlusion were identified through a novel ray-tracing method. A database
of model errors was designed and used to evaluate the quality of the scan data at a component
level. The average required time for implementing the proposed quality assessment method was
estimated to be 31.33 minutes. The quantitative analysis of the data quality enabled us to determine
appropriate corrective data collection to improve the quality of data. The results showed that the
data quality of the three case study buildings was improved by an average of 3.085 mm. The
advantages of our method are that first, the scan data quality is evaluated at the component-level
granularity. Second, our proposed method identifies the missing data and their sources within the
time-frame of a scan session. Moreover, the quantitative results of the quality assessment are used
to identify the quality issues of the collected data during the scan session, and take necessary
corrective actions to improve the data quality. The proposed method could benefit from several
improvements. First, there is still room for improving the time-efficiency of different processing
steps using parallel programming. Second, the quality evaluations are based on the mean values of
point spacing and noise. Therefore, another potential improvement is to evaluate the data quality

125
by using local point spacing and noise. Nonetheless, such improvement will potentially increase
the computing time at different processing stages. Lastly, the proposed method only focuses on
the 3D modeling errors. Another possible advancement is to incorporate 3D object recognition
methods to enable data quality assessment for BIM modeling purposes.

126
Chapter 13 CONCLUSIONS, LIMITATIONS, AND FUTURE WORK

This chapter reviews the major conclusions of the research presented in this dissertation.
Additionally, the limitations are summarized, potential improvements are elaborated, and future
research opportunities are discussed.
13.1 Conclusion
Generating accurate as-built model of buildings has been under the research spotlight for the recent
years. The precision of TLS systems has made them popular tools for documenting the as-built
conditions of buildings. However, factors such as environment conditions, building’s geometry,
site occlusions, and scanner operator’s errors result in data quality issues such as missing data, low
point density, and noise. Existence of data quality issues impedes the automated reconstruction of
as-built models, and often leads to inaccurate as-built models resulting in erroneous decision-
makings. Due to the large size (millions of points) and complexity of the scan data, it is not feasible
to identify and quantify the data quality issues with naked eyes during the scan data collection
[14]. Therefore, there is a need for an automated method that evaluates the quality of data,
identifies the quality issues during the data collection, and provides quantitative analysis so
corrective actions could take place for improving the data quality before the data is used for
generating as-built model. This dissertation proposed a novel method to (I) identify the data quality
requirements for generating accurate architectural as-built models from the scan data, and (II)
evaluate the quality of data during the data collection session with respect to the requirements for
extracting and modeling architectural elements of buildings’ exterior. The data quality
requirements were derived through analyzing the impacts of quality issues on scan data with
respect to the specific geometric properties of different buildings.
As the starting point of the proposed study, a set of scan quality factors were identified through
literature review and phone interviews with scanning specialists. As the result, the four quality
factors of measurement accuracy, registration accuracy consistency, resolution (i.e. point density),
and scan coverage were determined to be frequently measured as the criteria for representing a
high-quality scan data (chapter 7). Our findings from the literature indicated that data quality
factors such as point density, measurement accuracy, and scan coverage are challenging to be
identified and evaluated during the scan session. This was due to the facts that: (I) The existing
body of knowledge is limited in identifying the impacts of these quality issues on the accuracy of
building’s as-built model, and (II) Evaluation of these quality factors requires obtaining prior
information about the geometric properties of buildings and their influence on the scan data
quality. In order to address these limitations, we focused on investigating the relationships between
the quality factors, buildings geometric features, and scan data collection methods.
In chapter 8, we presented one of our preliminary studies which was focused on investigating the
possibility of integrating the scan quality factors and prior information about the geometric
properties of buildings into the data collection practices for improving the quality of scan data. A
framework was proposed through which the data collected by a UAV was used to guide the TLS
scan planning with a prior information about the geometry of the building. Using the proposed

127
framework, we were able to collect scan data that successfully satisfied the pre-defined LOD
requirements. The results of this study indicated that having access to a priori knowledge about
the geometry of the scanning target could assist in identifying proper scanning settings for meeting
the project’s data quality requirements.
A known challenge with identifying the impacts of different quality issues on the scan data is that
the replication of the quality issues through controlled experiments is time-consuming and often
impossible. However, modeling the quality issues and evaluating their impacts using simulated
scanning scenarios and synthetic data could address this challenge. To better understand whether
the synthetic data provides a proper representation of real-life scan conditions, we used simulation-
based techniques with synthetic data for a study in which the displacements of highway retaining
walls were monitored using real-life laser scan data (chapter 9). The results of this study indicated
that synthetically generated point clouds were closely similar to real-life scans.
In contrast to the previous works that used the BIM models of buildings to study the quality of
scan data, in this research we assumed that no prior BIM model is available. Therefore, we
proposed an automated method for reconstructing building façade elements using rectified ground
images (chapter 10). The objective was to generate mock-up 3D models for buildings and use them
as priori information to better understand the geometric features of the buildings. The method
started by collecting ground images from a building’s façade. The gradient profile of every image
was studied to predict the locations of the façade elements. The image features were used to
classify the elements as windows or doors, and to detect whether their shape was rectangular or
arched. For every wall on the building, a split grammar was defined based on the corresponding
detected architectural elements. Finally, the split grammars reconstructed the building’s façade
structure using procedural modeling. The results indicated that the mock-up models could
represent the geometry of the buildings with average dimension error less than 40 cm.
Additionally, the windows and doors on the buildings’ facades were detected with an average
accuracy of 91.48%. The proposed mock-up model generation method enabled access to prior
geometric information of buildings using basic ground-based images, and through a fast processing
time.
Using the mock-up models and the knowledge that the synthetic data provide a proper
representation of real-life scan data, we replicated the scan data characteristics of point spacing
and noise at different levels, and studied their impacts on the generated as-built models (chapter
11). These data characteristics were selected as they impact the quality factors such as
measurement accuracy, data coverage, and resolution. Moreover, it is feasible to compute them
during the scan data collection. We generated 350 synthetic data with different point spacing and
noise levels for three case study buildings using their mock-up models. Each synthetic data went
through a 3D modeling process and the generated 3D models were compared with the original
mock-up models to determine the impacts of point spacing and noise on the model errors. Although
the overall pattern of the model errors was nearly similar for all the three case study buildings, we
learned that the specific geometry of each building also influences the as-built model’s accuracy.
Therefore, using a similar set of data quality requirements for all building is erroneous. After the
impact analysis, we used the derived model errors to predict the errors of the as-built models

128
generated from the real-life scan data of the case study buildings. The results indicated that the
derived errors predicted the as-built models’ errors with an average prediction accuracy of 3 mm.
Lastly, we created a relational database of the model errors for the three case study buildings. The
database and the mock-up models were used to evaluate the quality of real-life scan data. The
mock-up models were aligned with the buildings’ point cloud to enable an automated component-
level data quality assessment. We also proposed a method for identifying the missing data in the
point clouds by implementing a rapid ray-tracing procedure. The average estimated time for
assessing the quality of the scan data for the three case study buildings was measured as 31.33
minutes. The data quality reports provided an x-ray vision by augmenting the scan data with the
predicted as-built model errors based on the scan data characteristics of the entire building
components. We used the quality reports to pinpoint the quality issues of building components
after the scan data collection is completed. The quantitative assessment of data quality was used
to identify proper corrective actions for improving the scan data quality. The results showed that
the quality of data was improved by an average of 3.085 mm, upgrading the generated as-built
models to the highest LOD level defined by the U.S. GSA. As oppose to the current data quality
guidelines that do not consider building’s geometry, we proposed a data quality assessment method
in which the data quality requirements were quantitatively defined by considering the specific
geometric features of the buildings. Even though similar error patterns were seen in the as-built
models of buildings where the geometric properties were similar, our quantitative impact analysis
of the data quality issues indicated that the error magnitudes were different for every building.
Therefore, we argue that generating the mock-up models of buildings and performing a prior data
quality issue impact analysis, as presented in chapter 11, are essential to derive a set of accurate
data quality requirements. Note that access to such information is specifically critical for the
exterior of buildings where a wide range of diversity in the architectural elements is expected. For
instance, such diversity is less expected in the case of as-built pipe modeling as all the pipes are
known to have cylindrical shapes.
13.2 Limitations and Future Work
The façade element recognition approach, presented in this dissertation, focused on detecting and
classifying windows and doors as the main façade elements in buildings. Future work will include
adding more semantics to the mock-up models by sifting through the façade to identify other
common façade elements, such as balconies and railings. Even though the proposed method uses
the co-existence of strong gradients across vertical and horizontal directions at each façade region
to predict the elements’ locations, its accuracy is still dependent on the regularity of a façade’s
structure. The future work includes (1) investigating the possibility of using learning algorithms
that are robust against façade irregularities and are not dependent on a specific architectural style;
and (2) adding a pattern recognition step that could detect misclassified elements and identify
irregularities in the elements arrangement to improve the façade reconstruction. The improvement
of the accuracy could ensure achieving GSA LOD level 1 and potentially higher levels for mock-
up models. Moreover, the detection is limited to those elements on the façade that are visible in
the pictures. If a part of a building’s façade is not visible from ground images (due to occlusions,

129
line-of-sight limitations or poor camera angle), the method will not be able to detect the elements
and thus the generated mock-up model will be inaccurate.
The current procedural modeling methods are not well established, specifically for BIM-related
research. Therefore, the mock-ups need to be converted to BIM models manually. A future
research track could investigate the integration of shape grammars and BIM fundamentals to make
the model, generated with procedural modeling, interoperable with BIM authoring tools.
With regards to the mock-up models, the height of each building was approximated using the
number of floors and the average floor-to-floor height. Future work will include estimating a
building’s height based on perspective projections or using objects in the image with known
dimensions.
A major limitation with the proposed noise modeling algorithm in chapter 11 is that the impact of
object’s material and surface roughness on the noise level are disregarded. An improved noise
modeling algorithm should incorporate the possible range of scanner’s signal strength reflected
from different dominant building materials (e.g. concrete, bricks, etc.). Additionally, a future
research direction will be studying the impacts of site-occlusions by modeling and projecting them
to the scan data. Moreover, real-life scan data usually contains fluctuations in point density.
Although we studied scan datasets with a wide range of point spacing, the fluctuations of point
density within an individual scan data were overlooked. In order to address this limitation, we are
investigating the possibility of integrating the proposed noise modeling method into the sensor
simulation software that are capable of generating scan data with non-uniform point density.
Lastly, this dissertation included analyzing the data from three case study buildings. In future
research, we plan to examine additional buildings with varying architectural typologies to
understand the impacts of data quality issues more accurately. Once a statistically adequate number
of buildings are studied, the findings could be applied to new buildings if instances with
sufficiently similar geometric features are found. Additionally, a future research could study the
possibilities of adding 3D object recognition techniques to the proposed quality assessment method
so the errors in detecting and modeling window and door objects could also be evaluated. Finally,
the scope of this research could potentially be extended to indoor scanning, as well as scanning
other types of infrastructures. More detailed identification of limitations and possible future
research directions could also be found at the end of each chapter.

130
Relevant Publications to Date

Peer-Reviewed Journal Papers

[1] Oskouie, P., Becerik-Gerber, B., and Soibelman, L. (2015), “Automated Measurement
of Highway Retaining Wall Displacements Using Terrestrial Laser Scanners” Journal of
Automation in Construction. Volume 65, pp. 86-101
[2] Oskouie, P., Becerik-Gerber, B., and Soibelman, L. (2017), “Automated Recognition
of Façade Elements for Creating As-is Mock-up Building Models” Journal of Computing
in Civil Engineering. Volume 31, Issue 6

Peer-Reviewed Journal Papers in Preparation

[3] Oskouie, P., Becerik-Gerber, B., and Soibelman, L. (201x), “Toward Assessing the
Quality of Scan Data: Analyzing the Impacts of Noise and Point Cloud Density on
Modeling the Architectural Exterior of Buildings from Point Clouds” Journal of
Computing in Civil Engineering.

Peer-Reviewed Conference Papers

1. Oskouie, P., Becerik-Gerber, B., and Soibelman, L. (2015), “A Data Quality-driven
Framework for Asset Condition Assessment Using LiDAR and Image Data” Proceedings
of ASCE Computing in Civil Engineering, pp. 240-248
2. Oskouie, P., Becerik-Gerber, B., and Soibelman, L. (2014), “Automated Cleaning of Point
Clouds for Highway Retaining Wall Condition Assessment” Proceedings of ASCE
Computing in Civil and Building Engineering, pp. 966-974

131
Appendices

Appendix 1. Interview questions

How do you define the scan data quality? In other words, how is the quality defined for scan
data?
- What parameters/features of scan data do you/your team check in-situ to control the
quality of data?
- What parameters/features of scan data are checked later during the data processing
stage to evaluate the quality of data?
- What metrics do you/your team use for evaluating the quality of a scan data (evaluating
different parameters that have influences on scan data quality)?

How do you/your team make sure the scan data quality is satisfactory? (Quality Assurance
and Control)
- What manual process/automated process do you currently use before performing the
actual scan to check the quality of the scan data? (Quality Assurance Stage)
o Instruments calibration?
o Weather conditions?
o Traffic conditions?
o Scan planning?
- What manual process/automated process do you currently use to check the quality of
the scan data? (Quality Control Stage)
o Is there any particular software, workflow, or process to use?

How many times have you personally experienced issues with scan data quality?
- Could you please mention some examples?
o What was the type of the project?
o What were the project quality requirements?
o What was/were the data quality issue(s)?
o How did the issues affect the process of scanning in-situ?
o How did the issues affect the processing of the data?
o How did you solve the issues(s)?

Is there currently any guideline/standard that defines scan quality or enlists scan data quality
requirements for different type of projects?
- Do you/your team define specific level of detail requirements for different geometric
features in projects?

132

How do you define scan quality with respect to the data processing requirements for Scan to
BIM process?
- Could you please refer to some of the challenges with the Scan to BIM process that are
caused by the data quality issues?

What do you think about the idea of defining scan plan based on specific LOD requirements
for the geometric features of a project?

What do you think about the idea of in-situ quality control of the scan data based on the
specific LOD requirements for the geometric features of a project?
- What is missing in the current data quality control process?
- What would you wish you had in-situ to make sure the data quality is satisfactory?

What do you think about the influence of the following parameters on the scan data quality?
(Could you rate them from 1 to 10, 1 being not important, and 10 being extremely
important?)
- Resolution (Point density, point spacing)? Do you define resolution differently? If so,
what are the parameters that define data resolution? What metric(s) do you think is/are
appropriate for evaluating these parameters?

- Coverage (Completeness, no shadow zones, no non-visible zones)? Do you define
coverage differently? If so, what are the parameters that define data coverage? What
metric(s) do you think is/are appropriate for evaluating these parameters?

- Consistency (Uniform resolution across the project, registration accuracy)? Do you
define consistency differently? If so, what are the parameters that define data
consistency? What metric(s) do you think is/are appropriate for evaluating these
parameters?

- Clarity (visual crispness with maximum data retention)? Do you define clarity
differently? If so, what are the parameters that define data clarity? What metric(s) do
you think is/are appropriate for evaluating these parameters?

- Manageability (data size, transfer, and process)? Do you define data manageability
differently? If so, what are the parameters that define data manageability? What
metric(s) do you think is/are appropriate for evaluating these parameters?

133
- Color Quality (alignment of images with the point cloud)? Do you define data color
quality differently? If so, what are the parameters that define data color quality? What
metric(s) do you think is/are appropriate for evaluating these parameters?

Can you think of any other parameter that was not mentioned?

134
References

[1] 3D Laser Scanning Quality Management Program Guide, U.S. GSA, 2012.
[2] The Business Value of BIM ,SmartMarket Report- McGraw Hill Construction, 2009.
[3] The Business Value of BIM for Owners, ,SmartMarket Report- McGraw Hill Construction, 2012.
[4] The Business Value of BIM for Owners, ,SmartMarket Report- McGraw Hill Construction, 2014.
[5] Geotechnical Design Manual, Deep Foundations, South Carolina Department of Transportation
(SCDOT), 2010.
[6] Geotechnical Design Manual, Abutments, Retaining Walls, and Reinforced Slopes, New York
State Department of Transporation, 2012.
[7] LRFD Bridge Design Specifications, American Association of State and Highway Transportation
Officials (AASHTO), Washington, D.C., 2010.
[8] Mechanically Stabilized Earth System Inspection Manual, Geotechnical Engineering Manual, New
York State Department of Transportation, 2007.
[9] Retaining Walls, Bridge Design Specifications, Caltrans, 2004.
[10] RIEGL VZ-400 Technical Specifications.
[11] United States Building Energy Efficiency Retrofits, Market Sizing and Financial Models,
Rockefeller Foundation 2012.
[12] What Determines the Quality of a 3D Laser Scan, 2012.
[13] A. Adan, D. Huber, 3D reconstruction of interior wall surfaces under occlusion and clutter, 3D
Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), 2011 International
Conference on, IEEE, 2011, pp. 275-281.
[14] J. Ahn, K. Wohn, Interactive scan planning for heritage recording, Multimedia Tools and
Applications 75 (7) (2016) 3655-3675.
[15] M. Alba, M. Scaioni, Automatic Detection of Changes and Deformations in Rock Faces by
Terrestrial Laser Scanning, Remote Sensing and Spatial Information Sciences, Vol. XXXVIII,
International Archives of Photogrammetry, UK, 2010.
[16] A. Aldoma, Z.-C. Marton, F. Tombari, W. Wohlkinger, C. Potthast, B. Zeisl, R.B. Rusu, S. Gedikli,
M. Vincze, Point cloud library, IEEE Robotics & Automation Magazine 1070 (9932/12) (2012).
[17] D.E. Alzamora, D. Enrique, S.A. Anderson, T.R. Board, Review of mechanically stabilized earth
wall performance issues, Transportation Research Board (TRB), 2009 Annual Meeting CD-ROM,
2009.
[18] P. Anderson, R. Gladstone, J. Sankey, State of the Practice of MSE Wall Design for Highway
Structures, Geotechnical Engineering State of the Art and Practice, 2012, pp. 443-463.
[19] H. Andreasson, A.J. Lilienthal, 6D scan registration using depth-interpolated local image features,
Robotics and Autonomous Systems 58 (2) (2010) 157-165.
[20] E.B. Anil, P. Tang, B. Akinci, D. Huber, Deviation analysis method for the assessment of the
quality of the as-is Building Information Models generated from point cloud data, Automation in
Construction 35 (2013) 507-516.
[21] S. Azhar, Building information modeling (BIM): Trends, benefits, risks, and challenges for the
AEC industry, Leadership and Management in Engineering (2011).
[22] V. Balali, M. Golparvar-Fard, Evaluation of Multiclass Traffic Sign Detection and Classification
Methods for U.S. Roadway Asset Inventory Management, Journal of Computing in Civil
Engineering (2015) 04015022.
[23] V. Balali, M. Golparvar-Fard, Segmentation and recognition of roadway assets from car-mounted
camera video streams using a scalable non-parametric image parsing method, Automation in
Construction 49 (2015) 27-39.
[24] B. Becerik-Gerber, F. Jazizadeh, G. Kavulya, G. Calis, Assessment of target types and layouts in
3D laser scanning for registration accuracy, Automation in Construction 20 (5) (2011) 649-658.

135
[25] T. Bennett, Exploring Reality Computing for Civil Infrastructure, Informed Infrastructure, 2015.
[26] R.R. Berg, B.R. Christopher, N.C. Samtani, Design of Mechanically Stabilized Earth Walls and
Reinforced Soil Slopes–Volume II, 2009.
[27] P.S. Blaer, P.K. Allen, View planning for automated site modeling, Robotics and Automation,
2006. ICRA 2006. Proceedings 2006 IEEE International Conference on, IEEE, 2006, pp. 2621-
2626.
[28] W. Boehler, M.B. Vicent, A. Marbs, Investigating laser scanner accuracy, The International
Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 34 (Part 5) (2003)
696-701.
[29] F. Bosché, M. Ahmed, Y. Turkan, C.T. Haas, R. Haas, The value of integrating Scan-to-BIM and
Scan-vs-BIM techniques for construction monitoring using laser scanning and BIM: The case of
cylindrical MEP components, Automation in Construction 49, Part B (0) (2015) 201-213.
[30] F. Bosché, E. Guenet, Automating surface flatness control using terrestrial laser scanning and
building information models, Automation in Construction 44 (0) (2014) 212-226.
[31] F. Bosche, C.T. Haas, B. Akinci, Automated recognition of 3D CAD objects in site laser scans for
project 3D status visualization and performance control, Journal of computing in civil engineering
23 (6) (2009) 311-318.
[32] F.N. Bosche, S. O'Keeffe, The Need for Convergence of BIM and 3D Imaging in the Open World,
(2015).
[33] D.J. Bouvier, C. Gordon, M. McDonald, An approach for occlusion detection in construction site
point cloud data, Proceedings of the Congress on Computing in Civil Engineering, 2011, pp. 234-
241.
[34] I.K. Brilakis, L. Soibelman, Shape-Based Retrieval of Construction Site Photographs, Journal of
Computing in Civil Engineering 22 (1) (2008) 14-20.
[35] D. Bryde, M. Broquetas, J.M. Volm, The project benefits of building information modelling (BIM),
International Journal of Project Management 31 (7) (2013) 971-980.
[36] M. Chen, L. Soibelman, B. Becerik-Gerber, A Proactive Scan Planning Framework for Courtyard-
Centric Buildings, (2016).
[37] G. Cheok, Franaszek, M., Phase III: Evaluation of an acceptance sampling method for 2D/3D
building plans, National Institute of Standards and Technology (NIST), Building and Fire Research
Laboratory, Gaithersburg, MD, 2012.
[38] G. Cheok, K. Saidi, M. Franaszek, J. Filliben, N. Scott, Characterization of the range performance
of a 3D imaging system, NIST TN 1695 (2011).
[39] A. Cohen, A.G. Schwing, M. Pollefeys, Efficient structured parsing of facades using dynamic
programming, 2014 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2014,
pp. 3206-3213.
[40] G. Csurka, C. Dance, L. Fan, J. Willamowski, C. Bray, Visual categorization with bags of
keypoints, Workshop on statistical learning in computer vision, ECCV, Vol. 1, Prague, 2004, pp.
1-2.
[41] N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, 2005 IEEE Computer
Society Conference on Computer Vision and Pattern Recognition (CVPR'05), Vol. 1, IEEE, 2005,
pp. 886-893.
[42] M. Deru, K. Field, D. Studer, K. Benne, B. Griffith, P. Torcellini, B. Liu, M. Halverson, D.
Winiarski, M. Rosenberg, US Department of Energy commercial reference building models of the
national building stock, (2011).
[43] J.R. Diebel, S. Thrun, M. Brünig, A Bayesian method for probable surface reconstruction and
decimation, ACM Transactions on Graphics (TOG) 25 (1) (2006) 39-59.
[44] A. Dimitrov, M. Golparvar-Fard, Segmentation of building point cloud models including detailed
architectural/structural features and MEP systems, Automation in Construction 51 (0) (2015) 32-
45.
[45] J. Dunnicliff, Geotechnical Instrumentation for Monitoring Field Performance, Wiley, 1993.

136
[46] C. Eastman, C.M. Eastman, P. Teicholz, R. Sacks, BIM handbook: A guide to building information
modeling for owners, managers, designers, engineers and contractors, John Wiley & Sons, 2011.
[47] H. El Houari, A.F. El Ouafdi, Generalized Gaussian kernel for triangular meshes denoising,
Engineering & MIS (ICEMIS), International Conference on, IEEE, 2016, pp. 1-7.
[48] M. Eybpoosh, B. Akinci, M. Bergés, Effects of Planning and Data Collection Approaches on the
Quality of Processed Laser Scanned Data: Lessons Learned, Construction Research Congress
2012@ sConstruction Challenges in a Flat World, ASCE, 2012, pp. 950-959.
[49] M.A. Fischler, R.C. Bolles, Random sample consensus: a paradigm for model fitting with
applications to image analysis and automated cartography, Commun. ACM 24 (6) (1981) 381-395.
[50] M. Franaszek, G.S. Cheok, C. Witzgall, Fast automatic registration of range images from 3D
imaging systems using sphere targets, Automation in Construction 18 (3) (2009) 265-274.
[51] D. Freedman, P. Diaconis, On the histogram as a density estimator: L 2 theory, Probability theory
and related fields 57 (4) (1981) 453-476.
[52] T. Gao, B. Akinci, S. Ergan, J. Garrett, Constructing as-is BIMs from progressive scan data, Proc.,
Gerontechnology 11 (2012) 75.
[53] T. Gao, S. Ergan, B. Akinci, J. Garrett, Evaluation of Different Features for Matching Point Clouds
to Building Information Models, Journal of Computing in Civil Engineering 0 (0) (2014) 04014107.
[54] L. Gimenez, J.-L. Hippolyte, S. Robert, F. Suard, K. Zreik, Review: reconstruction of 3D building
information models from 2D scanned plans, Journal of Building Engineering 2 (2015) 24-35.
[55] M. Golparvar-Fard, J. Bohn, J. Teizer, S. Savarese, F. Peña-Mora, Evaluation of image-based
modeling and laser scanning accuracy for emerging automated performance monitoring techniques,
Automation in Construction 20 (8) (2011) 1143-1155.
[56] M. Golparvar-Fard, F. Peña-Mora, S. Savarese, Application of D4AR–A 4-Dimensional
augmented reality model for automating construction progress monitoring data collection,
processing and communication, ITcon 14 (2009) 129-153.
[57] M. Golparvar-Fard, F. Peña-Mora, S. Savarese, Monitoring of construction performance using
daily progress photograph logs and 4d as-planned models, Computing in Civil Engineering (2009),
2009, pp. 53-63.
[58] J. Gong, C.H. Caldas, Data processing for real-time construction site spatial modeling, Automation
in Construction 17 (5) (2008) 526-535.
[59] GSA, BIM Guide for 3D Imaging, U.S. General Services Administration, 2009.
[60] U. GSA, Gsa Bim Guide for 3D Imaging, Washington, DC: US General Services Administration.
http://www. gsa. gov/graphics/pbs/GSA_BIM_Guide_Series_03. pdf (2009).
[61] A. Habib, K.I. Bang, A.P. Kersting, D.-C. Lee, Error budget of LiDAR systems and quality control
of the derived data, Photogrammetric Engineering & Remote Sensing 75 (9) (2009) 1093-1108.
[62] Y. Ham, M. Golparvar-Fard, EPAR: Energy Performance Augmented Reality models for
identification of building energy performance deviations between actual measurements and
simulation results, Energy and Buildings 63 (2013) 15-28.
[63] Y. Ham, M. Golparvar-Fard, Rapid 3D energy performance modeling of existing buildings using
thermal and digital imagery, Construction Research Congress 2012@ sConstruction Challenges in
a Flat World., ASCE, 2012, pp. 991-1000.
[64] K. Han, M. Golparvar-Fard, BIM-Assisted Structure-from-Motion for Analyzing and Visualizing
Construction Progress Deviations through Daily Site Images and BIM, International Workshop on
Computing in Civil Engineering (doi: 10.1061/9780784479247.074) (2015) 596-603.
[65] J. Hern, ndez, B. Marcotegui, Morphological segmentation of building facade images, 2009 16th
IEEE International Conference on Image Processing (ICIP), 2009, pp. 4029-4032.
[66] M. Hossain, G. Kibria, M. Khan, J. Hossain, T. Taufiq, Effects of Backfill Soil on Excessive
Movement of MSE Wall, Journal of Performance of Constructed Facilities 26 (6) (2012) 793-802.
[67] D.F. Huber, M. Hebert, Fully automatic registration of multiple 3D data sets, Image and Vision
Computing 21 (7) (2003) 637-650.

137
[68] M.R. Jahanshahi, J.S. Kelly, S.F. Masri, G.S. Sukhatme, A survey and evaluation of promising
approaches for automatic image-based defect detection of bridge structures, Structure and
Infrastructure Engineering 5 (6) (2009) 455-486.
[69] V. Jampani, R. Gadde, P.V. Gehler, Efficient Facade Segmentation Using Auto-context, 2015
IEEE Winter Conference on Applications of Computer Vision, 2015, pp. 1038-1045.
[70] G. Kavulya, F. Jazizadeh, B. Becerik-Gerber, Effects of color, distance, and incident angle on
quality of 3D point clouds, Computing in Civil Engineering (2011) 21-21.
[71] T. Kersten, M. Lindstaedt, Image-Based Low-Cost Systems for Automatic 3D Recording and
Modelling of Archaeological Finds and Objects, Progress in Cultural Heritage Preservation, Vol.
7616, Springer Berlin Heidelberg, 2012, pp. 1-10.
[72] M. Kim, H. Sohn, C. Chang, Localization and Quantification of Concrete Spalling Defects Using
Terrestrial Laser Scanning, Journal of Computing in Civil Engineering 0 (0) (2014) 04014086.
[73] T. Knaak, Establishing Requirements, Extracting Metrics and Evaluating Quality of LiDAR Data,
Certainty 3D, LLC, Orlando, FL, 2014.
[74] T. Knaak, Establishing Requirements, Extracting Metrics and Evaluating Quality of LiDAR Data,
Certainty 3D LLC., Orlando, Florida 2014.
[75] M. Koziński, R. Gadde, S. Zagoruyko, G. Obozinski, R. Marlet, A MRF shape prior for facade
parsing with occlusions, 2015 IEEE conference on computer vision and pattern recognition
(CVPR), IEEE, 2015, pp. 2820-2828.
[76] S. Lazebnik, C. Schmid, J. Ponce, Beyond bags of features: Spatial pyramid matching for
recognizing natural scene categories, 2006 IEEE Computer Society Conference on Computer
Vision and Pattern Recognition (CVPR'06), Vol. 2, IEEE, 2006, pp. 2169-2178.
[77] S.C. Lee, R. Nevatia, Extraction and integration of window in a 3D building model from ground
view images, Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the
2004 IEEE Computer Society Conference on, Vol. 2, IEEE, 2004, pp. II-113-II-120 Vol. 112.
[78] T. Leung, J. Malik, Representing and recognizing the visual appearance of materials using three-
dimensional textons, International journal of computer vision 43 (1) (2001) 29-44.
[79] J. Li, W. Huang, L. Shao, N. Allinson, Building recognition in urban environments: A survey of
state-of-the-art and future challenges, Information Sciences 277 (2014) 406-420.
[80] S. Loch-Dehbi, L. Plümer, Predicting building façade structures with multilinear Gaussian
graphical models based on few observations, Computers, Environment and Urban Systems 54
(2015) 68-81.
[81] D.G. Lowe, Object recognition from local scale-invariant features, Computer vision, 1999. The
proceedings of the seventh IEEE international conference on, Vol. 2, Ieee, 1999, pp. 1150-1157.
[82] A. Martinović, M. Mathias, J. Weissenberg, L. Van Gool, A three-layered approach to facade
parsing, European conference on computer vision, Springer, 2012, pp. 416-429.
[83] A. Martinovic, L. Van Gool, Bayesian grammar learning for inverse procedural modeling,
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 201-
208.
[84] P. Müller, P. Wonka, S. Haegler, A. Ulmer, L. Van Gool, Procedural modeling of buildings, Acm
Transactions On Graphics (Tog) 25 (3) (2006) 614-623.
[85] P. Muller, G. Zeng, P. Wonka, L.V. Gool, Image-based procedural modeling of facades, ACM
Trans. Graph. 26 (3) (2007) 85.
[86] P. Musialski, P. Wonka, D.G. Aliaga, M. Wimmer, L. Gool, W. Purgathofer, A survey of urban
reconstruction, Computer graphics forum, Vol. 32, Wiley Online Library, 2013, pp. 146-177.
[87] D. Ok, M. Kozinski, R. Marlet, N. Paragios, High-Level Bottom-Up Cues for Top-Down Parsing
of Facade Images, 2012 Second International Conference on 3D Imaging, Modeling, Processing,
Visualization & Transmission, 2012, pp. 128-135.
[88] M.J. Olsen, E. Johnstone, N. Driscoll, S.A. Ashford, F. Kuester, Terrestrial laser scanning of
extended cliff sections in dynamic environments: Parameter analysis, Journal of Surveying
Engineering 135 (4) (2009) 161-169.

138
[89] P. Oskouie, B. Becerik-Gerber, L. Soibelman, Automated Cleaning of Point Clouds for Highway
Retaining Wall Condition Assessment, 2014 International Conference on Computing in Civil and
Building Engineering, 2014.
[90] P. Oskouie, B. Becerik-Gerber, L. Soibelman, Automated measurement of highway retaining wall
displacements using terrestrial laser scanners, Automation in Construction (2016).
[91] P. Oskouie, B. Becerik-Gerber, L. Soibelman, A Data Quality-driven Framework for Asset
Condition Assessment Using LiDAR and Image Data, ASCE Computing in Civil Engineering,
2015, pp. 240-248.
[92] P. Oskouie, B. Becerik-Gerber, L. Soibelman, A Data Quality-driven Framework for Asset
Condition Assessment Using LiDAR and Image Data, ASCE International Workshop on
Computing in Civil Engineering, ASCE, Austin, TX, 2015.
[93] P. Oskouie, B. Becerik-Gerber, L. Soibelman, A Data Quality-driven Framework for Asset
Condition Assessment Using LiDAR and Image Data, ASCE Computing in Civil Engineering,
2015, pp. 240-248.
[94] N. Otsu, A threshold selection method from gray-level histograms, Automatica 11 (285-296)
(1975) 23-27.
[95] P. Passe D., Mechanically Stabilized Earth Wall Inspector's Handbook, Florida Department of
Transportation, 2000.
[96] V. Pătrăucean, I. Armeni, M. Nahangi, J. Yeung, I. Brilakis, C. Haas, State of research in automatic
as-built modelling, Advanced Engineering Informatics 29 (2) (2015) 162-171.
[97] A.D. Payne, A.A. Dorrington, M.J. Cree, D.A. Carnegie, Characterization of modulated time-of-
flight range image sensors, IS&T/SPIE Electronic Imaging, International Society for Optics and
Photonics, 2009, pp. 723904-723904-723911.
[98] A. Pradhan, F. Moon, Formalized Approach for Accurate Geometry Capture through Laser
Scanning, Computing in Civil Engineering (2013), 2013, pp. 597-604.
[99] A.R. Pradhan, F.L. Moon, Formalized Approach for Accurate Geometry Capture through Laser
Scanning, Computing in Civil Engineering (2013) (2013) 597-604.
[100] S. Pu, G. Vosselman, Knowledge based reconstruction of building models from terrestrial laser
scanning data, ISPRS Journal of Photogrammetry and Remote Sensing 64 (6) (2009) 575-584.
[101] M. Recky, F. Leberl, Windows Detection Using K-means in CIE-Lab Color Space, Pattern
Recognition (ICPR), 2010 20th International Conference on, 2010, pp. 356-359.
[102] RIEGL, RIEGL-VZ-400 Technical Specifications.
[103] H. Riemenschneider, U. Krispel, W. Thaller, M. Donoser, S. Havemann, D. Fellner, H. Bischof,
Irregular lattices for complex shape grammar facade parsing, Computer Vision and Pattern
Recognition (CVPR), 2012 IEEE Conference on, IEEE, 2012, pp. 1640-1647.
[104] P. Rodriguez-Gonzalvez, D. Gonzalez-Aguilera, G. Lopez-Jimenez, I. Picon-Cabrera, Image-based
modeling of built environment from an unmanned aerial system, Automation in Construction 48
(0) (2014) 44-52.
[105] G. Roth, M.D. Levine, Extracting geometric primitives, CVGIP: Image Understanding 58 (1)
(1993) 1-22.
[106] R.B. Rusu, Semantic 3d object maps for everyday manipulation in human living environments, KI-
Künstliche Intelligenz 24 (4) (2010) 345-348.
[107] R.B. Rusu, S. Cousins, 3d is here: Point cloud library (pcl), Robotics and automation (ICRA), 2011
IEEE International Conference on, IEEE, 2011, pp. 1-4.
[108] R.B. Rusu, Z.C. Marton, N. Blodow, M. Dolha, M. Beetz, Towards 3D Point cloud based object
maps for household environments, Robotics and Autonomous Systems 56 (11) (2008) 927-941.
[109] N.C. Samtani, E.A. Nowatzki, Mechanically Stabilized Earth Walls and Reinforced Soil Slopes
Design & Construction Guidelines, FHWA-NHI-00-0043, Federal Highway Administration,
Washington, D.C., 2001.

139
[110] C. Schmid, Constructing models for content-based image retrieval, Computer Vision and Pattern
Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on,
Vol. 2, IEEE, 2001, pp. II-39-II-45 vol. 32.
[111] W.R. Scott, G. Roth, J.-F. Rivest, View planning for automated three-dimensional object
reconstruction and inspection, ACM Computing Surveys (CSUR) 35 (1) (2003) 64-96.
[112] A. Serna, J. Hern, x00E, ndez, B. Marcotegui, Adaptive parameter tuning for morphological
segmentation of building facade images, Signal Processing Conference (EUSIPCO), 2012
Proceedings of the 20th European, 2012, pp. 2268-2272.
[113] J. Shen, Synthesization of edge noises for touch probe and laser sensor measurement, Computer-
Aided Design and Applications 4 (1-4) (2007) 247-256.
[114] Z. Shen, P. Tang, O. Kanaan, Y. Cho, As-Built Error Modeling for Effective 3D Laser Scanning
on Construction Sites, Computing in Civil Engineering (2013), ASCE, 2013, pp. 533-540.
[115] Z. Shen, P. Tang, O. Kanaan, Y. Cho, As-Built Error Modeling for Effective 3D Laser Scanning
on Construction Sites, Computing in Civil Eng., 2013, pp. 533-540.
[116] H. Shimazaki, S. Shinomoto, A method for selecting the bin size of a time histogram, Neural
computation 19 (6) (2007) 1503-1527.
[117] M. Song, Z. Shen, P. Tang, Data quality-oriented 3D laser scan planning, Construction Research
Congress 2014: Construction in a Global Network, 2014, pp. 984-993.
[118] M. Song, Z. Shen, P. Tang, Data Quality-oriented 3D Laser Scan Planning, Construction Research
Congress 2014, 2014, pp. 984-993.
[119] S. Soudarissanane, R. Lindenbergh, M. Menenti, P. Teunissen, Incidence angle influence on the
quality of terrestrial laser scanning points, Proceedings ISPRS Workshop Laserscanning 2009, 1-
2 Sept 2009, Paris, France, ISPRS, 2009.
[120] S. Soudarissanane, J. Van Ree, A. Bucksch, R. Lindenbergh, Error budget of terrestrial laser
scanning: influence of the incidence angle on the scan quality, Proceedings 3D-NordOst (2007) 1-
8.
[121] G. Stiny, Introduction to shape and shape grammars, Environment and planning B 7 (3) (1980)
343-351.
[122] A. Stuedlein, M. Bailey, D. Lindquist, J. Sankey, W. Neely, Design and Performance of a 46-m-
High MSE Wall, Journal of Geotechnical and Geoenvironmental Engineering 136 (6) (2010) 786-
796.
[123] Y. Su, Y. Hashash, L. Liu, Integration of Construction As-Built Data Via Laser Scanning with
Geotechnical Monitoring of Urban Excavation, Journal of Construction Engineering and
Management 132 (12) (2006) 1234-1241.
[124] X. Sun, P. Rosin, R. Martin, F. Langbein, Fast and effective feature-preserving mesh denoising,
IEEE transactions on visualization and computer graphics 13 (5) (2007).
[125] X. Sun, P.L. Rosin, R.R. Martin, F.C. Langbein, Noise analysis and synthesis for 3D laser depth
scanners, Graphical Models 71 (2) (2009) 34-48.
[126] P. Tang, B. Akinci, Automatic execution of workflows on laser-scanned data for extracting bridge
surveying goals, Advanced Engineering Informatics 26 (4) (2012) 889-903.
[127] P. Tang, B. Akinci, Extracting Surveying Goals from Point Clouds to Support Construction and
Infrastructure Inspection, Construction Research Congress 2009, 2009, pp. 1164-1173.
[128] P. Tang, B. Akinci, D. Huber, Quantification of edge loss of laser scanned data at spatial
discontinuities, Automation in Construction 18 (8) (2009) 1070-1083.
[129] P. Tang, F.S. Alaswad, Sensor modeling of laser scanners for automated scan planning on
construction jobsites, Amr Kandil and Cai, Hubo (arg.). Construction Research Congress, 2012.
[130] P. Tang, D. Huber, B. Akinci, A comparative analysis of depth-discontinuity and mixed-pixel
detection algorithms, 3-D Digital Imaging and Modeling, 2007. 3DIM'07. Sixth International
Conference on, IEEE, 2007, pp. 29-38.

140
[131] P. Tang, D. Huber, B. Akinci, R. Lipman, A. Lytle, Automatic reconstruction of as-built building
information models from laser-scanned point clouds: A review of related techniques, Automation
in Construction 19 (7) (2010) 829-843.
[132] O. Teboul, I. Kokkinos, L. Simon, P. Koutsourakis, N. Paragios, Shape grammar parsing via
reinforcement learning, Computer Vision and Pattern Recognition (CVPR), 2011 IEEE
Conference on, IEEE, 2011, pp. 2273-2280.
[133] A. Torralba, K.P. Murphy, W.T. Freeman, M.A. Rubin, Context-based vision system for place and
object recognition, Computer Vision, 2003. Proceedings. Ninth IEEE International Conference on,
IEEE, 2003, pp. 273-280.
[134] R. Tyleček, R. Šára, Spatial pattern templates for recognition of objects with regular structure,
German Conference on Pattern Recognition, Springer, 2013, pp. 364-374.
[135] US-GSA, BIM Guide for 3D Imaging, http://www.gsa.gov (2009).
[136] E. Valero, A. Adán, F. Bosché, Semantic 3D Reconstruction of Furnished Interiors Using Laser
Scanning and RFID Technology, Journal of Computing in Civil Engineering (2015) 04015053.
[137] M. Varma, A. Zisserman, Texture classification: Are filter banks necessary?, Computer vision and
pattern recognition, 2003. Proceedings. 2003 IEEE computer society conference on, Vol. 2, IEEE,
2003, pp. II-691-698 vol. 692.
[138] P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, Computer
Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer
Society Conference on, Vol. 1, 2001, pp. I-511-I-518 vol.511.
[139] R. Volk, J. Stengel, F. Schultmann, Building Information Modeling (BIM) for existing buildings—
Literature review and future needs, Automation in Construction 38 (2014) 109-127.
[140] C. Wang, Y.K. Cho, C. Kim, Automatic BIM component extraction from point clouds of existing
buildings for sustainability applications, Automation in Construction 56 (2015) 1-13.
[141] X. Wang, A.R. Hanson, Surface texture and microstructure extraction from multiple aerial images,
Computer Vision and Image Understanding 83 (1) (2001) 1-37.
[142] Z. Wang, L. Zhang, T. Fang, P.T. Mathiopoulos, X. Tong, H. Qu, Z. Xiao, F. Li, D. Chen, A
multiscale and hierarchical feature extraction method for terrestrial laser scanning point cloud
classification, IEEE Transactions on Geoscience and Remote Sensing 53 (5) (2015) 2409-2425.
[143] M. Weinmann, M. Weinmann, S. Hinz, B. Jutzi, Fast and automatic image-based registration of
TLS data, ISPRS Journal of Photogrammetry and Remote Sensing 66 (6) (2011) S62-S70.
[144] C. Witzgall, S.Cheok, G., Experiences with point cloud registration, 2002.
[145] P. Wonka, M. Wimmer, F. Sillion, W. Ribarsky, Instant architecture, ACM, 2003.
[146] C. Wu, VisualSFM: A Visual Structure from Motion System, 2011.
[147] X. Xiong, A. Adan, B. Akinci, D. Huber, Automatic creation of semantically rich 3D building
models from laser scanner data, Automation in Construction 31 (2013) 325-337.
[148] A. Yakubenko, I. Mizin, A. Konushin, Automatic Extraction of Regular Grids from Rectified
Facade Image, в трудах конференции Графикон, 2010, pp. 100-106.
[149] Y. Yu, K. Zhou, D. Xu, X. Shi, H. Bao, B. Guo, H.-Y. Shum, Mesh editing with poisson-based
gradient field manipulation, ACM Transactions on Graphics (TOG), Vol. 23, ACM, 2004, pp. 644-
651.
[150] C. Zhang, V.S. Kalasapudi, P. Tang, Rapid data quality oriented laser scan planning for dynamic
construction environments, Advanced Engineering Informatics 30 (2) (2016) 218-232.
[151] R. Zhang, A. Zakhor, Automatic identification of window regions on indoor point clouds using
LiDAR and cameras, IEEE Winter Conference on Applications of Computer Vision, 2014, pp.
107-114.

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Semantic modeling of outdoor scenes for the creation of virtual environments and simulations

PDF

Point cloud data fusion of RGB and thermal information for advanced building envelope modeling in support of energy audits for large districts

PDF

Understanding human-building-emergency interactions in the built environment

PDF

A framework for comprehensive assessment of resilience and other dimensions of asset management in metropolis-scale transport systems

PDF

Towards health-conscious spaces: building for human well-being and performance

PDF

3D deep learning for perception and modeling

PDF

Scanning using a smart phone for heritage conservation: a case study using the Reunion House

PDF

Understanding human-building interactions through perceptual decision-making processes

PDF

Behavioral form finding using multi-agent systems: a computational methodology for combining generative design with environmental and structural analysis in architectural design

PDF

User-centric smart sensing for non-intrusive electricity consumption disaggregation in buildings

PDF

A radio frequency based indoor localization framework for supporting building emergency response operations

PDF

Green learning for 3D point cloud data processing

PDF

Machine learning-driven deformation prediction and compensation for additive manufacturing

PDF

Point-based representations for 3D perception and reconstruction

PDF

3D object detection in industrial site point clouds

PDF

Modeling the spatio-temporal variability of solar radiation on buildings: a case study of Lewis Hall

PDF

Face recognition and 3D face modeling from images in the wild

PDF

Learning controllable data generation for scalable model training

PDF

Data-driven 3D hair digitization

PDF

Differentially private learned models for location services

Asset Metadata

Creator Oskouie, Pedram (author)

Core Title In-situ quality assessment of scan data for as-built models using building-specific geometric features

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Civil Engineering

Publication Date 09/21/2019

Defense Date 08/07/2017

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag 3D feature extraction,3D point cloud,building as-built modeling,building information modeling,data quality assessment,image processing,laser scanning,OAI-PMH Harvest

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Becerik-Gerber, Burcin (committee chair), Madni, Azad (committee member), Soibelman, Lucio (committee member)

Creator Email oskouie@usc.edu,pedram.oskouie@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c40-432017

Unique identifier UC11265158

Identifier etd-OskouiePed-5747.pdf (filename),usctheses-c40-432017 (legacy record id)

Legacy Identifier etd-OskouiePed-5747.pdf

Dmrecord 432017

Document Type Dissertation

Rights Oskouie, Pedram

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA