University of Southern California Dissertations and Theses

BIM+AR in architecture: a building maintenance application for a smart phone

(USC Thesis Other)

BIM+AR in architecture: a building maintenance application for a smart phone

PDF

Download

Open document

Flip pages

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

BIM+AR in Architecture

A Building Maintenance Application for a Smart Phone

by

Ruisong Zheng

A Thesis Presented to the
FACULTY OF THE USC SCHOOL OF ARCHITECTURE
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
MASTER OF BUILDING SCIENCE

May 2022

Copyright 2022 Ruisong Zhen
ii

ACKNOWLEDGEMENTS

I must thank my parents; they provided the full support for my three-year study at USC.
I would first like to thank my thesis chair Karen Kensek (kensek@usc.edu, USC School of
Architecture). She consistently allowed this paper to be my own work but steered me in the right
direction whenever she thought I needed it. Teaching me to divide a huge challenge into lots of
little independent problems and solve them.

I would also like to thank Professor Marc Schiler (marcs@usc.edu, USC School of Architecture)
and Professor Joon-Ho Choi (joonhoch@usc.edu USC School of Architecture). Without their
passionate participation and input, the validation survey could not have been successfully
conducted.

Finally, I must express my very profound gratitude to all the faculty and students in MBS for
providing me with unfailing support and continuous encouragement throughout my years of study
and through the process of researching and writing this thesis. This accomplishment would not
have been possible without them. Thank you.

iii

TABLE OF CONTENTS
ACKNOWLEDGEMENTS .......................................................................................................................................... ii
TABLE OF CONTENTS ............................................................................................................................................ iii
LIST OF TABLES...................................................................................................................................................... vii
LIST OF FIGURES ................................................................................................................................................... viii
ABSTRACT .............................................................................................................................................................. xiii
Chapter 1 ....................................................................................................................................................................... 1
Introduction ................................................................................................................................................................... 1
1.1 AR (Augmented Reality) ......................................................................................................................................... 1
1.1.1 History of AR ............................................................................................................................................... 5
1.1.2 Application of AR in Architecture ............................................................................................................... 6
1.1.3 Application of AR (Augment Reality) Software ........................................................................................ 11
1.1.4 Methods to Show AR Model ..................................................................................................................... 13
1.2 VR (Virtual Reality) and MR (Mixed Reality) ...................................................................................................... 14
1.2.1 VR (Virtual Reality) .................................................................................................................................. 15
1.3 BIM (Building Information Modeling) .................................................................................................................. 20
1.3.1 Development of BIM ................................................................................................................................. 20
1.3.2 Application of BIM .................................................................................................................................... 21
1.4 Unity 3D ................................................................................................................................................................ 23
1.4.1 Connecting Unity 3D with BIM ................................................................................................................. 25
1.5.1 Traditional Building Maintenance ............................................................................................................. 27
1.5.2 BIM+AR in Building Maintenance and Facility Management .................................................................. 30
1.5.3 One Sample Application of BIM+AR in Pipeline Maintenance (Pei-Huang, 2019) ................................. 32
1.5.4 BIM in Unity with AR ............................................................................................................................... 34
1.6 Summary ................................................................................................................................................................ 34
Chapter 2 ..................................................................................................................................................................... 36
Introduction ................................................................................................................................................................. 36
2.1 BIM and Facility Management .............................................................................................................................. 36
2.1.1 BIM in Architectural Design ...................................................................................................................... 37
2.1.2 Facilities management Based on BIM ....................................................................................................... 39
2.1.3 Limitations and Improvements of BIM Technology .................................................................................. 41
2.2 Augmented Reality ................................................................................................................................................ 42
2.2.1 AR in Architecture ..................................................................................................................................... 42
2.2.2 AR in Construction Management............................................................................................................... 45
2.2.3 AR for underground infrastructure (Sánchez, 2015) .................................................................................. 46
2.2.4 Combine AR with other new technologies in the field of construction ..................................................... 47
2.3 BIM and AR .......................................................................................................................................................... 51
2.3.1 BIM + AR in Building Maintenance and FM ............................................................................................ 52
iv

2.3.2 AR and BIM in Architecture ...................................................................................................................... 56
2.3.3 BIM and AR in Whole Building Life Cycle .............................................................................................. 57
2.4 Traditional Facility Management and BIM with AR Enables Facility Management ............................................ 58
2.4.1 Traditional Facility Management Method.................................................................................................. 59
2.4.2 Future of BIM and AR in FM .................................................................................................................... 59
2.5 Summary ................................................................................................................................................................ 62
Chapter 3: Methodology .............................................................................................................................................. 64
Introduction ................................................................................................................................................................. 64
3.1 Workflow and Methodology .................................................................................................................................. 64
3.2 Revit Part ............................................................................................................................................................... 65
3.2.1 Load and Modify Watt Hall Model ............................................................................................................ 67
3.2.2 Using Gamma AR to Check the Model ..................................................................................................... 68
3.2.3 Export Model to. FBX ............................................................................................................................... 70
3.3 3ds Max Part .......................................................................................................................................................... 71
3.3.1 Import Revit file into 3ds Max ................................................................................................................... 73
3.3.2 Change Materials to Standard .................................................................................................................... 74
3.3.3 Check Render Diagram to Confirm the Materials Convert Successfully ................................................... 75
3.3.4 Reset the Pivot Point .................................................................................................................................. 76
3.3.5 Create Animation ....................................................................................................................................... 78
3.3.6 Save the Texture in Local Disk .................................................................................................................. 84
3.4 Unity 3D Part ......................................................................................................................................................... 85
3.4.1 Import and Modification of FBX file from 3dx Max ................................................................................. 86
3.4.2 Marking Building Model ........................................................................................................................... 88
3.4.3 Edit Information of Models........................................................................................................................ 89
3.4.4 Set up a Start point ..................................................................................................................................... 90
3.4.5 Create the Application AR-FM .................................................................................................................. 93
3.4.6 Package the Project .................................................................................................................................... 98
3.5 User Interface ...................................................................................................................................................... 100
3.7 Summary .............................................................................................................................................................. 101
Chapter 4 ................................................................................................................................................................... 103
Introduction ............................................................................................................................................................... 103
4.1 Final Tool AR-FM: User Interface ...................................................................................................................... 105
4.2 Welcome Panel .................................................................................................................................................... 106
4.3 Operation Scene for AR-FM ............................................................................................................................... 109
4.3.1 Select the Component .............................................................................................................................. 110
4.3.2 Edit the Component ................................................................................................................................. 111
4.3.3 Highlight the Target Component ............................................................................................................. 113
4.3.4 Using Start Point to Locate and Load the Watt Hall Model..................................................................... 114
v

4.3.5 Hide or Show the Target Component ....................................................................................................... 118
4.3.6 Turn on or turn off the Camera ................................................................................................................ 119
4.3.7 Showing Information about Components and Direction of Operation..................................................... 120
4.3.8 Playing Animation and Image .................................................................................................................. 122
4.4 Made the Tool AR-FM ........................................................................................................................................ 125
4.4.1 UIController ............................................................................................................................................. 125
4.4.2 Highlighter.cs ........................................................................................................................................... 127
4.4.3 Camera Tracking in Both Real-world and Visual World ......................................................................... 128
4.4.4 UI Controller ............................................................................................................................................ 129
4.5 Create AR-FM ..................................................................................................................................................... 130
4.6 Summary .............................................................................................................................................................. 131
Chapter 5 ................................................................................................................................................................... 133
Introduction ............................................................................................................................................................... 133
5.1 Model Preparation Process in Revit and 3dsMax ................................................................................................ 133
5.1.1 Load Watt Hall Model ............................................................................................................................. 133
5.1.3 Add Core Components ............................................................................................................................. 135
5.1.5 Import Revit file into 3ds Max ................................................................................................................. 136
5.1.6 Change Materials to Standard .................................................................................................................. 137
5.2 Import 3D Model to AR-FM ............................................................................................................................... 138
5.2.1 Set up Start Point ..................................................................................................................................... 140
5.2.2 Unpack the Prefab .................................................................................................................................... 141
5.2.3 Make AR-FM as the Parent of Watt Hall Model ..................................................................................... 142
5.3 Build the Tool to Android System ....................................................................................................................... 144
5.3.1 Android System Setting ........................................................................................................................... 144
5.3.2 Convert the Application from Laptop to Android Device ....................................................................... 146
5.4 Case Study in Watt Hall ...................................................................................................................................... 147
5.4.1 Welcome Panel ........................................................................................................................................ 149
5.4.2 Load AR Model at Start Point .................................................................................................................. 150
5.4.3 Back to Welcome Panel ........................................................................................................................... 151
5.4.4 Edit Component ....................................................................................................................................... 151
5.4.5 Show Direction of AR-FM ...................................................................................................................... 154
5.4.6 Turn on/ off Camera ................................................................................................................................. 155
5.4.7 Show and Hide Component ..................................................................................................................... 155
5.4.8 Toolbar Operation .................................................................................................................................... 156
5.4.9 Special Function for the Component in Green ......................................................................................... 157
5.4.10 Special Function for the Wall with Components Inside ......................................................................... 158
5.5 Feedback of Volunteers ....................................................................................................................................... 161
5.6 Summary .............................................................................................................................................................. 165
vi

Chapter 6 ................................................................................................................................................................... 167
Introduction ............................................................................................................................................................... 167
6.1 Research Background and Methodology ............................................................................................................. 167
6.2 Limitations and Future work ............................................................................................................................... 174
6.2.1 Limitation of the Animation .................................................................................................................... 175
6.2.2 Limitation for Performance of AR-FM .................................................................................................... 175
6.2.3 Filter Unimportant Elements .................................................................................................................... 176
6.2.4 Navigation System to AR-FM ................................................................................................................. 176
6.2.5 Short-Term Plan ....................................................................................................................................... 177
6.2.6 Long-Term Plan ....................................................................................................................................... 177
6.3 Discussion of Results ........................................................................................................................................... 178
6.4 Analyze the Whole FM Process by Using AR-FM.............................................................................................. 178
6.4.1 Understanding the FM Task ..................................................................................................................... 179
6.4.2 Planning for the FM Task ........................................................................................................................ 180
6.4.3 Acting for Real Work ............................................................................................................................... 180
6.4.4 Giving Feedback to System ..................................................................................................................... 181
6.5 Conclusion ........................................................................................................................................................... 181
6.6 Summary .............................................................................................................................................................. 183
REFERENCES .......................................................................................................................................................... 184

vii

LIST OF TABLES
Table 1. 1 Application of AR software in sorts of fields .............................................................. 12
Table 1. 2 AR APPs in many fields .............................................................................................. 12

Table 4. 1 Software and plugins to create AR-FM ..................................................................... 103
Table 4. 2 List of C# scripts in AR-FM (All scripts were created in Visual Studio) ................. 104
Table 4. 3 Features and their corresponding scripts ................................................................... 125
Table 4. 4 Software and plugins to create AR-FM ..................................................................... 131

Table 5. 1 Feedback of Volunteer #1 .......................................................................................... 162
Table 5. 2 Feedback of Volunteer #2 .......................................................................................... 163
Table 5. 3 Feedback of Volunteer #3 .......................................................................................... 163
Table 5. 4 Feedback of Volunteer #4 .......................................................................................... 164
Table 5. 5 Feedback of Volunteer #5 .......................................................................................... 164

Table 6. 1 Scripts created for AR-FM ........................................................................................ 170

viii

LIST OF FIGURES
Figure 1. 1 AR model shown by camera......................................................................................... 3
Figure 1. 2 AR glass to show AR model ........................................................................................ 3
Figure 1. 3 New Challenges in Retail and How 3D And AR Help ................................................ 4
Figure 1. 4 Using AR showing in mobile game Pokemon Go........................................................ 5
Figure 1. 5 Kubity showing AR ...................................................................................................... 6
Figure 1. 6 Using digital sand to show AR model in design .......................................................... 7
Figure 1. 7 Construction VR ........................................................................................................... 8
Figure 1. 8 AR technology for comprehensive components ........................................................... 9
Figure 1. 9 AR apply in a construction site .................................................................................... 9
Figure 1. 10 Using AR to preview the house ................................................................................ 10
Figure 1. 11 IKEA application to preview furniture ..................................................................... 11
Figure 1. 12 Using floor plan of the building picture to show Whole AR model ........................ 13
Figure 1. 13 Put AR model ........................................................................................................... 14
Figure 1. 14 AR, VR and XR ........................................................................................................ 14
Figure 1. 15 the major milestones in the journey of virtual reality ............................................... 15
Figure 1. 16 Using VR device to play game Arizona Sunshine ................................................... 16
Figure 1. 17 VR device with Enscape ........................................................................................... 17
Figure 1. 18 VR in Horizon World ............................................................................................... 18
Figure 1. 19 Virtual Reality- The Next Frontier in Safety Training ............................................. 19
Figure 1. 20 VR in Horizon World ............................................................................................... 20
Figure 1. 21 Development of BIM (Aryani, 2014) ....................................................................... 21
Figure 1. 22 Development of BIM ................................................................................................ 24
Figure 1. 23 Unity 3D for VR ....................................................................................................... 25
Figure 1. 24 Import BIM Model into Unity 3D ............................................................................ 26
Figure 1. 25 Traditional Building Maintenance ............................................................................ 28
Figure 1. 26 BIM+AR facility management ................................................................................. 30
Figure 1. 27 How BIM + AR applying (Wang, 2019) .................................................................. 32
Figure 1. 28 System structure and modules of BARMS (Huang, 2019) ...................................... 33

Figure 2. 1 BIM for Facility Management program at GEO ........................................................ 37
Figure 2. 2 Proposed model in this research (Tulubas Gokuc, 2017) ........................................... 38
Figure 2. 3 Process architecture (Shalabi et al, 2017) ................................................................... 40
Figure 2. 4 Using AR to show architecture model........................................................................ 43
Figure 2. 5 Design process of the application Cal Poly AR ......................................................... 43
Figure 2. 6 Visual tour of Forbidden City..................................................................................... 44
Figure 2. 7 Major components of the proposed Social BIMCloud framework ............................ 46
Figure 2. 8 Workers using AR technology to create curved benches ........................................... 49
Figure 2. 9 Applications of Augmented Reality Integrations for Construction and Design......... 51
Figure 2. 10 Using AR to show site information (Wang, 2014) ................................................... 53
Figure 2. 11 Add AR model in real site (Wang, 2014) ................................................................. 53
Figure 2. 12 Using AR technology to help workers recognize a component (Posada, 2015) ...... 54
Figure 2. 13 Content viewer for the navigation (Gerstweiler, 2016) ............................................ 56
Figure 2. 14 Using AR technology show the pipe system ............................................................ 57
Figure 2. 15 BIM+AR work in whole life cycle of a building (Kuula, 2012) .............................. 58

ix

Figure 3. 1 Methodology............................................................................................................... 65
Figure 3. 2 Revit part diagram ...................................................................................................... 66
Figure 3. 3 Whole Model of Watt Hall ......................................................................................... 67
Figure 3. 4 Third floor of Watt Hall ............................................................................................. 67
Figure 3. 5 Add pipes to Watt Hall mode ..................................................................................... 68
Figure 3. 6 Upload IFC file in Gamma AR on the website .......................................................... 69
Figure 3. 7 Using Gamma AR to locate the model ....................................................................... 70
Figure 3. 8 Model that directly import from Revit to Unity 3D ................................................... 71
Figure 3. 9 Model after modification in 3ds Max then import into Unity 3D .............................. 71
Figure 3. 10 Diagram of 3ds Max part .......................................................................................... 72
Figure 3. 11 Import FBX file ........................................................................................................ 73
Figure 3. 12 Set up import option ................................................................................................. 74
Figure 3. 13 Successfully imported model in 3ds Max ................................................................ 74
Figure 3. 14 Change the material to standard ............................................................................... 75
Figure 3. 15 Check the transform of building material ................................................................. 76
Figure 3. 16 Before resetting pivot point for one component ....................................................... 77
Figure 3. 17 After resetting pivot point for one component ......................................................... 77
Figure 3. 18 Before resetting whole pivot point ........................................................................... 77
Figure 3. 19 After resetting whole pivot point .............................................................................. 78
Figure 3. 20 Animation creation area in 3ds Max ........................................................................ 78
Figure 3. 21 Set up of animation creation .................................................................................... 79
Figure 3. 22 Move six pipes outside the wall ............................................................................... 80
Figure 3. 23 Separate six pipes to single model............................................................................ 80
Figure 3. 24 Step1(Uninstall)-Remove the valve ......................................................................... 81
Figure 3. 25 Step2 (Uninstall)-Move pipe 1 ................................................................................. 81
Figure 3. 26 Step3 (Uninstall)-Remove the target item ................................................................ 82
Figure 3. 27 Step4 (Uninstall)-Replace a new item ...................................................................... 82
Figure 3. 28 Step5 (Install)-Move the pipe back .......................................................................... 83
Figure 3. 29 Step6 (Install)-Put the valve above water line .......................................................... 83
Figure 3. 30 Step7 (Install)-Tighten the valve on the pipe ........................................................... 84
Figure 3. 31 Save rendering file .................................................................................................... 85
Figure 3. 32 Diagram of Unity 3D work process .......................................................................... 86
Figure 3. 33 Plugins in Unity 3D .................................................................................................. 87
Figure 3. 34 AR Session Origin .................................................................................................... 88
Figure 3. 35 Final in Unity 3D ...................................................................................................... 89
Figure 3. 36 Using pop-window to show material information .................................................... 90
Figure 3. 37 Set up for start point ................................................................................................. 91
Figure 3. 38 Set up for start cube .................................................................................................. 92
Figure 3. 39 Diagram of creating AR-FM .................................................................................... 93
Figure 3. 40 Highlight function of AR-FM .................................................................................. 94
Figure 3. 41 Moving, rotating and scaling function of AR-FM.................................................... 94
Figure 3. 42 Turn on camera of AR-FM (Left), Turn off camera of AR-FM (Right) .................. 94
Figure 3. 43 Welcome panel of AR-FM ....................................................................................... 95
Figure 3. 44 Add picture on the wall ............................................................................................ 95
Figure 3. 45 Panel on the Wall to select pipe ............................................................................... 96
Figure 3. 46 Selecting the pipe by buttons on panel ..................................................................... 96
x

Figure 3. 47 Add pipe picture on the wall .................................................................................... 97
Figure 3. 48 Render components in Unity 3D .............................................................................. 97
Figure 3. 49 Setting the model ...................................................................................................... 98
Figure 3. 50 Building setting for package up ................................................................................ 99
Figure 3. 51 Modification in project setting ................................................................................. 99
Figure 3. 52 Welcome panel ....................................................................................................... 100
Figure 3. 53 AR model operation panel ...................................................................................... 100

Figure 4. 1 Welcome panel for AR-FM ...................................................................................... 104
Figure 4. 2 Scripts applied in each function ............................................................................... 105
Figure 4. 3 Welcome panel for AR-FM ...................................................................................... 106
Figure 4. 4 Operation scene for AR-FM ..................................................................................... 106
Figure 4. 5 Text box and picture in welcome panel .................................................................... 107
Figure 4. 6 Editor for welcome panel ......................................................................................... 107
Figure 4. 7 Publish setting of UI ................................................................................................. 108
Figure 4. 8 Importing UI panel in Unity 3D ............................................................................... 108
Figure 4. 9 Change building environment to Android ................................................................ 109
Figure 4. 10 Scripts in UI panel .................................................................................................. 110
Figure 4. 11 Operation scene for AR-FM ................................................................................... 110
Figure 4. 12 Mesh Collider and Mesh Renderer for AR-FMM .................................................. 111
Figure 4. 13 Moving mode.......................................................................................................... 112
Figure 4. 14 Rotation mode ........................................................................................................ 112
Figure 4. 15 Scaling mode .......................................................................................................... 112
Figure 4. 16 Plugin to show the operation model ....................................................................... 113
Figure 4. 17 Using button to select pipe and highlight it ............................................................ 114
Figure 4. 18 Script “Highlighter” in closed state ........................................................................ 114
Figure 4. 19 View of start point in AR-FM ................................................................................ 115
Figure 4. 20 Import ARFoundation ............................................................................................ 115
Figure 4. 21 Add AR Session Origin and AR Session into project ............................................ 116
Figure 4. 22 AR Session Origin .................................................................................................. 116
Figure 4. 23 AR Session ............................................................................................................. 117
Figure 4. 24 Hide cube when finish setting the location of ARFoundation plugins ................... 117
Figure 4. 25 Switch of hiding or showing component ................................................................ 118
Figure 4. 26 Controlling hiding or showing component ............................................................. 118
Figure 4. 27 Turn off the camera ................................................................................................ 119
Figure 4. 28 Turn on the camera ................................................................................................. 120
Figure 4. 29 Click to show information of component ............................................................... 121
Figure 4. 30 Showing direction ................................................................................................... 121
Figure 4. 31 Add information of some components ................................................................... 122
Figure 4. 32 Panel to select animation to play ............................................................................ 123
Figure 4. 33 Animation to guide users how to replace valve with text explanation ................... 123
Figure 4. 34 Import three videos ................................................................................................. 124
Figure 4. 35Video Player to control reading videos ................................................................... 124
Figure 4. 36 Click the button with component behand ............................................................... 126
Figure 4. 37 Judge whether users click the component or button ............................................... 126
Figure 4. 38 Link the pipes with corresponding water pipe ....................................................... 127
xi

Figure 4. 39 Clear all the highlight when click the pillars .......................................................... 128
Figure 4. 40 Camera Tracking in AR-FM .................................................................................. 128
Figure 4. 41 Judge whether the target component has subset before further work .................... 130
Figure 4. 42 UI panel in project AR-FM .................................................................................... 131
Figure 4. 43 Transfer different operation system for AR-FM .................................................... 131
Figure 4. 44 Workflow about adding scripts to functions .......................................................... 132

Figure 5. 1 Whole Model of Watt Hall ....................................................................................... 134
Figure 5. 2 Third floor of Watt Hall ........................................................................................... 134
Figure 5. 3 Watt Hall model after removing unnecessary elements ........................................... 135
Figure 5. 4 Add pipes to Watt Hall model .................................................................................. 136
Figure 5. 5 Check the transform of building material ................................................................. 138
Figure 5. 6 Import plug-in into project ....................................................................................... 139
Figure 5. 7 Import Watt Hall model into project ........................................................................ 139
Figure 5. 8 Mesh Renderer for Watt Hall model ........................................................................ 140
Figure 5. 9 Start Point ................................................................................................................ 141
Figure 5. 10 Unpack prefab ........................................................................................................ 142
Figure 5. 11 Edit Watt Hall model .............................................................................................. 143
Figure 5. 12 Give AR function to model .................................................................................... 144
Figure 5. 13 Building Setting for AR-FM .................................................................................. 145
Figure 5. 14 Building Setting for AR-FM .................................................................................. 145
Figure 5. 15 Build AR-FM.......................................................................................................... 146
Figure 5. 16 Find AR-FM APK file in local folder .................................................................... 146
Figure 5. 17 Send AR-FM file to device..................................................................................... 147
Figure 5. 18 Find AR-FM on device ........................................................................................... 147
Figure 5. 19 Operation scene of AR-FM .................................................................................... 148
Figure 5. 20 Welcome panel for AR-FM .................................................................................... 149
Figure 5. 21 Watt Hall model view in start point ....................................................................... 150
Figure 5. 22 Back to welcome panel to restart model................................................................. 151
Figure 5. 23 Moving component ................................................................................................. 152
Figure 5. 24 Component after moving ........................................................................................ 152
Figure 5. 25 Rotate component ................................................................................................... 153
Figure 5. 26 Components after rotating ...................................................................................... 153
Figure 5. 27 Scaling components ................................................................................................ 153
Figure 5. 28 Component after scaling ......................................................................................... 154
Figure 5. 29 Highlight component ............................................................................................. 154
Figure 5. 30 Direction panel ....................................................................................................... 155
Figure 5. 31 Before hide ............................................................................................................. 156
Figure 5. 32 After hide ................................................................................................................ 156
Figure 5. 33 Before hiding the toolbar ........................................................................................ 157
Figure 5. 34 After hiding the toolbar .......................................................................................... 157
Figure 5. 35 Information of component ...................................................................................... 158
Figure 5. 36 Panel on the wall .................................................................................................... 158
Figure 5. 37 Show inside image on the wall ............................................................................... 159
Figure 5. 38 Click button to choose pipe .................................................................................... 159
Figure 5. 39 Second wall of panel .............................................................................................. 160
xii

Figure 5. 40 Modify whole model .............................................................................................. 160
Figure 5. 41 Modify parts model ................................................................................................ 161
Figure 5. 42 Volunteer using panel on the wall .......................................................................... 161
Figure 5. 43 Volunteer using panel on the wall .......................................................................... 162
Figure 5. 44 Workflow of creating AR-FM ................................................................................ 165
Figure 5. 45 Features of AR-FM................................................................................................. 165

Figure 6. 1 Enire Workflow of Creating AR-FM ....................................................................... 169
Figure 6. 2 Scripts to realize the functions in AR-FM ................................................................ 170
Figure 6. 3 Import UI panel from Fairy GUI to Unity 3D .......................................................... 171
Figure 6. 4 Attach codes in Visual Studio to Unity 3D .............................................................. 171
Figure 6. 5 Import exterior plugins in Unity Assert Store ......................................................... 172
Figure 6. 6 Manage Unity 3D inherent plugins in Package Manager ......................................... 172
Figure 6. 7 Import Videos to AR-FM project ............................................................................. 173
Figure 6. 8 List of features in AR-FM ........................................................................................ 174
Figure 6. 9 Diagram of UI panel for AR-FM.............................................................................. 178
Figure 6. 10 Workflow of FM..................................................................................................... 179
Figure 6. 11 AR-FM help FM managers know the requirement of work ................................... 180
Figure 6. 12 Using AR-FM in case study ................................................................................... 182

xiii

ABSTRACT
Building information modeling (BIM) is critical in the architecture, engineering, and
construction industry, especially for 3D modeling and the addition of geometric and non-
geometric elements. Virtual reality (VR) and, more recently, augmented reality (AR) technology
are being developed from a specialized computer technology software to applications used in
many areas. Based on the contributions and challenges discussed in many research papers in
recent years, it is apparent that the integration of BIM and AR elements for positioning wall
components in existing buildings is feasible and can facilitate the maintenance and repair of
components in the future. A workflow and user interface were developed to track a Revit model
in a Unity workspace for an augmented reality overlay.

The model building chosen was the third floor of School of Architecture office building
in University of Southern California. Revit, 3ds Max, Unity 3D, and other software were used as
experiments to model the elements in the wall based on the AR foundation and matching existing
models with real-world architecture. Different from the tracking location method in other
research, a start location was created to make it easier to match the digital with the real world.
Then, users can click to choose the wall, the floor or the roof and hide it to show the inside
components. And the components can also be moved by the button on device to realize the
human-computer interaction. In addition to AR imaging, a demonstration animation was added
to show the optimal disassembly method of a complex structure group or complex components in
the wall. As a result, AR-FM app realized the transferring of BIM model to Unity 3D and
visualize the MEP system and architecture system. In future, more function will be added to
make the tool more comprehensive and help the work process during facility management.
xiv

Hypothesis: AR can be used with the building information model to locate and view the location
of hidden components of a building.
Keywords: Building Information Modeling (BIM), Facility Management (FM), Augmented
Reality (AR), Animation
Research Objectives:
• Establish a method that takes a building information model to an AR application.
• To achieve human-computer interaction in the new AR app.
• To create a BIM-based AR application called AR-FM for pre-rehearsals of building
maintenance in facility management.

1

Chapter 1
Introduction
In this chapter, these following parts are introduced: Augmented Reality (AR), and the
related technology of AR: Visual Reality (VR) and Mixed Reality, Building Information
Modeling (BIM), the engine Unity 3D and information of Facility Management (FM). Some
concepts are presented in this chapter, including an introduction and methodology to these
techniques and methods. At the same time, some appropriate comparisons and contrasts were
made.
1.1 AR (Augmented Reality)
Augmented reality (AR) is a technology that calculates the position of a camera to
display a digital model on top of the corresponding image, which is usually an image of the real
world. This technology adds virtual models made by people to existing objects in the real world.
The display world and the virtual world are complementary to each other to interact on the
screen. This technology was first proposed in 1990 (Amin, 2015). With the development of
technology and electronic equipment, it has gradually been applied by lots of fields in life, not
limited in the architecture, but in medical area it can be used in the surgery process to show the
patient's physical condition, education and so on from a single computer software technology
(Amin, 2015).

By overlaying digital images onto the real world, the real world is enhanced. In this
visual augmented reality, users use "glasses" (often called head mounted displays) to project the
images and combine the real world with computer graphics to achieve the purpose of augmented
2

reality (Dey, 2018). Augmented reality technology includes multimedia, three-dimensional
modeling, real-time video display and control, multi-sensor fusion, real-time tracking and
registration, scene fusion, and other new technologies (Arth, 2018). Augmented reality provides
information that can be different from what humans can perceive but can be seen through "AR
glasses" under normal circumstances. Augmented reality glasses are the medium that connects
the real world with the artificial virtual world. People can create objects they want to present
through 3D model software and display 3D models on the screen after the system recognizes
them and positions them.

For some method to make people be able to see AR model based on this technology, the
simplest way is by an AR camera. Most cameras on the phone and the front camera on laptops
can show this function. By aligning the model with the tracking image, when the tracking image
is shown in front of the camera, the model will be shown superimposed on it (Figure 1.1). In
other methods, AR glasses can be a good choice. Through the glasses, the model can arrive
directly at the human’s eyes, which will be a more real way to see AR (Figure 1.2).

3

Figure 1. 1 AR model shown by camera

Figure 1. 2 AR glass to show AR model (Retrieved August 16, 2021, from
https://justtotaltech.com/best-augmented-reality-glasses/)
For example, there are many examples of people experiencing AR. When people buy
shoes online, they can't try them on directly like shopping in the store, so it's difficult to know
the actual effect of the shoes. At this time, AR technology can help people solve this problem.
After identifying human’s feet through technical software, AR technology can help people "put
4

on people’s shoes" through their mobile phone camera and mobile phone screen (Figure 1.3). In
this case, people can better see the upper body effect of shoes, instead of trying to wear shoes
after express delivery to home, as in the traditional way. AR technology can help people “see”
the items like that they can check whether the available size of shoes is suitable to them.

Figure 1. 3 New Challenges in Retail and How 3D And AR Help (Retrieved August 16, 2021,
from https://www.wsj.com/articles/ar-sneaker-apps-why-the-future-of-footwear-is-digital-
11580750963)
In 2016, a mobile phone game named Pokemon Go came out. This game adopted AR
technology to show the pokemons inside the game, the users can use the interaction to realize the
human-computer interaction to accomplish some of the operation with the visual world (Figure
1.4). Also, by showing the pokemon with AR technology, people who play this game will feel
that the pokemon will be in their real life.
5

Figure 1. 4 Using AR showing in mobile game Pokemon Go

1.1.1 History of AR
Ivan Sutherland, the father of AR, was a computer scientist who won the Turing Award
in 1988 (Hyman, 2012) Originally, Ivan completed the entire system in 1968. At the time,
however, the concept of augmented reality did not exist in the industry. In 1990, Tom Caudell, a
researcher of Boeing Company, proposed the phrase "augmented reality." In the years that
followed, AR slowly began to show its way in a variety of professional fields (Berryman, 2012).
In 1992, for example, the U.S. Air Force developed a virtual help system and Columbia
University's KARMA Repair Help System. Humans, however, did not bring AR effects outdoors
through their phones. In 2000, Bruce Thomas developed a project called ARQuake to bring AR
to the real world outdoors. ARQuake is modification in Quake, designed for outdoor games
using augmented reality. The user wears a head-mounted display that allows the user to view
both the Quake game and the real world. The ARQuake was modified by the researchers to get
its view information from GPS and orientation sensors, so as users who walk around, Quake
6

moves in sync with the real world. The monsters and buildings seem to be in the real world as if
they were there, and then you can play Quake in the real physical world (Piekarski, 2002).

1.1.2 Application of AR in Architecture
With the rise of augmented reality technology, more industries begin to develop
applications of this technology and apply it to practical work. The same is true in the field of
architecture, where augmented reality could be an asset. For example, in the work of
architecture, AR technology can be used for showing a 3d model on a real site (Figure 1.5).

Figure 1. 5 Kubity showing AR (Retrieved August 16, 2021 from
https://download.cnet.com/Kubity-Go-AR-VR-more-for-SketchUp-Revit/3000-20418_4-
78557431.html)
7

In other words, the digital sand table is another way that AR will be accepted to help
designers and construction companies to discuss their plans by adopting AR. To be specific, in
traditional ways the designers need to discuss their plan about the building just based on the
drawing of it. While adopting AR technology, they can discuss it on a plate by using AR
technology to show the model and analyze it. Augmented reality in architecture and construction
can show the building model before the building is built. It can also show the designers’ idea
about improving the building, which will be helpful to save time and cost (Zlatanova, 2002). In
the architectural design stage, augmented reality technology can make the architectural design
step by step vividly displayed, so that the designer's inspiration can be better expressed in front
of the decision makers, which is helpful to improve the efficiency of product design process
(Figure 1.6)

Figure 1. 6 Using digital sand to show AR model in design (Retrieved August, 2021, from
https://eos.org/science-updates/augmented-reality-turns-a-sandbox-into-a-geoscience-lesson )
At the same time, when architecture is mentioned, people will think of construction,
which has long been an inseparable link with architecture. “The usage of devices and
8

technologies in digital workbench which will be sent directly from the site to the office reducing
the time of effort in delivering reports” (Abdelaziz, 2020).

In the construction planning stage, AR technology can play the role of assistant decision-
making, such as to the decision-making through augmented reality vividly demonstrates the
building is in a few years or even 10 years after completion of construction effect.

Different from the application in other fields, the advantage of AR technology in the
construction process can be said to subvert the whole traditional construction method. The most
representative point is that each construction site will set up a safety experience area and safety
education area.

The purpose of these zones is to provide workers with better construction safety
education and experience. And AR technology can be a good simulation of some of these
contents, inserting in the appropriate position of the appropriate notice, to ensure that the whole
process can run better (Figure 1.7).

Figure 1. 7 Construction VR (Retrieved August 16, 2021, from https://jasoren.com/construction/)
9

Besides these, some components reorganization and detail methodology of
comprehensive structure (Figure 1.8) and the layout and placement of construction site objects
(Figure 1.9) can also be accomplished by using AR technology.

Figure 1. 8 AR technology for comprehensive components (Retrieved August 16, 2021, from
https://www.constructionexec.com/article/virtual )

Figure 1. 9 AR apply in a construction site (Retrieved August 16, 2021, from
https://www.vgis.io/wp-content/uploads/2019/10/vGIS-Construction-BIM-AR-augmented-reality-
HoloLens.jpg)
10

After the building is done, owners and occupants can also use AR. In the interior design
stage of architecture, AR technology can help complete auxiliary measurement, create
professional house plan through AR, estimate workload, and view the internal space from a
three-dimensional perspective (Setti, 2016). The specific application is mainly in furniture
decoration, add furniture and decoration objects, support purchase decisions and determine the
feasibility of creativity, and preview the appearance of the new home. At the same time, for the
seller of the house, users can view the decoration effects of the product in their house before
purchase (Figure 1.10). Also, the finished furniture model can be added into the real world by
using AR technology. For example, in IKEA, in their application, the customers can put the
furniture in their own house which is also the production of AR technology (Figure 1.11).

Figure 1. 10 Using AR to preview the house (Retrieved March 1, 2022, from
https://www.livehome3d.com/useful-articles/ar-in-home-design )

11

Figure 1. 11 IKEA application to preview furniture (Retrieved August 16, 2021, from
https://hip2save.com/tips/ikea-shopping-tips/ )

1.1.3 Application of AR (Augment Reality) Software
AR software can be divided into design and production software and application
software. Other examples include the design and production of software. The main fields are
computer science, media, education, and games (Table 1.1). AR application software is widely
used in the fields of medical treatment, computer, art design, military, furniture and, of course,
architecture (Table 1.2) (Azuma, 1997).

12

Table 1. 1 Application of AR software in sorts of fields
Application of AR Software in sorts of fields
Name of Field Software
Computer Science A-Frame, Apertus AR, ARToolKit,
Openillusionist, Layar SDK, Wikitude
SDK
Broadcast Media Arit AR
Education zSpece for Education
Games Untiy 3D, Unreal Engine, Vuforia, ARkit

Table 1. 2 AR APPs in many fields
AR APPs in sorts of Fields
Computer Science Mission to Mars AR
Broadcast Media GIPHY World, Snapchat, Civilizations
AR
Education Mondly
Games The Machines, Pokemons Go
Construction Houzz
Costume Design YouCam Makeup
Arts Google Lens, SketchAR
Living IKEA Place, Augment, MeasureKit
Business ROAR, Amikasa
Medical AccuVein
Military HUD 3.0

As a result of these developments and applications, augmented reality (AR) provides
people with a new way to interact with the real world. In practical application, it creates a
modified version of the real world on the screen of desktop computer or mobile device and uses
digital (or virtual) information to realize the interaction between the virtual world and the real
world through the screen, so as to enrich the real world (Grubert, 2013).

13

1.1.4 Methods to Show AR Model
Generally, there are two ways to show AR model, they are tracking and putting. On the
one hand, tracking to show AR model is to pre-link the AR model with tracking images in
software such as Unity 3D, and then present the same printed tracking images in the real world.
After the file containing the model and tracking picture is packaged and imported into the device,
the tracking picture can be recognized by the camera of the device, and the AR model will be
displayed in the real world, and the position is consistent with that set in the software (Figure
1.12).

Figure 1. 12 Using floor plan of the building picture to show Whole AR model (Retrieved August
16, 2021 from https://www.upwork.com/services/product/augmented-reality-app-with-vuforia-
and-arcore-1351858704457875456 )

On the other hand, the once the AR model has been established, the model will be based
on a target, the target will automatically find the flat area, and when users touch the screen, the
model will be shown (Figure 1.13).

14

Figure 1. 13 Put AR model (Retrieved August 18, 2021, from
https://medium.com/@kristen.carter/build-your-next-ar-vr-web-app-using-javascript-
32d3252e5756 )
1.2 VR (Virtual Reality) and MR (Mixed Reality)
AR uses computer technology to create virtual models or information and present images
to the real world through cameras and sensors. VR focuses on a user experience solely in a
digital world. Mixed reality blends the physical and digital worlds. These two realities define the
extremes of a spectrum called the virtual continuum. (Figure 1.14) (Figure 1.15) (Aruanno,
2019).

Figure 1. 14 AR, VR and XR (Retrieved August 23, 2021, from
https://medium.com/@northof41/what-really-is-the-difference-between-ar-mr-vr-xr-
35bed1da1a4e )
15

Figure 1. 15 the major milestones in the journey of virtual reality (Retrieved from August 23,
2021, https://ib.cricket/the-story-of-virtual-reality /)
1.2.1 VR (Virtual Reality)
“Virtual reality (VR) is the use of computer modeling and simulation that enables a
person to interact with an artificial three-dimensional (3-D) visual or other sensory environment.
VR applications immerse the user in a computer-generated environment that simulates reality
using interactive devices, which send and receive information and are worn as goggles, headsets,
gloves, or body suits” (Shen, 2021). For example, the current popular VR games make use of this
technology, so that players can more truly experience the fun of the game. The viewing angle
range of the human eye under a fixed viewing angle is usually 120 degrees. Cinema screens are
generally within this range. Beyond this range, people can't see the image. VR realizes all
degrees panoramic 3D immersion, which will give people a completely different experience. The
front all degrees refer to the perspective in the horizontal direction, and the same is true in the
vertical direction, which allows the experimenter to see the whole picture of "heaven" and
"earth". In addition, VR can realize interaction through gesture control or tactile feedback, while
3D technology can only operate in one direction and cannot complete the process of interaction
(Guo, 2019).
16

In the field of architecture, VR technology has also made great contributions. For
example, VR security experience hall realizes dynamic roaming and VR interaction through
computer simulation and VR glasses (https://ib.cricket/the-story-of-virtual-reality/ Cricket,
2019).

For example, for one game named Arizona Sunshine, this is a famous VR game in Steam.
In this game the users need to equip VR device such as the handles, to operate the game
characters. By clicking the button on handle, they can shoot at the target (Figure 1.16).

Figure 1. 16 Using VR device to play game Arizona Sunshine (Retrieved August 23, 2021, from
https://vr.fandom.com/wiki/Arizona_Sunshine )
As for Facebook, they changed their corporate name to Meta which include Facebook,
Instagram and WhatsApp. Their CEO Zuckerberg also mentioned that VR technology will be
adopted into messenger calling to create a marketplace to build a more social environment for the
users (Zuckerberg, 2021). For example, some software like the Enscapein Revit can also combine
17

with VR technology. While connecting VR device with Enscape, the glass will show the user the
site of the model, the handle can be operated to move in this site and users can turn around in the
real world then their view in the visual world will also be changed (Figure 1.17) This decision
could seriously launch VR into the mainstream. One immediate release was Horizon Worlds. It is
a virtual reality world of avatars.

Figure 1. 17 VR device with Enscape (Retrieved August 23, 2021, from
https://enscape3d.com/community/blog/knowledgebase/using-virtual-reality-headset/)
In 2020, Facebook announced Horizon Worlds, a VR social platform that has attracted a
lot of attention compared to existing VR social apps. Horizon Worlds is currently in beta, and is
currently restricted to developers who can explore, build Worlds, develop games, and so on (Figure
1.18).

18

Figure 1. 18 VR in Horizon World (Retrieved August 23, 2021, from
https://www.youtube.com/watch?v=brd0jPjYPU0 )
Experienced personnel understand the experience projected through the virtual on-site
construction environment, conduct realistic demonstration for places prone to safety problems,
feel the whole process of safety accidents in advance, and understand the importance of safety
problems (Wang, 2009).

This makes the safety protection training more targeted and no longer carries out the
safety education of "talking on paper", so that workers pay more attention to their personal safety
during construction and avoid some safety accidents. The experience system covers housing
construction, highway, tunnel, subway, bridge, power plant, coal mine, public safety, and other
industries.

In the system, the experimenters can more realistically feel the real effects of more than
120 kinds of safety accidents, such as falling from height, tower crane transportation injury,
19

support framework collapse, electric shock injury and portal falling, so as to make them awe,
Improve the awareness of safe construction (Li, 2018) (Figure 1.19).

Figure 1. 19 Virtual Reality- The Next Frontier in Safety Training (Retrieved August 23, 2021,
from https://www.bbntimes.com/technology/how-virtual-augmented-reality-are-revolutionizing-
the-mining-industry )

The biggest difference between AR and VR lies in their actual forms. For AR technology,
virtual elements are added to reality, so the user sees both the virtual objects (or sounds) and the
real environment.

For VR technology, all the scenes it shows are virtual, which brings the experimenter into
the virtual world created by VR technology. Different from AR, VR technology is to create a
virtual scene, whereas AR is a combination of real scene and virtual scene, so cameras are
basically needed. Based on the pictures taken by the camera, it is combined with virtual pictures
for display and interaction.
20

1.3 BIM (Building Information Modeling)
BIM is different from the previous software programs like CAD (computer aided design)
in architectural design and structural design. BIM technology establishes a virtual building in the
computer through digital means. The virtual building will provide a single, complete, and logical
building information base and geometric data information. Non-geometric information can also
be displayed (Eastman, 2011).
BIM is both the building information model and the process of building information
modeling. It is an integrated workflow based on coordinated and reliable information about a
project from design, construction to operation. CAD is computer-aided design, which simply
uses computer system to assist design by using vector objects like lines, circles, text whereas
BIM uses components like walls, windows, door, roofs, and furniture (Figure 1.20).

Figure 1. 20 VR in Horizon World (Retrieved August 25, 2021, from
https://knowledge.autodesk.com/support/revit/learn-explore/caas/video/youtube/lesson/143344-
courseId-100332.html )
1.3.1 Development of BIM
The concept of BIM originated from Professor Charles Eastman of Georgia Institute of
Technology in late 1970 (Eastman, 2010). Its development has broadened its vision from BIM as
a 3d model to BIM as a process that includes design, evaluation, construction process, building
21

life cycle, performance, and technology. It is divided into pre-construction stage, construction
stage, and post construction stage. The application of BIM in construction projects began in the
mid-2000s (Reddy, 2011). At this time, BIM has been effectively and efficiently introduced into
the computerized and integrated information management system of construction projects
(Aryani, 2014). Currently, BIM has developed more perfectly and has been widely used in major
building firms for architectural design, construction, and even facility management and
operations. When BIM was developed in 1979, perhaps predecessors had expected that BIM was
the future of the construction industry. This goal has been majorly achieved (Figure 1.21).

Figure 1. 21 Development of BIM (Aryani, 2014)

1.3.2 Application of BIM
The essence of building information model (BIM) application is that these data can run
through the whole life of the project and continue to play a role in the construction and later
operation management of the project (Hardin, 2015). Building information model (BIM) is one
of the most promising new developments in the construction, engineering and construction
(AEC) industry. Through BIM Technology, the accurate building virtual model is digitally
22

constructed (Azhar, 2009). It has five characteristics: visualization, coordination, simulation,
optimization and drawing.

The shortcomings existing in the design stage, such as redundant drawings, high error
rate, frequent changes and difficult cooperation and communication, could be solved by BIM.
The value advantages brought by BIM are far greater than the traditional CAD mode, to realize
one modification and other automatic modifications and improve the design efficiency and
design quality (Zima, 2020). By using Revit (a popular BIM software program by Autodesk), all
the details including clarity and naturalness between components that the quality of digital model
will be enhanced by the supporting of the insert technology. One can also import the model into
UE4 (Unreal Engine), so as to disassemble the house model and record the use of materials and
location (Mirshokraei, 2019).

In Revit, Enscape is a software that supports VR technology. After importing the model
into Enscape, users can experience what the model looks like inside by connecting to a VR
device. Moving and turning in the virtual world can be separated by the controller and moving in
the real world. This is an application of BIM software to VR technology.

The application of AR technology to BIM software includes the following aspects:
project decision-making, communication and teaching, inspection and progress viewing and
underground space image presentation (Park, 2018). With BIM+AR technology, users can
actually see the building in real scale before construction. By loading the BIM model to the
mobile device, AR technology is used to conduct virtual walk around the planning red line to
23

check whether there are any conflicts or changes, so as to avoid mistakes. In this way, each
construction party can see the impact of any changes in real time (Omar, 2016).

1.4 Unity 3D
Unity3D is a powerful cross platform 3D engine and user-friendly development
environment. Unity is used to create 3D games and applications for mobile phones, desktops,
web pages, and consoles (Linowes, 2012). Unity is a cross-platform game development engine
that allows users to build anything from simple mobile apps to complex desktop games and apps
as well as AR/VR games. Unity is used for scripting, scene creation, animation, app architecture
development, level design, motion design, and physics implementation and as for its modeling, it
contains box modeling, NURBS, digital sculpting, procedural modeling, and image-based
modeling. (Theoktisto, 2019). Unity 3D can play a significant role in the field of AR and VR
technology; it is possible to create the visual elements people, AR would like and connect them
into real world.

Unreal Engine is another software program like Unity. The Unreal Engine AR framework
provides a rich and unified framework for building augmented reality applications using Unreal
Engine on IOS and Android handheld platforms. This framework provides a single development
path for the two platforms, allowing developers to use a single code path to build augmented
reality applications for the two platforms. The handheld AR blueprint template provides a
complete sample project that demonstrates the augmented reality capabilities available in the
illusion engine.

24

The AR engines Unity 3D and Unreal Engine are not BIM software programs although
they can take geometry from those programs. In Unity 3D. The most basic way to create AR or
VR application is to use Plugin named AR Foundation. The model or the objects could either be
established inside this software or just imported from other files like fbx.

By creating AR application, Unity 3D can combine some Plugin like Vuforia, the AR
core and so on to create the tracking method or putting method like mentioned in 1.1.4, and after
packaging into the device, in the specific environment the model will be shown (Figure 1.22).

Figure 1. 22 Development of BIM (Retrieved August 28, 2021, from
https://stackoverflow.com/questions/53517667/unity-ar-how-can-i-trigger-a-button-in-the-
scene )
As for the creation in VR, users can find the Source Code (SC) in Github and to structure
the whole site area, and then, some defining of the space and the model will be established. The
whole method is like AR, packaging it and show it in the device (Figure 1.23).

25

Figure 1. 23 Unity 3D for VR (Retrieved August 28, 2021, from
https://www.youtube.com/watch?v=tKN9rSl4pWo)

1.4.1 Connecting Unity 3D with BIM
After modeling the building in BIM software like Revit, one can export it into Unity 3D.
As for the model, an .fbx file format is enough to make the translation; just export it from BIM
and while Unity3D is turning on, click the .fbx file and choose Import.

However, the disadvantage of this method is that all the texture and render material will
be missing; the model in Unity3D will look white and needs to be colored. One of the ways to
keep the texture and the rendering is to use other software like 3ds Max to export texture and
rendering respectively. It will show better of the model’s color.

On the other hand, as for the data of BIM model, Dropbox is a good choice to upload and
download into Excel. Then, using C# scripts to link it with the model in Unity will add the
26

material or other information with the model. Materials are not the only thing lost in the
translation from BIM to a game engine.

The data/information in the BIM model such as the fact that it is a specific type of object
(like a door) that has specific attributes (like cost and fire rating) are lost. Game engines are
mainly concerned with geometry and materials (Figure 1.24).

Figure 1. 24 Import BIM Model into Unity 3D (Retrieved August 28, 2021, from
https://3drepo.com/how-to-import-bim-models-to-unity / )
Building maintenance techniques are used to maintain the performance and livability of
buildings. A building needs a skilled team composed of many people, in the electrical, plumbing
and HVAC as well as cable, water pipes and other areas of different maintenance methods.
Without regular maintenance services, buildings can quickly become an unfriendly environment
to live and work in.

27

1.5.1 Traditional Building Maintenance
In the traditional building maintenance, the overall process is very cumbersome, very
time-consuming and cost ineffective. Due to the lack of testing equipment, some buildings still
need to use the design drawings many years ago to determine the placement position of internal
structures and components after they are completed more than ten years, which will be a very
troublesome process. In terms of performance detection, it is difficult to determine the position
of some components in the wall, sometimes the workers have to dismantle some walls to see the
internal structure (Seeley, 1987). Only one approximate location can be determined, and the
maintenance of the ventilation system will be more difficult. The internal structure of the
ventilation system is cumbersome. It is necessary to test every place and find out the fault area
by eliminating the fault area, which will also take a long time.

FM is a relatively new research field. Early research dealt with showing what is inside of
a wall. Only a few complete FM architectural designs have taken this into account, which has
greatly changed the definition and understanding of FM in this regard. The definition adopted in
2006 is as follows: "integrating processes into an organization to maintain and develop agreed
support and efficiency services, the application of the concept of architectural knowledge
transfer to architectural design is far from new, and the development of information technology
is expected to promote knowledge transfer in the mid-1990s, British Facilities Management
Association (BIFM). By far the largest national FM Association in the world commissioned the
construction of this building in 2000. (BRE) conducted a study to introduce facility expertise into
the design process (Chen, 2021). This led to a report that analyzed the reasons why facility
28

managers participated in the design process and why they were often excluded from the design
process (Figure 1.25) (Göçer., 2001).

Figure 1. 25 Traditional Building Maintenance (Retrieved August 28, 2021, from
https://streetsla.lacity.org/careers )
CAD and then BIM has been used more recently, as a way of supplementing physical
drawings. Sometimes, other types of software are linked with these digital drawings or models:
Computer Maintenance Management System (CMMS), Electronic Document Management
System (EDMS), Computer Assisted Facilities Management (CAFM) and Equipment
Management System (EMS). These four management systems can analyze different elements in
the building, making the building data perfectly integrated into the model (Kensek, 2017). As a
result, architects and engineers can better predict the performance of new buildings and have a
basis for later renovation and maintenance of buildings. As BIM is gradually integrated into the
whole building life cycle, the way of cloud computing for building planning and maintenance
will be more popular (Kensek, 2017).

The advantages for owners and managers are obvious: the space can be managed; the
model can help to fill FM databases quickly, save manual time and money, and manage assets
29

effectively. The model can be used for building performance simulation and debugging (Kensek,
2015). Manipulation simulation tools may be able to use data from the model; and completion
information can be used for renovation later. The model can provide the data required by FM,
which can be useful for maintenance and repair, energy management, and commissioning of
buildings, especially if facilities management concerns are articulated at an early stage of the
design (Kensek, 2015). Other benefits include better visualization of system components, ease of
modification, the ability to filter data for employee use, and the advantage of having a
coordinated system that integrates BIM and FM (Kensek, 2015).

In the traditional process of building facilities maintenance and management, the most
complicated point lies in the direct handover of architectural data between two building groups.
For example, in the process of maintenance, the maintenance team is often different from the
design team, which leads to different positioning and performance of facilities. However, the
popularization of BIM cloud data will make the maintenance and handover process of facilities
better with all parties more easily able to access the cloud data. For example, the sharing of the
model information will be kept online, different from tradition saving of the floor plan drawing.

One important approach is to review the BIM requirements for facility management. The
best way to make BIM data useful during the operational phase of the building life cycle is to
determine in advance what information the facility management system will need (Davila, 2018).
By incorporating facility management into the initial planning phase of the building information
model, gaps can be bridged, and stakeholders can be aligned with common goals (Davila, 2018).
The team may initially incorporate the asset location and unique ID into the BIM. With the
30

correct data structure, more asset details can be easily added to the BIM in the future (Porter,
2018).

1.5.2 BIM+AR in Building Maintenance and Facility Management
For the maintenance mode combining BIM and AR technology, one advantage is to use
digital geometry and overlay it in a real environment (Figure 1.26); one can project the building
information model into an environment for maintenance personnel to follow (Figure 1.26).

Figure 1. 26 BIM+AR facility management(Retrieved August 29, 2021, from
https://www.researchgate.net/publication/291377453_Investigating_human_and_technological_
requirements_for_successful_implementation_of_a_BIM-
based_mobile_augmented_reality_environment_in_facility_management_practices )
An AR system is needed to illustrate the application and integration potential of BIM in
facility management, considering the correlation between different systems and their operation in
the real built environment (Diao, 2019). When BIM + AR technology is adopted, people will
find that the steps are much simpler. For example, zoning the house through this technology will
make the house modular, and the structure can be disassembled at will. At the same time, after
31

augmented reality display, it will be consistent with the real effect of house demolition. At the
same time, AR technology will also clearly display the components inside the house, including
location and size, which saves time for maintenance. For example, the path of the ventilation
system will become clear briefly.

In terms of project operation and maintenance management, BIM + AR technology has
four characteristics (Theoktisto, 2019). First, it can mix display for remote assistance. After
completing the AR procedure, the disassembly steps of each important part can be displayed in
augmented reality, so that the maintenance personnel and residents can see it inside the
component. The second is that visual maintenance guidance can be carried out. Through
augmented reality, virtual operation can be carried out on the displayed image to ensure and
predict the success of the operation, resulting in greatly reducing the error rate of actual
operation. The third is the early warning display of high-risk equipment, which can ensure the
safety in the process of building maintenance. The last point is to quickly locate the equipment
(Figure 1.27). In general, combined with the above advantages, BIM + AR technology has a
great improvement over traditional technologies and methods in building operation and
maintenance. It can well ensure the quality and efficiency of building operation (Wang, 2019).

32

Figure 1. 27 How BIM + AR applying (Wang, 2019)
1.5.3 One Sample Application of BIM+AR in Pipeline Maintenance (Pei-Huang, 2019)
One previous case of complex pipeline maintenance research based on BIM and AR
technology was done by Pei-Huang Diao and Naai-Jung Shih. The maintenance of internal
facilities is not limited to the use of original drawings (that will lead the whole work process to
use more time and less rate of fault tolerance), but the use of AR technology for accurate
positioning and separation of internal pipeline. For different buildings, there are different levels
and layout, which makes it difficult to make targeted maintenance strategies. The relationship
between the original design information and its positioning in real space is often imperfect, and
even the displacement of internal components will occur in the process of building aging.
Therefore, it is difficult to find a perfect method to determine the relative position of the pipe
from the other side of the wall (Diao, 2019).

According to the research, due to the design of mechanical, electrical, and plumbing
(MEP) systems, its complex organization often makes maintenance difficult. This complexity
33

requires not only a full understanding of the expertise of design drawings, but also the expertise
of recording and maintaining various mechanical components and equipment.

The results show that after separating the AR model of the pipeline for inspection and
analysis and targeted maintenance strategy design, finding the location of pipes will become very
accurate in the maintenance of older buildings. At the same time, the accurate design in advance
will also maximize the efficiency and greatly improve the fault tolerance rate. (Figure 1.28) For
example, at first people can add the pipe and other components model in Revit, which need to
make sure that the size and area of the building is in proportion to real-world architecture. Then,
it can be exported into the AR engines, to accomplish augmented reality parts. When package it
into the mobile device like iPhone or iPad, with the tracking parts, the model will be shown on
screen and match the real world. By adopting this technology, once the building is in
maintenance, the workers will find the accurate location of the components according to the
showing on screen, and their work efficiency will be enhanced (Wang, 2014).

Figure 1. 28 System structure and modules of BARMS (Huang, 2019)
34

1.5.4 BIM in Unity with AR
BIM combined with AR can bring new value to the operation and maintenance links of
the project life cycle. In the maintenance link, using AR to directly convey work instructions and
work instructions to AR glasses wearers can improve work accuracy and efficiency. In terms of
operation and maintenance, the work can be highly simplified, and users can view important
information without holding drawings or looking through manuals. Through the combination of
AR glasses and BIM model, people can clearly show the elements and construction hidden in the
house structure, such as the built-in ventilation system and the optical and thermal materials on
the house surface (Meža, 2014). At the same time, in the maintenance or repair stage, the
maintenance results can also be displayed in advance to let users clearly see the appearance of
the house after maintenance. Make the whole process more visual, show the whole process in the
most vivid form, and show the charm of architecture (Meža, 2014).

1.6 Summary
This chapter gives an introduction of AR, VR, and BIM, Unity 3D, and the comparison
of traditional and BIM+AR facility management methodology. Currently, BIM and VR and AR
technology can be better combined to simulate the application of architectural models. This has
been done in the design and construction stages of a building. BIM plus AR is also starting to be
used for building maintenance to increase fault tolerance. It has great potential for visualization
and localization for when building components are “hidden” within walls and floors. This
process can also help reduce the fault tolerance in which an old building needs to be maintenance
by workers who don’t know the location of the inside cables or pipes and so on. The AR model
35

can also be linked to text, drawings, and animations to guide the workers in fixing broken
components and enhancing their efficiency through better visualization.

36

Chapter 2
Introduction
In this chapter, previous research is discussed about BIM and facility management,
augmented reality, BIM and AR, and traditional facility management and BIM with AR enabled
facility management.
2.1 BIM and Facility Management
In the process of building maintenance, the integration of BIM elements can make every
component of the whole building clear and facilitate the handover of projects between the teams
(Kensek, 2014). Data exchange formats need to be agreed by the entire team, leading to
collaboration, and contributing to the Information Delivery Process (Patacas, 2018). With BIM
modelling that integrates real-time data, FM professionals can plan for building systems that
allow preventive maintenance and help to understand the real-time health conditions of the
operations systems. For this, extending BIM modelling through to meet the needs of FM is
essential (Patacas, 2018). At the Schiphol Airport, geospatial information, or architectural
information by itself cannot match the requirement of the design work. Therefore, pre-planning
the site layout and material use through BIM can greatly improve the work efficiency (Figure
2.1).

37

Figure 2. 1 BIM for Facility Management program at GEO (Retrieved September 24, 2021, from
https://www.geospatialworld.net/news/schiphol-to-organize-a-bim-for-facility-management-
program-at-geodesignbim-2018/ )

2.1.1 BIM in Architectural Design
BIM is part of the digital transformation in the architecture, engineering, and construction
(AEC) industry. Three models are proposed to study BIM technology.
The three models were applied in Yeliz’s research and can show the ability of the
designer to complete the task, the organization of a design firm, and the level of designers match
the tasks (Tulubas Gokuc, 2017). The data used was collected through a survey of architectural
design professionals employed by large design companies in the United States. The results show
that the matching between BIM technology and design tasks and organizational capabilities may
affect project cost, time, and quality (Tulubas Gokuc, 2017).

The purpose of the Yeliz team research was to develop a model to investigate the impact
of BIM on designers' perceived performance of design companies (Figure 2.2). In these models,
they assumed that the impact of BIM on the performance of the design company depends on the
38

nature of the design task at hand, the organizational ability of the company, and the ability of
individual users. It was expected to clarify the factors affecting the adoption of BIM by design
companies and the impact of BIM on the performance of design companies (Tulubas Gokuc,
2017).

Figure 2. 2 Proposed model in this research (Tulubas Gokuc, 2017)
Two limitations were shown. First, there are fewer employees who are good at using BIM
than those who are good at using other software, which leads to a small application area of BIM.
Secondly, in the regression analysis, all the elements are not put into the analysis, which leads to
the uncertainty of the experimental results. However, according to the data and result, BIM does
enhance the efficiency of the work especially in the almost 40% beyond the normal budget
change. In the future BIM will be even better suited the field of architecture.
39

2.1.2 Facilities management Based on BIM
BIM based facility management technology is a very important application of BIM in the
field of construction. It provides a process to minimize corrective maintenance and to relate the
cause of equipment failure to relevant maintenance information and required related operations
(Yam, 2001). FM practitioners can use this approach to reduce the lead time required for
corrective maintenance responses in a way that maximizes the integrity of the building
throughout its life cycle and focuses their resources on productive maintenance tasks (Yam,
2001).

Building maintenance and repair is critical to the operation of equipment in any facility,
because this work involves more than one stakeholder, in addition to the design and development
team, and later project maintenance team, etc. At the same time, building maintenance also
relates to the service cycle of the building and the service life of the whole building, and it even
relates to the safety of the building. This therefore requires an integrated information system to
capture and retrieve data related to building equipment (Yam, 2001).

In some facility management applications, a direct digital controller (DDC) reports
failures and failures of building equipment to the FM information system to allow facility
managers to monitor and optimize facility performance (Shalabi,2017). In others, facility
maintenance data is typically stored and managed in a separate FM database, called a
computerized Maintenance Management System (CMMS) (Shalabi, 2017) (Figure 2.3).

40

Figure 2. 3 Process architecture (Shalabi et al, 2017)
Automated data and information sharing between each other is another essential factor, in
addition to FM information systems (such as BEMS and BAS) reporting to the FM team when
they detect anomalies in the building's heat and cold (Shalabi, 2020). After receiving the notice,
the maintenance team searched for maintenance information to help solve the problem. The
disadvantage of this process is that due to the lack of information support, most of the time was
wasted on non-productive tasks, which greatly reduces the efficiency of FM and maintenance
team (Cai, 2015).

In this research, the current facility manager almost completely depended on the FM
information system. However, the function of visualization and interaction is not perfect. This
project is the first one that can deal with the alarms in the retrieving historical data from FM
software. As a result, not only the information that the facility manager needs is collected and
provided, but some other elements such as the guidelines of FM from other resources are also
included.

Other researchers think that the expected benefits of investigating the implementation of
building information modeling include facilities management, maintenance and energy
management (Stowe 2015). BIM is seen as a solution for sharing data between multiple systems,
41

which is one of the outstanding advantages of BIM cloud computing. BIM, with its data
repository capabilities, minimizes lead times for unproductive activities required for
maintenance.

2.1.3 Limitations and Improvements of BIM Technology
The main advantages of BIM are its virtual design capabilities and engineering
management. However, there are still some challenges and aspects that need to be improved in
the increasingly mature BIM at this stage. First, management issues focus on the implementation
and use of BIM. For example, in the current environment, there is no clear consensus among
companies on how to implement or use BIM (Sacks, 2011). This has many restrictions on BIM
based cooperation between companies (Bargstädt, 2015). In addition, little progress has been
made in establishing model BIM contract documents (Sacks, 2012). At the legal level, there is no
relevant law to restrict the work in the BIM field (Sacks, 2011) Another controversial issue
among AEC industry stakeholders (i.e., owners, designers, and construction personnel) is who
should develop and operate building information models and how to allocate development and
operation costs (Bargstädt, 2015).

One can optimize BIM performance and improve the work efficiency of the company or
suppliers using BIM for design, for example, by shortening the learning time of BIM college and
produce the high-quality products produced by software suppliers that are considered reliable by
customers and meet the expectations set in advertising, to avoid the contradiction of interests of
both parties has a certain obstacle to the development of the cooperation process (Zhang, 2015).
42

In addition, processes and policies accepted by all stakeholders must be developed to promote
the use of BIM and manage today's ownership and risk management issues (Bargstädt, 2015).

In addition, BIM still have some aspects such as the management factors including the
lack of senior support and client demand that need to improve in future (Coates, 2010). For some
representative opinions, such as combining with Augmented Reality technology will be
discussed in 2.3, the simplification of operation difficulty, the resource sharing platform, and the
teaching system (Coates, 2010).

2.2 Augmented Reality
Augmented reality technology nowadays is applied in both architecture and construction.
Different from the direct design, it is usually combined with some existing technologies such as
BIM or cloud computing (Wang, 2014). A great application of AR technology would improve
the efficiency of work process; many areas in AEC will be benefit from AR technology (Chi,
2013).

2.2.1 AR in Architecture
In the field of architecture, before or during the design stage, one of the most important
works for the designers is to provide clearer renderings for customers to better understand the
details and information of the project by using visual technology to show the content of the
project in an interactive way (Calori, 2015). In this case, adopting augmented reality to the work
process would be a good choice (Calori, 2015). In the experiment of Miller group, an application
named Cal Poly AR was created. The program used visual technology to overlay AR models into
43

the real world, such as walls, floors, MEP, and furniture. At the same time, the digital model was
embedded into the environment (Calori, 2015) (Figure 2.4).

Figure 2. 4 Using AR to show architecture model (Retrieved September 24, 2021, from
https://coarchitects.com/cal-poly-ar-an-augmented-reality-app/ )
The movement in the real world will not affect the relative viewing angle of the model’s
position. In other words, users can view the model anywhere. The advantage of this is that it can
more clearly show the content of the project to the customer. One can also view the model
through simple operation and carry out more independent and interesting exploration (Calori,
2015). To create the AR model, it started in as a Revit model, went to 3ds Max, then to the AR
app developed in Unity 3D (Figure 2.5).

Figure 2. 5 Design process of the application Cal Poly AR (Retrieved September 24, 2021, from
https://coarchitects.com/cal-poly-ar-an-augmented-reality-app/ )
Visual tours nowadays become more and more popular; by wearing AR glasses or head
mounted dispays, people can not only see the scenery and content in the real world, but also see
44

the information added after visual processing, such as the explanation of the items visited during
the trip or the superimposed model (Figure 2.6). This experience will greatly increase the
interactivity of the whole process and more resources will be experienced by users (Li, 2022).

Figure 2. 6 Visual tour of Forbidden City (Retrieved January 25, 2022, from
https://plos.figshare.com/articles/figure/The_screenshot_of_the_virtual_tour_interface_/179439
81 )
For example, in Li’s group experience of Virtual Tourism in the Forbidden City, the
application they designed allows visitors to display the introduction information of cultural relics
by clicking on cultural relics, so that visitors can have better interaction and experience in the
whole process at the same time, during the whole process, visitors can have a deeper
understanding of the Forbidden City. However, there are still some deficiencies. For example,
the authenticity needs to be strengthened and the whole exhibition can be more interactive
through rich links in the whole exhibition process (Quincas, 2022).

45

2.2.2 AR in Construction Management
For use in field construction management, AR can load virtual construction content over
a real construction site, and the field personnel can extract data with high requirements for
professional activities from the assembly (Li, 2018). This can reduce the huge loss caused by
mistake of construction organization, drawing and information transmission in site construction
management, and reduce the time for construction personnel to read drawings and find the
problem (Li, 2018). This can effectively improve the work efficiency of workers, assist
construction personnel to manage, and strengthen the training of on-site construction personnel
(Li, 2018). For this technology, it can even load obvious identification information highlighting
important nodes, then the workers or the managers can check it to find more detail about the real
situation of the site which is conducive to on-site security management (Xiang, 2014). AR is
applied to site layout planning by presetting building materials and equipment in a virtual
environment, planning equipment and corresponding access, and loading it into the site. This
method can complete site layout quickly and accurately (Jiao et al., 2013) (Li, 2018). The
combination of BIM (building information modeling) and BSNS (business social networking
services) can be further extended to the "cloud" (Jiao et al., 2013) (Figure 2.7).

46

Figure 2. 7 Major components of the proposed Social BIMCloud framework, (Jiao et al., 2013)
In their experiment, they proposed a cloud-based BIM server framework, Social
BIMCloud, which facilitates BIM information exchange through dynamic merging and splitting
of building models. Designed to solve the problems of low data transmission speed and
inconsistent data in distributed environments, and applied throughout the data upload process,
data interoperability is promoted through open BIM standards (such as IFC) and direct
integration with construction software. a cloud-based NoSQL database (Jiao et al., 2013).

2.2.3 AR for underground infrastructure (Sánchez, 2015)
Sánchez and his team used an AR model to show the positioning and deployment of
underground infrastructure to reduce unnecessary damage.

The research addressed spatial interaction and visualization techniques suitable for
mobile AR applications as well as new mobile device designs. Their approach automatically
focuses on the user's location, while a key aspect of creating a geographic model is capturing
47

geospatial data from a variety of sources. Most existing geospatial databases are already used
efficiently in other environments such as cadastral surveys or utility asset management.

Augmented reality could benefit in large part from highly accurate to up-to-date data,
which is crucial for the owners of those apps. Other possible data sources include servers for
virtual Earth browsers, such as Microsoft Virtual Earth or Google Earth. As a result, the
application of this technology provides a great deal of information. Overall, interactive
visualizations seem appropriate for end users; both field staff and management claim that the
proposed method provides an effective and very useful method for outdoor inspection tasks,
potentially saving time and money.

2.2.4 Combine AR with other new technologies in the field of construction
Another application of AR technology in the construction field is the training of
construction site staff. The advantage of this application is that it can better ensure the safety of
workers and the fault tolerance allowed in the construction process.

Construction equipment is very dangerous. Improper operation will not only damage or
scrap the machine, but even threaten the life safety of workers. Therefore, the operator training
program is a very important part in the field of facility management, and with the increasing
demand of construction companies for construction projects, training has become a more
important factor to save time and money (Wang, 2017).

48

Another technology that needs to be mentioned is named Fologram, which can enable the
users to design, create AR elements, and share it. To be specific, like Unity 3D, it also has its
high API to their users which mean people who use this software will have high autonomy to
create the work they like (Coogan, 2018). A representative example is that it can relate to Rhino
and Grasshopper. Fologram for Rhino uses Rhino's default meshing pipeline (Jahn, 2018).
Fologram also tracks spatial information, such as the location of devices or hands, and transmits
this information back to the device in real time. By using this spatial information to trigger the
change of parameter model, designers can use Fologram to build customized interactive mixed
reality experience in grasshopper (Jahn, 2018).

In a research project that combined Fologram and All Brick, the purpose was to improve
the accuracy, reliability, and control of part of the construction of Royal Hobart Hospital. By
using AR technology, workers cannot use drawings during construction, because the whole AR
model has been imported into glasses (Aarni, 2020). In this case, hundreds of meters of complex
arc benches have been successfully built (Figure 2.8). At the same time, bricklaying workers at
the work site are constantly creating new technologies and applications of augmented reality,
such as AR holographic technology related to cutting ceramic tiles (Aarni, 2020) .

49

Figure 2. 8 Workers using AR technology to create curved benches (Retrieved September 24
2021, from https://aec-business.com/fologram-and-all-brick-introduce-an-entirely-new-way-of-
building-in-ar/)
Through this experiment, some conclusions can be drawn. For example, the application
of this technology will give architects more opportunities to participate in the process involved
(Aarni, 2020) .

As shown by the bricklaying example in Fologram, the introduction of AR technology
could be a good choice for instruction. Augmented reality training is different from the actual
traditional teaching methods. It can be used to explain concepts through simulated scenes and
virtual reality interaction and provide immersive experience for learners to train in a real
environment. Compared with real exercises, AR can significantly reduce costs and risks and
provide unlimited training conditions / scenarios, especially for equipment with high risk or
difficult operation. The use of real exercises can be costly and risky (Wang, 2017).

If the material handler accidentally drops the material when carrying it to the scaffold
platform, it will have a very dangerous impact on the goods, equipment, and personal safety.
50

First trying it with AR may lead to insights such as paths between places that might be less
dangerous. In addition, the application of augmented reality technology can reduce or eliminate
the results of environmental problems, such as noise, dust, and so on. At the same time, due to
the uniqueness of AR technology, the advanced level of training can be improved. In fact, it is
envisaged that augmented reality can enable some traditional forms of training to be replaced, or
at least greatly reduce the need for training as a separate process (Wang, 2017). The details
include the following two points: first, the operating procedures include pre-operation inspection,
equipment safety inspection and equipment fault inspection. Second, features and objects
without fixed configuration can be modeled and displayed by AR system. The route of cargo
handling can be simulated (Wang, 2017).

To solve the above training problems, Wang's team designed arts (real - World training
system, AR - based real - World Training System), so that the construction equipment operator
trainees can use the actual heavy construction equipment to practice the operation on the actual
construction site. Its main goal is to design and prototype wearable AR system, enable novices to
use heavy equipment for construction operations with the support of digital information, and
explore how AR tools can effectively support training compared with other training methods.
(Figure 2.9).

51

Figure 2. 9 Applications of Augmented Reality Integrations for Construction and Design (Wang,
2017)
2.3 BIM and AR
BIM is a significant resource that can be of benefit in FM; for example, a cloud shared
databased can be set up for sharing information about preventative maintenance and retrofit
(Kensek, 2015). BIM can be used during the whole building life cycle, besides the design and
management (Honic, 2019).

It is necessary to evaluate the effectiveness and availability of AR applications in order to
make owners or project participants accept and use AR technology. Among them, effectiveness
refers to the effectiveness measurement that AR can provide to improve engineering activities in
terms of improving production efficiency, shortening working hours, reducing operational errors.
The usability of AR refers to the ease and comfort of using AR for communication and decision-
making (Wang, 2017).

There are many applications for the combination of augmented reality technology and
BIM technology (Jiao et al, 2013). In other words, the combined technology can be used at all
stages of architectural design, from early architectural design to late maintenance of architectural
52

facilities (Eastman, 2011). Designed to reduce the incidence of omissions and omissions, BIM-
AR integration improves efficiency by facilitating quality management during implementation
(Mirshokraei, 2019).

2.3.1 BIM + AR in Building Maintenance and FM
The application of BIM and AR technology in construction facility management has
become more and more popular in recent years (Gheisari, 2016). This comprehensive technology
has been widely used not only in the construction industry, but also in automobile repair and
other industries (Gheisari, 2016). For example, based on AR model and 3D model, the
positioning and repair of vehicle internals can be carried out after the camera tracks and
identifies the vehicle (Diao, 2019). Because the building space has different levels and layouts,
and combines different parts, intensive design work is needed to optimize the use of the limited
space available to the building system and its users. In contrast, troubleshooting site conditions
using original design drawings becomes very challenging, especially when specific maintenance
work requires additional effort or experience. For these reasons, image-based AR is selected to
meet the needs of moving between indoor and outdoor (Diao, 2019).

In Wang’s research, by connecting BIM with AR technology, more information around
buildings will be superimposed into the real world through cameras. The result is to plan for the
whole construction process and show more detailed information (Figure 2.10).

53

Figure 2. 10 Using AR to show site information (Wang, 2014)
At the same time, the AR model of the whole building can be superimposed on the real
world, so that workers and designers can more clearly carry out simulation field research (Figure
2.11).

Figure 2. 11 Add AR model in real site (Wang, 2014)
In the remanufacturing process, ending the downtime of equipment in the shortest time is
the best way to ensure productivity, and AR technology is one of the elements that can help
optimize this problem (Posada, 2015). The premise of the use of this technology is some
understanding of user needs, which is a very important basis for the reference of manufacturing
process. Factors including, but not limited to ergonomics, communication, situational awareness,
intelligence source and others will be used as the source to determine which user demand factor
54

(Posada, 2015). In the program created by the experiment, the information of many components
will be displayed. The use of artificial intelligence and computer vision technology greatly
promotes the operator's system support in the tracking process, scene registration, object
recognition and segmentation (Figure 2.12). This is due to the use of visual devices to support
various maintenance operations (Posada, 2015).

Figure 2. 12 Using AR technology to help workers recognize a component (Posada, 2015)
At the same time, cross tabulation is drawn according to these factors and the actual
demand trend. It is concluded that by integrating 5g and 6G technologies into AR applications
through wireless networks, the network can be upgraded to support these technologies, to
improve the speed of corrective and preventive maintenance in these two processes, and
minimize the communication between people with machine operation, so as to improve the
maintenance efficiency. However, there are still some deficiencies in the experiment. For
example, when all workers wear AR devices, security will be a problem.

At the same time, more factors should be considered according to the different conditions
of each user to make the efficiency of practical application higher.
55

In addition to facility management, AR and BIM are also integrated with construction
management through conceptual system framework to reduce the occurrence of construction
defects. Augmented reality is considered as a method to extract information from BIM model to
improve the efficiency and efficiency of employees performing tasks (Shirowzhan, 2020).

A low-cost mobile combination AR and BIM tool is developed to access information
facilities (Shirowzhan, 2020). AR technology can promote the process of mapping architectural
documents to 3D real-world entities, which provides great potential for the integration of AR and
BIM (Shirowzhan, 2020).

Position based AR sensors may not provide sufficient accuracy. AR studies have made
significant progress but aligning the AR model with reality (marking and tagging) is sometimes
difficult to do. Setting tags in the work environment can be time-consuming. The viability of tag-
based AR applications may be limited. Therefore, unmarked AR is developed using SLAM
technology to solve this problem (Gerstweiler, 2016). SLAM technology can provide continuous
three-dimensional position, and the positioning accuracy of the mobile device is cm.

This allows it to be used for localization and 3D enhancement purposes, such as
navigation tasks or location-based information visualization. In addition, a few AR systems use
3D animation functions to show the method detail of the work such as the steps sequence or the
information of building material, which have proved to be very useful in assembly tasks. Its main
purpose is to visualize and edit mapping data for interactive AR content and place 3D objects.
56

By loading the metadata of the mapping process and loading the visualization framework
containing slam point cloud and 2D image markers in global coordinates (Gerstweiler, 2016). At
the same time, in Gerstweiler's experiment, a path planning algorithm is integrated to calculate
the path between two points. For AR navigation, the arrow is placed along the path in the scene
(Figure 2.13). The path will be displayed to the user in the camera. Since these are overlapping
information, it is also necessary to deal with occlusion to present a real and understandable
visualization of the image path (Gerstweiler, 2016).

Figure 2. 13 Content viewer for the navigation (Gerstweiler, 2016)
2.3.2 AR and BIM in Architecture

AR will have a profound impact on the art and engineering of architecture (Chi, 2013).
The research showed the application of AR technology in AEC and FM, by connection with a
simple user interface and cloud computing, to make it possible to set up AR equipment in
comprehensive environment.

57

Another major advantage of AR is the use of building information modeling (BIM) in
conjunction with maintenance and building assets. Since BIM is digital 3D, building owners and
managers can basically see-through walls before and after construction. Combined with sensors,
information from BIM can let future building managers know when to replace building assets,
where to find them in the ceiling and walls of the building, and even order the necessary parts
before starting work (Figure 2.14).

Figure 2. 14 Using AR technology show the pipe system inside the wall (Retrieved September 24
2021, from https://www.archdaily.com/914501/9-augmented-reality-technologies-for-
architecture-and-construction )
2.3.3 BIM and AR in Whole Building Life Cycle

BIM cam be used with the information content existing in building information is
brought into building information, and then augmented reality (AR) technology is used to
transfer BIM Models to mobile users and build life cycle applications. Augmented reality can be
used to gain practical benefits for building maintenance work, to obtain more building
information, including the diagnosis of building facilities failure and route guidance inside the
58

building. Overall, BIM and AR technology can help the management of building life cycle, such
as the maintenance of the building, and more functions need to be added to improve the
experiment (Figure 2.15) (Kuula, 2012).

Figure 2. 15 BIM+AR work in whole life cycle of a building (Kuula, 2012)
2.4 Traditional Facility Management and BIM with AR Enables Facility Management
For the traditional building facilities management technology, in the early stage, the
information collection and location query of facilities were completely dependent on drawings
(Azhar, 2011). Subsequently, 3D technologies such as BIM are gradually introduced to make the
traditional facility management technology more visual and accurate (Azhar, 2011).

Augmented reality technology was combined with a 3D model, and interacts with the real
world, making the whole management process clearer (Kim, 2014). At the same time, relying on
AR modeling, one can accurately locate the facilities and construction, so as to facilitate the
handover between the future construction team, the project team and the later maintenance team.
59

It can be said that AR technology has promoted the facility management to a higher level and
reformed the traditional maintenance mode (Azhar, 2011).

2.4.1 Traditional Facility Management Method
Previously, one of the problems in the construction industry was limited experience in the
use, maintenance, and operation of existing buildings (Gerber, 2012). Early research in this field
includes FM aspects that should be considered in different stages of design (Gerber, 2012).

The understanding of facility management has developed to the importance of efficient
facilities for improving the quality of life and enterprise productivity (Xu, 2019). Advanced
technology promotes the facility from "entity" to "intelligent biology" (Xu, 2019). So far,
facilities have become more anthropomorphic and fuller of human-like cognitive abilities, such
as being able to perceive, learn and act. However, the development of "cognitive FM" is still in
its infancy. With the help of traditional technology and BIM, compared with relying solely on
drawings, the current technology has been relatively perfect, which can become more accurate
through the blessing of computer software, and can also accurately detect the use status of
facilities (Xu, 2019).

2.4.2 Future of BIM and AR in FM

Semi-structured interviews with a panel of industry experts about augmented reality (AR)
and building information modeling (BIM) were conducted. 43 companies in the UK construction
industry were sampled (Danker, 2014). The results were analyzed by topic combined with data
collected from literature review. A nine-item industry and practitioner questionnaire were
60

developed on five core themes (Danker, 2014). Nine main application areas were identified
(Danker, 2014):

1. Conflict detection.
2. On-site service visualization such as the reparation or the information of production and
materials.
3. On-site health and safety.
4. Visualization of site construction sequence.
5. Help visualizes inside components
6. Improved communication between architects, contractors, and subcontractors. Specify
and communicate details to accurate specifications to ensure delivery of high quality,
7. Facility management - location of components
8. Visualizing big data (wireless) For example, show the cloud data by
visualize interactively including BIM data
9. Visualization of BIM on-site to realize FM.

The utilization of AR and BIM continues to grow in the construction industry. It is still a
new technology that has a lot of potential to change architecture and construction products and
processes. AR and BIM use are combined with 3D scanning, and wireless sensor networks have
added synergies with 3D scanning. BIM and these emerging technologies will improve overall
efficiency in design, delivery, maintenance and project breakdown. AR technologies are
considered the most valuable industry today during the emerging technologies (Danker, 2014).
Practitioners identify the ability of architects, contractors, and contractors to improve
61

communication and incorporate visualization. BIM allows for better testing and improved design
prototypes in field environments.

According to Google Trends, the search term "BIM" increased steadily from 2004 to
2014, from 3 to 21 percent of Google's assessment of its popular goat-rating. The term CAD
(Computer-aided design), which is synonymous with traditional computer-aided 2D drawing in
the industry, reduced BIM- volume2-ECaADE32 from 93% to 45%. This reflects the traditional
building industry away from the traditional building information modeling of computer aided
design. Augmented Reality is the term searched more than Visual Reality compared to
Augmented Reality from 2004 to 2014 (Danker, 2014). In addition, interest is sure to increase
with the release of Google Glass in 2014. Google Glass is a removable device that combines the
visual representation of digital information with the user's view of the real world to enhance
media. The absorption of new possibilities and potentially disruptive technologies will further
heighten interest in enhancing practical applications (Danker, 2014).

As for AR technology combined with BIM technology, BIM and AR based experts agree
that the adoption and progress of BIM is sustainable. BIM was recognized as a step toward high-
quality, customized computer-aided design solutions, and the adoption of BIM and augmented
reality was recognized by all team members (Danker, 2014). It provides efficiency, as well as
industry efficiency in any new technology, and it measures its cost effectiveness. While
accessibility as a human resource technology is expected to improve its cost effectiveness in use,
this will lead to increased adoption (Danker, 2014). In the future, the construction industry, the
use of AR and BIM will continue to grow, and new technology will also be emerging. BIM and
62

the new technology will improve the design, project delivery, to make an effective work process
(Danker, 2014).

In the future, the construction industry, the use of AR and BIM will continue to grow,
and new technology will also be emerging. BIM and the new technology will improve the
design, project delivery, to make an effective work process and the result of this study provides
more information about the impact and possibility, And the impact of human resources and BIM
on the construction industry (Danker, 2014).

BIM now has certain limitations, including the limitations of capturing and echoing the
real world. Problems such as AR can be mitigated by using embedded technology. AR
recognizes how the integration of AR and BIM technology can solve the problem of insufficient
information separation of physical information and virtual information in current on-site BIM
engineering (Chi, 2012).

2.5 Summary

In this chapter, previous research was discussed about BIM and facility management,
augmented reality, BIM and AR, and traditional facility management and BIM with AR enabled
facility management. In previous research cases, it can be seen that AR technology has been used
in architectural field, including architectural design, facility management and the education of
architecture knowledge, but that there are still some defects in the combination of AR technology
and BIM technology especially with the difficulty of tracking. The combination of BIM and AR
63

technology still needs further development. There are several issues to be solved, especially with
improving the workflow to create an AR model, aligning the AR model with the real world, and
improving the user interface of the AR system for use by facility managers.

64

Chapter 3: Methodology
Introduction
To realize the development of potential efficiencies for a facility management work process,
an AR tool named AR-FM was created. In this tool, a building information model can be edited in
a Unity 3D type of app, where viewing and some manipulation can occur in an augmented reality
environment. A test case for the third floor of Watt Hall in Los Angeles, California, will be used
to explain the methodology. First the model of Watt Hall was created (with a simple set of pipes
and ducts) and exported to 3ds Max where materials were added. Then the model was viewed in
Unity 3D and given an AR framework. Then AR-FM can be used to view the 3d model in the real
world. Revit was for creating the building model; 3ds Max was used for materials, textures; and
in Unity 3D, the model was visualized in the real world. In AR-FM, the users can use the user
interface to control and apply several functions.

3.1 Workflow and Methodology
There are four steps in the workflow (Figure 3.1):
1. Exporting BIM model from Revit to 3ds Max. Watt Hall third floor will be used for the
example to explain the methodology more clearly.
2. In 3ds Max, change the standard of materials, textures and render of the model. If an
animation (for example of dismantling and installation) is needed, create the animation in
3ds Max.
3. In Unity 3D, using AR Foundation to set up the primary location to help align the AR
model (from the 3ds Max model) with real world.
65

4. A user interface system will be established to help users use the tool’s functions and give
them the guidance.

Figure 3. 1 Methodology
3.2 Revit Part
In the first section, the task was completed in Revit (Figure 3.2). First, the Revit model of
Watt Hall, which was provided by USC facility department, was loaded into Revit as the case-
study model of AR-FM application. Some mechanical elements were added to the model of Watt
Hall. Then, some modifications were made to the model, and it was exported to Gamma AR,
and some corrections were made. For example, due to problems such as later reconstruction and
model format, the size of many components deviated from the real-world model.

Therefore, in this section, the software named Gamma AR was used to check whether the
model was aligned with the real world. At the same time, after Revit was exported as an IFC file,
66

it could be directly uploaded to Gamma cloud and downloaded to the device for AR model
loading. When the model could be matched with real-world components, the model could be
exported for reference. Finally, the model was exported to FBX form to facilitate the next
modification in 3ds Max.

Figure 3. 2 Revit part diagram
67

3.2.1 Load and Modify Watt Hall Model
First, the entire model of Watt Hall was opened and displayed in Revit (Figure 3.3).
However, only the third floor is important for this case study, so other floors and sites were
hidden or deleted such as the outdoor elements and some interior components. (Figure 3.4).

Figure 3. 3 Whole Model of Watt Hall

Figure 3. 4 Third floor of Watt Hall
68

There are two reasons for this work. First, AR-FM application is used for facility
management of MEP in this case study. Therefore, furniture, stairs, etc. can be deleted. Another
reason is that the model file is very large. It has been tested in the AR program created by the
first complete model. The file was too large to manipulate. Finally, in Unity 3D, due to the
integrity of the model, a single component could not be deleted and could only be hidden. In this
case, it would also take more memory. Therefore, only useful components should be retained in
Revit. Useful components were defined as components that are helpful for facility management,
such as walls, floors, roofs or various components of MEP system. At the same time, since AR-
FM application loads the model by matching virtual and reality positioning, the office around
Watt Hall would also be reserved to help further positioning. After integration, the model would
include walls, floors, some columns, pipes, cables and other elements (Figure 3.5).

Figure 3. 5 Add pipes to Watt Hall mode
3.2.2 Using Gamma AR to Check the Model
At the same time, considering the ultimate purpose of AR-FM application, its main goal
is FM for MEP. So, another important step was to add other elements such as ventilation system,
piping system, to this case study. For example, on the third floor of Watt Hall, there was a circle
69

of pipes in the real world, so the corresponding positions in the 3D model were further added. As
mentioned earlier, in order to better align models and objects between the virtual world and the
real world, Gamma AR software is used for subsequent calibration and selected because of its
convenience. Users only need to export the Revit model to IFC file and uploaded it to the cloud
of Gamma AR website (Figure 3.6), and they could download it on a mobile phone and use the
method of straight-line positioning (Figure 3.7). However, if an error is found after the final
Unity 3D package file is modified, the components need to be added, deleted or changed from
the Revit model at the beginning. The whole process would have to be repeated which is time-
consuming and greatly reduces the overall efficiency of the process (From Youtube, Moreira,
2022 https://www.youtube.com/watch?v=miZpOfxZrsc).

Figure 3. 6 Upload IFC file in Gamma AR on the website

70

Figure 3. 7 Using Gamma AR to locate the model
3.2.3 Export Model to. FBX
After going through the operation steps of deleting, modifying, adding and checking the
actual AR application of the building model, the operations that could be carried out in Revit
were completed. Therefore, the next step was to export it. In practice, Revit exports FBX files
can be read directly by Unity 3D and added at the same time. In Unity 3D, the FBX file could be
imported into the assert where the model elements like game objects, scripts, the images and
other projects file can be developed. FBX is one of the formats that can be supported by Unity
3D that is also exportable from Revit. However, only a part of the model information (the
geometry) can be exported from Revit; it does not include the textures (Figure 3.8). The entire
model would be white, which will affect the visual effect of the actual AR image. For example, a
white model would made it difficult for users to distinguish the direct boundary between the wall
71

and the ground, leading to misjudgment of the model content (Figure 3.9). After the Revit model
is set up, it was exported in the form of FBX file and imported into 3ds Max for further
modification to fix this problem.

Figure 3. 8 Model that directly import from Revit to Unity 3D

Figure 3. 9 Model after modification in 3ds Max then import into Unity 3D
3.3 3ds Max Part
After completing the above steps of building the model in Revit, the model was exported
to FBX format and imported into 3ds Max. The main purpose is to further optimize the Revit
model to make it more compatible with Unity 3D, such as saving model colors and rendering, to
keep the render of the components in Unity 3D could be seen as similar as those in Revit and to
make it controllable in the later stage, like resetting the pivot point in 3ds Max could make it
more controllable in Unity 3D. AR-FM needs the pivot point set to each individual object
because the pivot point is the items that control the components. The original pivot point is too
72

far away from the component; resetting in 3ds Max will make it back to the center of the
component. At the same time, because the model directly imported by Revit is not automatically
placed to the geometric center of components in terms of pivot points when entering Unity 3D
later, the dislocation of pivot points might affect the coincidence between the whole model
components, resulting in operation difficulties. At the same time, in AR-FM applications, since
3D coordinate axes are to be added as an operation tool for moving, rotating and scaling
components, the reset of pivot points became very important in 3ds Max. Pivot points for the
components were fixed for the building components in 3ds Max (Meža, 2014).

The last step in 3ds Max was to create an animation, in this case, the disassembly and
installation of a piping component for use in the AR model (Figure 3.10).

Figure 3. 10 Diagram of 3ds Max part

73

3.3.1 Import Revit file into 3ds Max
After the FBX file was exported from Revit, it was read directly by 3ds Max (Figure
3.11). After importing it, Autodesk Media & Entertainment was selected (Figure 3.12). After a
successful import, the model was divided according to each component on the left side of the
screen. The name of the component could select, and the corresponding part would be selected in
the model (Figure 3.13).

Figure 3. 11 Import FBX file

74

Figure 3. 12 Set up import option

Figure 3. 13 Successfully imported model in 3ds Max
3.3.2 Change Materials to Standard
As mentioned earlier, importing Revit files directly into Unity 3D made the entire model
lose its materials and textures; and the result would show a single white, which will have a bad
75

impact on the aesthetics of later created applications. One of the solutions is that the software 3ds
Max can be used to change the material information of the FBX file from Revit and then, the
FBX exported from 3ds Max will be able to keep the texture of the model (Figure 3.14).

Figure 3. 14 Change the material to standard
Therefore, 3ds Max will be a very important software to further process and modify the
original Revit model. The process was Revit to 3ds Max to Unity.

3.3.3 Check Render Diagram to Confirm the Materials Convert Successfully
After changing the material configuration, importing the model into Unity 3D again
maintained the same rendering level as in Revit. It is also very easy to check whether the change
was successful by selecting the material option in the render option, and then selecting get all
scene metal to find the conversion of materials (Figure 3.15).

76

、
Figure 3. 15 Check the transform of building material
3.3.4 Reset the Pivot Point
After solving the problems about materials and materials, another important process in
3ds Max was to reset the pivot point of components in the model to its own center, which is to
ensure that in the future AR function, when the UI system gives a single component the ability to
rotate, it could rotate about its own local axis. For the FBX format file directly exported in Revit,
due to format problems, the pivot point could not be automatically placed at the geometric center
of the component, so it is necessary to reset the pivot point further. The pivot point could be used
for the control of component in application AR-FM, so if it won’t be reset, the coordinate would
be shown outside of component (Figure 16). In 3ds Max, click it once, under the “Hierarchy”,
chose “Affect pivot only”, and “Center to Object”, the pivot point would back to the center of the
component (Figure 17). And then, all components were selected. In 3ds Max, select the hierarchy
on the right and select center to object (Figure 3.18). The pivot point of each component will be
reset to its own geometric center (Figure 3.19).

77

Figure 3. 16 Before resetting pivot point for one component

Figure 3. 17 After resetting pivot point for one component

Figure 3. 18 Before resetting whole pivot point

78

Figure 3. 19 After resetting whole pivot point

3.3.5 Create Animation
As described in 3.3, the function of the application is to assist the process of facility
management. One of its functions is to demonstrate the disassembly and installation steps of some
complex structural groups in the form of animation. 3ds Max is a good platform for FBX format
animation. Therefore, the next step was to create complex construction groups (Figure 3.20).
Below the 3ds Max interface was the area for creating animation, and the left and right time axes
represent the number of frames (Figure 3.21).

Figure 3. 20 Animation creation area in 3ds Max

79

Figure 3. 21 Set up of animation creation
After selecting the number of frames, clicked start to start the animation track within the
number of frames, and then this part of the model would become a dynamic model. Firstly, the
component group was chosen, six pipes inside the wall were selected (Figure 3.22) and then the
position of 100 frames was selected the six pipes were moved from the wall into outside and
separated into six individual parts (Figure 3.23), so that the whole borrowing process will be
displayed and the current FBX file will be saved. Another type of animation is about installing or
uninstalling the pipe to replace a new item, there are seven steps it was shown by frames (Figure
24-Figure 30). Then, select the whole disassembled structure group for the same operation, and
restore the structure group after selecting the frame interval, so that the installation process was
animated. Save the FBX file again, and the animation of the disassembly and installation process
of a complex structure group is completed.
80

Figure 3. 22 Move six pipes outside the wall

Figure 3. 23 Separate six pipes to single model

81

Figure 3. 24 Step1(Uninstall)-Remove the valve

Figure 3. 25 Step2 (Uninstall)-Move pipe 1

82

Figure 3. 26 Step3 (Uninstall)-Remove the target item

Figure 3. 27 Step4 (Uninstall)-Replace a new item

83

Figure 3. 28 Step5 (Install)-Move the pipe back

Figure 3. 29 Step6 (Install)-Put the valve above water line

84

Figure 3. 30 Step7 (Install)-Tighten the valve on the pipe
3.3.6 Save the Texture in Local Disk
In 3.3.3, the problem of material conversion to maintain the integrity of the appearance of
the model after importing Unity 3D is mentioned. This is only for the model with certain color
but no rendering effect in the basic material (Figure 3.31). The components of some models have
external rendering, so they were saved and imported again. Select asset tracking in the reference,
save the displayed file and re-import it into Unity 3D (Meža, 2014).

85

Figure 3. 31 Save rendering file

After these steps, a new FBX file was exported from 3ds Max, the work in 3ds Max is all
finished. Next step was to import it into Unity 3D to achieve the interaction between human and
computer.
3.4 Unity 3D Part
In this section, the loading process of the whole model in Unity 3D will be described,
including importing the files processed in 3ds Max, packaging the APK format of the file, how to
give the function to the AR-FM application through C# and other computer languages, and
downloading and installing it on the Android device. It includes the import of AR related plug-
ins, further modification of models, improvement of model information, and how to build the
components of AR world by establishing the initial position, to match augmented reality and
reality from the same position. At the same time, a user interface will be established, including
multiple functions including a welcome panel, and guidance will be added to assist users to
complete the whole process of facility management. Finally, after being packaged up to an
86

Android APK file, linked with the Android device and downloading the APK file, then AR-FM
can be used on the Android phone. (Figure 3.32).

Figure 3. 32 Diagram of Unity 3D work process

3.4.1 Import and Modification of FBX file from 3dx Max
The reason for choosing Unity 3D is that the software has a good API (application
programming interface), which allows users to develop more independently through functions
such as code import and create many personalized works. Opened the “Package Manager”,
download the plug-in named ARFoundation, ARCore XR plug-in, and XR management of visual
studio code editor (Figure 3.33). This is the AR function used to create the building model. The
Visual Studio code editor links C + + with Unity 3D to ensure that more computer language-
based features can be added. First, a Unity 3D project was created. In the AR-FM project, Unity
2019 was used Version 3.9 serves as a platform for creative applications. The plug-ins were
installed including ARFoundation, ARCore XR plugin, and XR Management into “assert”.
87

Figure 3. 33 Plugins in Unity 3D
3.4.1.1 AR Foundation (Wang, 2019)
The AR Foundation is a collection of ARkit XR plug-in (COM. Unity. XR. ARkit) and
ARCore XR plug-in. Although both ARkit SDK and ARCore SDK are used in the end, due to the
re encapsulation of Unity, the API called by C# is slightly different from that of professional
platforms (Konstantin, 2020).

The goal of AR Foundation is not limited to ARkit and ARCore. Its goal is to build a
unified and open AR development platform. Therefore, AR Foundation is likely to include other
AR SDKs in the next development to further enrich the AR development environment (Wang,
2019). In further development, AR Foundation not only supports mobile AR devices, but also
supports the development of wearable AR devices.

88

An AR Session Origin was added into this project. The ARSession object consists of two
components, one is ARSession, which manages Session, and the other is AR Input Manager, which
manages input related information. Session in AR is used to manage the status of AR applications
and process the life cycle of AR applications. It is the main entrance of the AR API and controls
the enabling or disabling of AR on the target platform. The life cycle of the Session is handled in
Unity's corresponding life cycle method, so that the AR application starts or pauses camera frame
collection, initializes, or releases related resources as needed. The AR scenario must include the
ARSession component, but the ARSession can be mounted on any scene object (Figure 3.34).
Generally, it is mounted on the ARSession object for easy management.

Figure 3. 34 AR Session Origin
3.4.2 Marking Building Model
After completing the above steps, components were marked in the model to add functions
and code. The reason is that different colors will be displayed for special components with
89

functions, so that the facility manager can clearly see which component is a functional
component within the scope of FM. For example, the color of the door to was set to pink, which
represented the position of the starting point.

Then one wall was set to blue, which means that the blue walls contain components such
as water pipes or cables. Components set to green would then be given the function of displaying
their material information after clicking. Components hidden in the wall were set in pink to
distinguish them from components in different positions. After completing these steps, the
functional partition of the whole model was more obvious (Figure 3.35).

Figure 3. 35 Final in Unity 3D
3.4.3 Edit Information of Models
One can display component information about the green area, and the collection of
information came from the material information in the Revit model. This function is to show the
90

information of the component like the material and size. First, give a pop-window function to the
components which were colored to green. Then, a text box with its information was shown on the
component. In the final application, when the component was clicked, the text box would appear
on the center of it. (Figure 3.36).

Figure 3. 36 Using pop-window to show material information

3.4.4 Set up a Start point
Different from the image scanning location and model placement location commonly used
in previous experiments and mentioned in Chapter 1, this case study adopts a new method to match
the virtual model and real-world components. The principle is to create an initial position at a
certain point in the model. When the software starts, the image of the initial viewpoint will be
displayed automatically (Figure 3.37). The starting point was set on the door marked with blue and
its height is 5 cm. In practical work, it is necessary to calculate the height to ensure that the model
is aligned with the real world. A cube is established, and then AR camera is assigned to the cube,
and functions such as "Box Collider" and "Mesh Renderer" are added. At the same time, the
91

function of tracking and the function of central starting point were added to "box collider" to locate
the overall model (Figure 3.38).

Figure 3. 37 Set up for start point
92

Figure 3. 38 Set up for start cube
As users move their devices in the real world, their position in the virtual world has changed
and compared to the real world. The authoring methods and code for other features such as
zooming in, zooming out, rotating, and moving will be detailed in Chapter 4.
93

3.4.5 Create the Application AR-FM
After the steps above, the next work was also in Unity 3D, to set up the application AR-
FM. Based on the model of Watt Hall that was imported from 3ds Max, function was created by
code in Visual Studio and UI panel (Figure 3.39). In Unity 3D, a UI panel was established in
“Canvas” with some buttons which would be given function in Visual Studio. To be specific, the
function in AR-FM included Highlight the component (Figure 3.40), Movement, Rotation and
change the size (Figure 3.41), hide and show the target component (Figure 3.42) and go back to
the welcome panel (Figure 3.43).

Figure 3. 39 Diagram of creating AR-FM

94

Figure 3. 40 Highlight function of AR-FM

Figure 3. 41 Moving, rotating and scaling function of AR-FM

Figure 3. 42 Turn on camera of AR-FM (Left), Turn off camera of AR-FM (Right)
95

Figure 3. 43 Welcome panel of AR-FM
3.4.5.1 Add Inside Wall Picture to the Wall
As for the wall in blue, there are 6 pipes inside, to made it more clearly for the facility
manager, a picture that could show the inside pipe group were added on it. It is created by the
screenshot in 3ds Max, and first, imported into Unity 3D and then dragged on the wall (Figure
3.44). And one panel with 6 buttons was also on the wall, when the users click one of the pipes on
the button (Figure 3.45), the pipe will be clicked and highlighted (Figure 3.46).

Figure 3. 44 Add picture on the wall
96

Figure 3. 45 Panel on the Wall to select pipe

Figure 3. 46 Selecting the pipe by buttons on panel
3.4.5.2 Wall with inside Components
First, for the blue walls, as mentioned earlier, the walls with internal components are
rendered blue to be recognized by the facility manager. First, a "canvas" was created on the wall
and a "panel" and "button" inside. There was a total of six water pipes inside the wall, so the six
buttons relate to the six water pipes, and the pop-up function was given here. A picture showed
the inside wall pipe is added on the wall to give a guideline of the inside wall structure to the users
(Figure 3.47). Whenever users click a button, the corresponding water pipe will be highlighted
97

and selected at the same time. Then, through the moving function, the whole water pipe was pulled
out of the wall, so that the facility manager will see the component information inside the wall
through AR Technology (Figure 3.48).

Figure 3. 47 Add pipe picture on the wall

Figure 3. 48 Render components in Unity 3D
3.4.5.3 other functions
In addition, functions including highlight and AR camera switch will also be created
through button link. These will be described in detail in the next chapter.
98

3.4.6 Package the Project
After setting up the software, in Unity 3D, click on the “File”, “Build and Run” (Figure
3.49). Then next step was to set up the building setting, package and export it to the device. First,
select the appropriate device type in the packaging interface, such as Android, and switch platform
(Figure 3.50).

Figure 3. 49 Setting the model
99

Figure 3. 50 Building setting for package up

Then, in “Project Setting”, chose the API of 7.0 Nougat and gave the name of the
application “AR-FM” (Figure 3.51). Clicked “Building and Run.” The APK file appeared in the
local disk. The Android device was connected to the laptop and the APK sent to it. When it was
installed, the application AR-FM could be loaded.

Figure 3. 51 Modification in project setting
100

3.5 User Interface
As for the user interface of AR-FM, it is made up with two parts, first is the welcome
panel (Figure 3.52), and the second part is the model operation panel (Figure 3.53). There are
several features shown on the screen when loaded the model, they are rotation, movement, scale,
turn on or turn off the toolbar and camera, the highlight of components, hide or show the
components and show the animation of installing or uninstalling the components group. For the
details of the function and the process of making them will be introduced specifically in Chapter
4.

Figure 3. 52 Welcome panel

Figure 3. 53 AR model operation panel

101

3.7 Summary
There are two important parts described in this chapter: one is the workflow that was
created to take a building model file from Revit to 3ds Max to Unity; the other the development
of a tool developed in Unity 3D called AR-FM.

In summary, the workflow from Revit to Unity is to create the model in Revit, and then,
gave the modification of models’ detail in 3ds Max, finally, in Unity 3D, import the code from
Visual Studio and gave the function to the model, connected with the UI panel, the application
AR-FM would be established.
Alignment of the Unity model to the real world is done like this in the AR model: give a
start point 5 centimeter high from the floor, and the vertical direction is located on the center of
the pink door. When the user started the application, put the camera of the device on the same
location in real world. Then the AR model would load and align to the real world.

AR-FM was built in Unity. Once a component is selected, it can be moved, rotated, scaled,
and highlighted by the users. If a component is moved, an original model will be saved as the
primary place, and all the components can be hidden or shown by the button. Walls in blue mean
that there are components inside it, the picture on the wall could show inside structure, and buttons
are also added on it, so users can click button to select the specific pipe. Also, the camera can be
turned on or turned off, to help the users to separate the AR model from real world.

In order to create the final tool for other people to use, plugins have to be downloaded. The
application was packaged from Unity 3D as an APK file for Android system. Users can send it to
102

their Android document when connect their device with laptop, then, install the APK file in their
phone, the tool would be loaded.

Chapter 4 will describe the code used to create the features of AR-FM.

103

Chapter 4

Introduction
In this chapter, some core codes and details about the methodology during the process of
creating the AR-FM app will be described. In Unity 3D, some functions were realized by
executing the developed scripts instead of inherent scripts. In addition, some functions could be
achieved by non-script functions like the user interface system (Figure 4.1). The software named
Fairy GUI was adopted due to the better compatibility and convenience compared with using the
UI system in Unity 3D (Figure 4.2).

Table 4. 1 Software and plugins to create AR-FM
Name Description
Unity 3D A platform that can edit models, integrate code, plugins and other
functions with models. Packages and generates the tool "AR-FM"
that can run on Android devices
Visual Studio Creates scripts that give AR-FM tools human-computer
interaction capabilities and adds them through the Unity 3D
platform
Fairy GUI A UI creation software with stronger compatibility and
performance, responsible for creating AR-FM Welcome Panel
and Operation Scene, as well as button creation
ARFoundation A cross-platform AR development kit that combines ARCore XR
Plugin, ARKit XR Plugin, Magic Leap XR Plugin, Windows XR
Plugin. It is an integral part of AR creation in Unity 3D.
Runtime transform
Handles
A plug-in that provides a handle for AR-FM when the user selects
the function of editing the component in the process of using it to
facilitate the user to perform further operations on the component
Video Player A plugin in Unity 3D that can read the video in project and play it
based on a UI panel

The features (described in Chapter 3) with red borders were accomplished by scripts, and
others were achieved by Fairy GUI (Table 4.2). In Fairy GUI, the UI scene were established with
104

the buttons but without function, the further additions of features would be done in Unity 3D and
Visual Studio.

Figure 4. 1 Welcome panel for AR-FM

Table 4. 2 List of C# scripts in AR-FM (All scripts were created in Visual Studio)
File name Description
WelcomePanelUI.cs
(MonoBehaviour)
A script that enables the users choose to start to load the Watt
Hall trest model or turn off the application AR-FM
EditorObjectSelection.cs
(ScriptableObject)
A script that the target widget selected by the user will be
identified, and then the following script functions will be
triggered:
1. By default, the moving function is automatically enabled for
the component, that is, the 3D coordinate axis that can be moved
appears.
2. Users can choose to rotate, zoom, highlight or hide the
component through the buttons.
3. If target item is the wall with components inside, a UI panel
will be active, users can use buttons on panel to select the
functions, if choosing component, the button will relate to the
target component, if playing animation or images, it will
automatically read the mp4 file or the jpg file which were
imported under the component file.
Highlighter.cs
(MonoBehaviour)
A script that once active the target object will have a glowing
effect. This effect is given to each widget and can be turned on
and off through the functions in EditorObjectSelection.cs
105

UIController.cs
(MonoBehaviour)
A script that can link the component inside the wall with buttons
on the UI panel on the wall. It can realize that the users click the
button on the panel to select the corresponding component on the
wall.

Figure 4. 2 Scripts applied in each function
4.1 Final Tool AR-FM: User Interface
There are two scenes of the user interface in AR-FM, the welcome panel (Figure 4.3) and
the operation scene (Figure 4.4). The smart phone used was a Samsung SE20 FE; there in Fairy
GUI, the resolution ratio of the UI scene is 2440*1080 to make sure the UI can be filled by the
enitre screen when AR-FM is working. Two functions were given to the "Start" and" Exit"
buttons separately. While users click "Exit," AR-FM will turn off while choosing "Start," the
scene will be transferred into the operation scene.
106

Figure 4. 3 Welcome panel for AR-FM

Figure 4. 4 Operation scene for AR-FM
4.2 Welcome Panel
In a welcome panel, in addition to,"Start" and "Exit," there are three other parts (Figure
4.5). First two text boxes are inserted, one is the text marked "AR-FM" and the other is the user
guide on the left side of the screen. At the same time, the floor plan of the third floor of Watt
Hall is inserted on the right side of the screen so that users can have a preliminary understanding
of the entire model when they start the software at the beginning (Figure 4.6). The aspect ratio of
the window would be changed for the device. In the case study in Chapter5, the device for case
107

study is Samsung S20 FE, therefore, the aspect ratio is 2440*1080, if this ratio is used in other
device, the buttons on operation will be moved, and the whole scene will look incongruous.
Therefore, before package the project into device, the aspect ratio of device should be checked
and modified in Fairy GUI.

Figure 4. 5 Text box and picture in welcome panel

Figure 4. 6 Editor for welcome panel

In Fairy GUI software, there was no importing of script, just create the two scenes with
buttons. Once they were all finished, chose to publish, and save to the resource folder of AR-FM
project in Unity 3D (Figure 4.7). Then, open AR-FM project in Unity 3D, the UI would be
shown in the project (Figure 4.8).
108

Figure 4. 7 Publish setting of UI

Figure 4. 8 Importing UI panel in Unity 3D
109

4.3 Operation Scene for AR-FM
To set up the operation scene, the first step in Unity 3D is to transfer the project
environment to Android to make sure the project can work appropriately no matter what the
resolution ratio or the compatibility (Figure 4.9). Two scripts are in the UI panel, the first “UI
Panel” is from Fairy GUI, and the second is used to enable users to control the components
(Figure 4.10). In the operation scene, users can click the target component to edit it or check the
direction again. They can also go back to the welcome panel to restart the application (Figure
4.11).

Figure 4. 9 Change building environment to Android

110

Figure 4. 10 Scripts in UI panel

Figure 4. 11 Operation scene for AR-FM

4.3.1 Select the Component
For all the components in AR-FM, the Mesh Renderer and Mesh Collider were added.
They would ensure that when the application is in work, every component will be an independent
unit (Figure 4.12).

Therefore, in an actual work application, when the user clicks the screen to select an
object, due to the existence of the collider, if the range selected by the user is within the range of
the collider of the component, the component can be selected. At the same time, the collider will
111

also turn the entire model into an individual composed of individual components, so that when
selected, the system displays the individual components, rather than the entire Watt Hall
building.

Figure 4. 12 Mesh Collider and Mesh Renderer for AR-FMM

4.3.2 Edit the Component
This section is about three functions: moving, rotating, and scaling. After the user selects
the target component, the buttons below can be used to switch between the three operations, and
the three operations correspond to different operation modes. Among them, the operation mode
of the moving function is to display a three-dimensional coordinate axis with the terminal as an
arrow at the position of the axis of the component (Figure 4.13), the rotation function is to
display a three-dimensional sphere at the axis (Figure 4.14), and the zoom function displays the
three-dimensional coordinates of the terminal as a cube axis (Figure 4.15). The operation modes
are all from “Assert Store” in Unity 3D. The name is “Runtime transform Handles” (Figure
4.16). By importing this plugin, once the component is selected, the operation mode will be
shown at the pivot point.

112

Figure 4. 13 Moving mode

Figure 4. 14 Rotation mode

Figure 4. 15 Scaling mode
113

Figure 4. 16 Plugin to show the operation model

4.3.3 Highlight the Target Component
Highlighting the component was applied in two areas, the first one is when users would
like to add a mark to items that they think are important, click the button named “Tag” and then,
it will be highlighted. The second one is using buttons to select the pipes inside the wall (Figure
4.17). To realize this function, all the components were added a script named “Highlighter”
(Figure 4.18). The code in the picture is out of the closed state. Once this code is activated, the
component it is in will enter the highlighted state. Therefore, this code can be thought of as a
light switch, and activation is equivalent to turning on the switch. Therefore, using the button to
control the highlight is equivalent to indirectly controlling the highlight switch. In AR-FM,
whether the button "Tag" or the button on the wall is to control whether the code named
"Highlighter" is activated to realize the process of highlighting objects.

114

Figure 4. 17 Using button to select pipe and highlight it

Figure 4. 18 Script “Highlighter” in closed state

4.3.4 Using Start Point to Locate and Load the Watt Hall Model
ARFoundation and its corresponding affiliated plugins are used to locate the start point
and load the 3d model. If the user clicks the start button on the Welcome Panel, the model will
be loaded and displayed at a specified location (Figure 4.19), that is, above the pink door marked
on the Welcome Panel. So, if the user starts AR-FM at the corresponding location of Watt Hall
in the real world, the AR model will be aligned to the real world.

115

Figure 4. 19 View of start point in AR-FM
To realize this work, first, the ARFoundation needs to be imported from “Window,”
“Package Manager” (Figure 4.20). Then, in project scene, add AR Session Origin and AR
Session (Figure 4.21). The former is to control the position of the AR camera, and at the same
time, when the device is running, once the position of the device has moved in the real world, the
AR model will also track and move in the virtual world (Figure 4.22). The latter is the master
switch of the entire AR modeling, controlling whether the AR model loading can run normally
(Figure 4.23).

Figure 4. 20 Import ARFoundation

116

Figure 4. 21 Add AR Session Origin and AR Session into project

Figure 4. 22 AR Session Origin

117

Figure 4. 23 AR Session

To make sure the view of AR model has a more precise location, a cube is created to help
locate the ARFoundation plugins. The core of the cube will be set at the 5 centimeters height at
the center of the door, then, copy the coordinate of paste into the plugins, to give the precise
location to it. After all these steps were finished, the cube is hidden to avoid it would be shown
in the model (Figure 4.24).

Figure 4. 24 Hide cube when finish setting the location of ARFoundation plugins
118

4.3.5 Hide or Show the Target Component
Hiding and showing components is another feature. When using the app, if users find
their target component is hidden by other objects, click it and choose button “Hide”, it would be
hidden, and once they finish checking, click show to put them back. This function is similar to
the function “Highlight”, in Unity 3D, every item can be hidden in its “Inspector” scene (Figure
4.25), the script will turn on or turn off the component in the selection (Figure 4.26).

Figure 4. 25 Switch of hiding or showing component

Figure 4. 26 Controlling hiding or showing component
119

4.3.6 Turn on or turn off the Camera
The scene shown by the real camera is sometimes a distraction to the viewing of the
entire AR model. Therefore, if people want to avoid this effect or would like to separate the AR
model, they can click the button “Camera” to turn off the real-world camera, which means the
AR model is isolated.

The method of this function was briefly mentioned in 4.3.3 and 4.3.5. In 4.3.3 and 4.3.5,
some functions can be achieved by controlling the hidden” button or showing in Unity 3D like
turning on or turning off the switch. Therefore, connecting the button “Camera” with the Editor
Camera will make it available to turn on or turn off the camera. While the button is clicked, the
Editor Camera will be turned off and click one more time and it will be back (Figure 4.27)
(Figure 4.28).

Figure 4. 27 Turn off the camera
120

Figure 4. 28 Turn on the camera
4.3.7 Showing Information about Components and Direction of Operation
In AR-FM, the components in green have a special function; when users click them, they
will automatically show a pop window to show the information of target component like the
material, size, and work performance. If users don’t want to see it, click close to turn it off
(Figure 4.29).

Most of the users were first contact with this type of application, during their work
process, they maybe forget how to use this. Therefore, clicking help can also show them the
direction of AR-FM so they don’t need to go back to the welcome panel to check the instructions
again (Figure 4.30).

121

Figure 4. 29 Click to show information of component

Figure 4. 30 Showing direction
These two functions were all based on the panel function in Unity 3D. As for the former,
a “Canvas” would first be added in the subset of target component which contains a panel and a
text box (Figure 4.31).

To ensure that it will not be displayed under normal conditions, the entire Canvas is
hidden after entering the text and typesetting the text and panel. The final step also controls the
122

switch of this canvas, click on the component means turn it on. The close button below the panel
can control turning off the Canvas.

Figure 4. 31 Add information of some components
As for the “Help” function, it was created in Fairy GUI; for the specific methodology
refer to 4.2.

4.3.8 Playing Animation and Image
The final functions are to give the users, especially the facility managers, more guidance
of the building. The current functions cannot ensure the users click the objects behind others; for
some groups of related objects inside a wall (like pipes and connectors), a good way is to show
the image of them or even providing an animation to show how to install or uninstall the group.
Once users choose the wall, a panel would be shown, and some choices appear with the buttons
(Figure 4.32). The method of pipes is the same as previous wall mentioned in 4.3.3.

The users have a choice of the three buttons: Replace the valve, Install Whole Model, or
Uninstall Whole Model. After clicking one, corresponding animation will play. During the
animation, the necessary tools needed for this process will be listed to give more guidance to the
users (Figure 4.33).

123

Figure 4. 32 Panel to select animation to play

Figure 4. 33 Animation to guide users how to replace valve with text explanation
To create this function, first, three videos were improved in Unity 3D at the sunset of the
target wall (Figure 4.34). Then, to make sure the system can read these videos, a video player
plugin needs to be added in this project (Figure 4.35). Three different option buttons correspond
to three videos; when clicked, the video will be activated and displayed on the panel.

124

Figure 4. 34 Import three videos

Figure 4. 35Video Player to control reading videos
125

4.4 Made the Tool AR-FM
Most of the above functions are completed based on the code. For the selection and
identification functions of components, the corresponding picture text and influence functions
are displayed or hidden in the form of control switches through buttons, the camera tracking
function, show information of components, selection of wall components, and functions. The
operation of the entire UI controller is implemented based on C# (Table 4.3). Therefore, in this
part, the creation process of these function codes will be explained in detail.

Table 4. 3 Features and their corresponding scripts
Features Corresponding Scripts
Recognize and Select Components EditorObjectSelection.cs
Camera Tracking Function CameraFlyController.cs/ EditorCamera.cs
Show Information of Components Highlighter.cs
Hiding or Showing Components EditorObjectSelection.cs
UI Controlling UIController.cs

4.4.1 UIController
The method is executed after the editor has executed the pressed command (Figure 4.36).
About this function of UIController. The reason why it was created and added to AR-FM is that
in the whole AR-FM, users can either click the buttons of the UI interface, or directly click the
components to edit. Therefore, this means that the UI interface is superimposed on the model
interface. When the user clicks a certain point on the screen, if the clicked position is located on
the button of the UI interface, as shown in the figure (Figure 4.37), then the UIConroller will
judge, and the result is that the user’s click operation will trigger the button function instead of
the widget that the button is obscuring. If the range of the motor is within the object and the
collider area, it will be judged as a clicked object, and then they can edit the object. This function
126

makes it easier to help users edit the components no matter clicks to check the information or use
button to edit them.

Figure 4. 36 Click the button with component behand

Figure 4. 37 Judge whether users click the component or button
Second, for selecting objects inside the wall, the user will first click on the wall and then
search for the subset, so here the system first needs to judge whether the problem is empty. If it
is not empty, the selection of the object will be performed, that is, after the user clicks on it, the
selection is to choose whether to move, to rotate, or to zoom, and to further classify.
127

There are six pipes on this wall, the six pipes correspond to the six buttons of Pipe1 and
Pipe6 respectively, and the function of UIController is to connect the buttons with the six
corresponding pipes to ensure that when the user clicks the button (Figure 4.38), the
corresponding pipe will be selected.

Figure 4. 38 Link the pipes with corresponding water pipe
4.4.2 Highlighter.cs
Highlighter.cs causes the glowing effect of the pillars. There is a switch, one state for
three rendering states. This is a production of an effect of an external glow. First, it has a
parameter that will be passed. This value points to this object, because different components
render different lights when highlighted, for example, green components will emit green light
when highlighted, while common components will emit yellow light. Therefore, it is necessary to
identify the rendering function of the entire component here to prevent wrong rendering effects
when the highlight function is selected. (Figure 4.39).

128

Figure 4. 39 Clear all the highlight when click the pillars
4.4.3 Camera Tracking in Both Real-world and Visual World
During the operation of AR-FM, the user will hold the device and move in the real world,
and the view of the virtual world will also move along with it. This is like adding a tracking
function to the AR model (Figure 4.40). When moving and changing, the view in the virtual
world also changes together.

Figure 4. 40 Camera Tracking in AR-FM
129

4.4.4 UI Controller
Since the UI interface in AR-FM is imported in the Fairy GUI, for better compatibility,
the script UIPanel.cs will be imported into the Unity 3D system. The building is made up of
individual componentsAs mentioned in 4.4.1, when the user selects a component and moves the
widget by dragging it, the UI button will be temporarily turned off to prevent when the user from
sliding the fingertip on the screen, the part of the button is touched to cause the system to
implement the function of the button, which affects the user's operating experience (Figure 4.41).
When users click on a green component or a blue wall, normally the green component
will display the material information first, and the blue wall will display the panel for button-
based operations.

Therefore, it will first read whether the target object has a sub file, because the green
components and blue walls, their material information or pictures and videos all exist in the form
of subsets under the component file. Therefore, making a judgment in this step can better
distinguish special components such as components that can display information and play videos
from normal components (which can only perform basic editing).

130

Figure 4. 41 Judge whether the target component has subset before further work
4.5 Create AR-FM
In the project “AR-FM” in Unity 3D, after creating the scripts and adding functions and
plugins above, the model has been preliminarily completed. The script will be added to the
corresponding widget, and the UI interface will be added to the project (Figure 4.42). For the
code part after the script is completed, it will be imported into Unity in the form of a CS file.
After selecting the component, click "Add Component" to add the script and its corresponding
function. After completing the above steps, select "Build Model" to export the APK file to the
local disk. Since AR-FM is a cross-platform plug-in based on ARFoundation, one only need sto
choose to switch to a different Platform in "Building Setting" to switch between different device
systems, such as iOS, windows or PS4 modes (Figure 4.43). The device used in the case study in
Chapter 5 is Samsung SE20. The exported APK file can only be used for this model and all other
Android devices.

131

Figure 4. 42 UI panel in project AR-FM

Figure 4. 43 Transfer different operation system for AR-FM
4.6 Summary
In this chapter, some core methods and scripts during creating AR-FM in Unity 3D were
introduced (Table 4.4). It can totally divide into three steps: linking buttons with a function,
making judgement for the further function for target component, and read the file or videos of
the components when their further work needs to be loaded (Figure 4.44).

Table 4. 4 Software and plugins to create AR-FM
132

Name Description
Unity 3D A platform that can edit models, integrate code, plugins and
other functions with models. Package and generate a tool
"AR-FM" that can run on Android devices
Visual Studio Create scripts that give AR-FM tools human-computer
interaction capabilities, and add them through the Unity 3D
platform
Fairy GUI A UI creation software with stronger compatibility and
performance, responsible for creating AR-FM Welcome
Panel and Operation Scene, as well as button creation
ARFoundation A cross-platform AR development kit that combines
ARCore XR Plugin, ARKit XR Plugin, Magic Leap XR
Plugin, Windows XR Plugin. It is an integral part of AR
creation in Unity 3D
Runtime transform
Handles
A plug-in can provide a handle for AR-FM when the user
selects the function of editing the component in the process
of using it to facilitate the user to perform further operations
on the component
Video Player A plugin in Unity 3D which can read the video in project
and play it based on a UI panel

Figure 4. 44 Workflow about adding scripts to functions
In chapter 5, directions on how to use the AR-FM app will be given.

133

Chapter 5
Introduction
In this chapter, the practical application of the app AR-FM will be discussed. First, the
work process for Revit will be introduced, especially on the preparation for the complete Watt Hall
Model. Then, the method about creating the tool AR-FM from Unity 3D to Android device will
be explained. The next part illustrates how to use this tool for a real-world example, which is the
most important part in this chapter. At the end of the chapter, the practical effect and limitation are
discussed based on the comments of five volunteers who tried the app.

5.1 Model Preparation Process in Revit and 3dsMax
A third-floor section of Watt Hall at the USC, School of Architecture, was chosen as the
case study. The Revit model was provided by the USC Facilities’ Planning Department. However,
some components like MEP system were not included in this model, so additional modification
need to be added to complete this model. This building has total 4 floors, one is underground, and
others are above ground.

5.1.1 Load Watt Hall Model
First, the entire model of Watt Hall is opened in Revit (Figure 5.1). However, only the
third floor would be taken into consideration, so the other floors and sites will be deleted (Figure
5.2).
134

Figure 5. 1 Whole Model of Watt Hall

Figure 5. 2 Third floor of Watt Hall5.1.2 Remove Unnecessary Components
While the third-floor component was separated from the whole model, some unnecessary
components needed to be deleted. There are two reasons for this work, first, the application AR-
FM is used for facility management, therefore, items like the furniture, the stairs and so on,
which are not related to the FM, could be deleted. Another reason is that the model file is very
large. It has been tested in the AR program created by the first complete model. Because the
stuck model deviates from the original position, affecting the normal operation and accuracy of
the whole application, unnecessary elements must be deleted under the conflict between
135

aesthetics and performance. Finally, the model will contain elements like the wall, floor, some
columns, the pipes, cables, and so on (Figure 5.3).

Figure 5. 3 Watt Hall model after removing unnecessary elements
5.1.3 Add Core Components
Like it was mentioned in 3.2.2, the main aim of this work is for FM. To affect the
premise of appropriately deleting unnecessary elements from the existing building model,
another important step is to add other elements related to FM, such as ventilation system, piping
system, and so on (Figure 5.4) not shown in the figure. These are FM related factors, which will
be endowed with more functions to assist in the process of improving FM in AR-FM app. As
mentioned earlier, to better align the models and objects between the virtual and real world,
gamma AR software is adopted for subsequent calibration.
136

Figure 5. 4 Add pipes to Watt Hall model5.1.4 Export Model to.FBX
Revit can export files in many formats including FBX. After exporting the FXB file to
3ds Max, the main purpose is to further optimize the Revit model and make it more compatible
with Unity 3D, such as the saving of model color and rendering and the correspondence between
each component. To have better operability in the later stage, the pivot point needs resetting.
Another important point is that one of the functions of this experiment that can provide
convenience for facility manager is to add an animation on the complex structure group to show
workers how to disassemble and reinstall this complex component group. The above information
will be explained in this section.

5.1.5 Import Revit file into 3ds Max
After you export an FBX file from Revit, it can be read directly by 3DS Max. After
importing it, select Autodesk Media & Entertainment. After successful import, the model can be
divided according to each component on the left side of the screen. You only need to select the
name of the component, and the corresponding part will be selected in the model.

137

5.1.6 Change Materials to Standard
Importing Revit files directly into Unity 3D will make the 3d model lose its material, and
the result will show a single white, which will make it difficult for visualization purposes. One of
the solutions is that the software 3ds Max can be used to change the material information of the
FBX file from Revit, and then, the FBX exported from 3ds Max will be able to keep the texture
of the model.
Therefore, 3ds Max is a very important software to further process and modify the
original Revit model, and its ultimate purpose is to promote the normal progress of the whole
experiment. After the FBX file is exported to Revit, import it into 3ds Max before importing
Unity 3D.
After changing the material configuration, importing the model into Unity 3D again will
maintain the same rendering level as in Revit. It is also very easy to check whether the change is
successful, that is, select the material option in the render option, and then select get all scene
metal to find the conversion of materials (Figure 5.5).
138

、
Figure 5. 5 Check the transform of building material
5.2 Import 3D Model to AR-FM
If the users want to create the scene, they first need to download the plugins into this project;
the plugins contain AR Foundation, ARCore XR Plugin, Visual Studio Code Editor XR
management (Figure 5.6), which are important for the creation of an AR model.
139

Figure 5. 6 Import plug-in into project
Once the import of plugins is finished, the FBX file from 3ds Max will be added into
project “AR-FM”, under the folder “Assert” (Figure 5.7). The components under the model are
set to the “Converts Units” to make sure all the individuals can be added the mesh collider, and
this step is to help the components be given the function like moving, rotation, highlighting,
showing or hiding components, turning on or turning off the camera, choosing the components
inside the wall, playing animation, and in the operation scene, users can also check the direction
about using this tool.(Figure 5.8).

Figure 5. 7 Import Watt Hall model into project
140

Figure 5. 8 Mesh Renderer for Watt Hall model
5.2.1 Set up Start Point
According to 3.4.4, AR-FM needs to have a start point to achieve the matching between
the real world and the visual model. The specific method is to make one point in the Watt Hall
model to be as the start point, when this application is loaded in device, the scene on the screen
will start at the view of this point, so if the users start AR-FM in this location in the real world, the
visual model can align with the real world. This method uses a cube to locate it. In AR-FM, it was
put on the center of the pink door, and the vertical height set to five centimeters (Figure 5.9). After
the cube is used to align the AR model to the real world, it will be hidden to avoid it being visible
in the model.
141

Figure 5. 9 Start Point
5.2.2 Unpack the Prefab
In AR-FM, all the components can be clicked by the users; therefore, it is necessary to
unpack the whole model group to the collection of individual components (Figure 5.10). As for
the script of “ClickObject”, see Chapter 4 the introduction of function.
142

Figure 5. 10 Unpack prefab
5.2.3 Make AR-FM as the Parent of Watt Hall Model
Finally, drag the Watt Hall model in “Hierarchy” to the “AR Session Origin”, “AR
Tracked Image Manager”, “Tracked Image Prefab” (Figure 5.11) (Figure 5.12). Then delete it in
the scene. Components that are not important for facility management like furniture would be
hidden or deleted if not already done so previously depending on the users’ needs.
143

Figure 5. 11 Edit Watt Hall model
144

Figure 5. 12 Give AR function to model
5.3 Build the Tool to Android System
In this section, the process from setting up the project like the “Target Device”, “Target
Minimum Vision,” and to how to download and install AR-FM application in the devices is
explained.

5.3.1 Android System Setting
Under the “Project Setting”, “Player”, “Android Setting”, change the product name to
“AR-FM.” The package is modified to “com.DefaultCompany.ARFM”. Minimum API Level
uses Android 7.0 ‘Nougat’ (Level 24), Target level is “Automatic”, API Compatibility Level is
_NET Standard 2.0, and the Target Architecture is ARMv7 (Figure 5.13) (Figure 5.14).
145

Figure 5. 13 Building Setting for AR-FM

Figure 5. 14 Building Setting for AR-FM
Next, in “Building Setting”, click “Build” (Figure 5.15). The APK file named “AR-FM”
will be saved in the project folder in the local disk (Figure 5.16).
146

Figure 5. 15 Build AR-FM

Figure 5. 16 Find AR-FM APK file in local folder

5.3.2 Convert the Application from Laptop to Android Device
The next step is connecting the Android device with the laptop; the device used in this
example is Samsung SE20. Send the APK file “AR-FM” from local disk to the “Device Name”,
147

“Internal Storage”, “Document”, “Android” (Figure 5.17). Then, turn on the device, install the
APK file, and the application AR-FM will be found on the main interface (Figure 5.18).

Figure 5. 17 Send AR-FM file to device

Figure 5. 18 Find AR-FM on device
5.4 Case Study in Watt Hall
This section is the most significant part in this chapter. After introducing how to create
the application AR-FM, the method of using AR-FM will be explained, including the features
and user directions. In addition, five volunteers who are not architecture majors were asked to go
148

to the Watt Hall third floor to use AR-FM by themselves and give their advice about the AR-FM
app.
AR-FM has two scenes, the first is a welcome panel, and the second scene is the
operation interface. In this part, the following 10 functions will be described (Figure 5.19):
1. Welcome Panel
2. Load AR Model at Start Point
3. Back to Welcome Panel
4. Edit Component
5. Show Direction of AR-FM
6. Turn on/ off Camera
7. Show and Hide Component
8. Toolbar Operation
9. Special Function for the Component in Green
10. Special Function for the Wall with Components inside

Figure 5. 19 Operation scene of AR-FM
149

5.4.1 Welcome Panel
When the users open AR-FM, a welcome panel will be shown on the screen (Figure
5.20). There are two parts of the welcome panel. The first part is a brief direction of how to use
the application. On the left is the direction of using this model. The users should follow these
steps:
• The start point is at a height of 5cm in the center of the second door on the lower right of
the floor plan
• When the camera of device is on the start point, click the START
• The components can be clicked to for movement, rotation, scaling, showing or hiding
components, turning on or turning off the camera, choosing the components inside the wall,
playing animation, and in the operation scene, users can also check the direction about using this
tool
• The components in green can show the information of the material
• The wall in blue contains components inside a wall; click to find it

Figure 5. 20 Welcome panel for AR-FM
On the right is the floor plan of Watt Hall third floor, and this model can be divided into 4
parts, first is the model with original color, these components have no special function; they can
150

only be edit by the user. As for the components in green, when the users click it, it can show the
material information.
As for the door in pink, this is the door with start point where the users should start the
application. For the wall in blue, there are components inside of the wall; users can not directly
click it, they need to use the buttons on the wall to choose the inside component and use other
functions like watch the inside pipe system picture or see the animation about how to install or
uninstall the components.
5.4.2 Load AR Model at Start Point
To make sure the AR model can be loaded and aligned with the real world, users need to
put the camera at the start point, which is at the door in pink, at the height of 5 centimeters. The
start point set up in Unity 3D; the reason for choosing this height is when turning the smart phone
sideways and put on the floor, the distance from the floor to the camera is 5 centimeters. Therefore,
the users need press the screen against the door, the camera needs to be put in the center of the
door. Considering that when the screen presses to the door, users can not click the button, so when
the START button is clicked, there are 5 seconds for loading the model, it will be convenient for
users to operate it. When finish the loading of model, the screen will transfer from the welcome
panel to the operation scene (Figure 5.21).

Figure 5. 21 Watt Hall model view in start point
151

5.4.3 Back to Welcome Panel
Sometimes the start point will give an error that the AR model cannot algin with the real
world, and some users are not totally familiar with how to use it, so on the lower right corner of
the operation scene, the button “Back to Main Panel” will help them go back to the first step,
they don’t need to turn off the application and restart it. Once they choose back to welcome
panel, they can use the button “START” again to reload the model (Figure 5.22).

Figure 5. 22 Back to welcome panel to restart model

5.4.4 Edit Component
There are four buttons for editing the component when click it: move, rotation, scale, and
tag. When the component is clicked, a three-dimensional coordinate axis at the component’s pivot
point, users can use the three-dimensional coordinate axis to operate the component to make
modifications. When the component changed, a copy item will be leaved at the original location
to remind the users the original translation, rotation, and scale of the component.
When moving the component, when the component is first clicked in the operation scene,
the system will default to use the moving function. So, a three-dimensional coordinate axis with
three arrows will show on the target component (Figure 5.23). When users press the arrow and
slide on the screen, the component can move according to the arrow (Figure 5.24), and if one
152

presses the center of the three-dimensional coordinate axis, components will be moved on the two-
dimensional surface.

Figure 5. 23 Moving component

Figure 5. 24 Component after moving
Rotating the component can make it easier for the users to see every side of the component.
While clicking the component, click the button “Rotation”, then a three-dimensional sphere will
appear on the location of the pivot point (Figure 5.25). When dragging the sphere, the component
will be rotated, and the users can see all the sides of the component (Figure 5.26).
153

Figure 5. 25 Rotate component

Figure 5. 26 Components after rotating
When choosing the scaling function, a three-dimensional coordinate axis with the cubes on
its end will appear (Figure 5.27). User can change the size of the component on the along the axis
and when they drag the center of the three-dimensional coordinate axis, it can be scaled on the
two-dimensional surface (Figure 5.28).

Figure 5. 27 Scaling components
154

Figure 5. 28 Component after scaling

During the using process of AR-FM, the users might want to mark a component that is
significant in the work, as it could be difficult to find. Pressing the button “tag,” will highlight the
object (Figure 5.29), and they can easily find it when they go back to the previous place.

Figure 5. 29 Highlight component
By combining these four functions (move, rotation, scale, and tag), users can be
visualizing a component in whatever view and size they want. A copy is created and used so that
the original is not changed.
5.4.5 Show Direction of AR-FM
According to the welcome panel of AR-FM, the direction and function of component are
introduced. But during the process when users are in the operation scene, if they want to check it
155

again, they cannot back to main panel because the operating document cannot be saved, so they
can click the button “help” to check it again (Figure 5.30).

Figure 5. 30 Direction panel

5.4.6 Turn on/ off Camera
When “to model” is loaded at the location of the start point, the AR components can align
with the real world. However, not every user can arrive at Watt Hall to do this work. When
people start AR-FM in other places, the real world on the screen will be a distraction during their
usage. Therefore, users can click the button “Camera” to choose to turn on or turn off the real-
world camera. When this camera is turned on, the real-world will show on the screen, for the
situation that users are in the Watt Hall, it will be helpful for them to algin the AR model with
real-worl. As for people turning off the real-world camera, the background will be black, the
screen will totally show the virtual world, users can use it everywhere without the distraction of
seeing the real world.

5.4.7 Show and Hide Component
As for Watt Hall model, there are more than 2000 more separate components; therefore,
it is inevitable that some individuals will block each other (Figure 5.31). To see the items that are
156

behind others and keep its original location, click the items beyond the target, click the button
“Hide”, then, it will disappear, and the target item will be shown (Figure 5.32). After checking
the target item, click button “Show,” the hidden components will be back.

Figure 5. 31 Before hide

Figure 5. 32 After hide

5.4.8 Toolbar Operation
On the top of the screen, there is the toolbar. If users feel the toolbar block some of the
screen (Figure 5.33), click button “Toolbar status switch,” then, it will be hidden (Figure 5.34).

157

Figure 5. 33 Before hiding the toolbar

Figure 5. 34 After hiding the toolbar

5.4.9 Special Function for the Component in Green
As it is introduced in the welcome panel, when clicking the green components, the
information of it will be shown; users can learn the material, the size, and even the performance
of it. And after checking it, they can click button “Close”, the information will be hidden (Figure
5.35).

158

Figure 5. 35 Information of component
5.4.10 Special Function for the Wall with Components Inside
There are two walls in this model that have special functions, because there are pipes
inside the wall. According to the introduction above, the users cannot click items behind others,
and if just the wall is hidden, the pipe in the space will be hard to define its specific location.
Therefore, as for the first wall, when click it, a panel with buttons will show first (Figure 5.36).

Figure 5. 36 Panel on the wall
There is a total of 6 pipes inside the wall The button “Show Inside” will show the inside
image of 6 pipes aside from the panel (Figure 5.37).
159

Figure 5. 37 Show inside image on the wall
On the other hand, if users want to choose the pipes inside the wall, they can use the
button to have their choice. When clicking one of the buttons, the relative pipe will be highlight
edand clicked; users can use the function in 5.4.3 to edit the pipe and move it outside the wall
(Figure 5.38).

Figure 5. 38 Click button to choose pipe
As for the other wall, the inside pipe group is more comprehensive, so the users,
especially the facility managers who are not familiar with the group will need more information
about it. For facility managers, if they need to repair part of the pipe or take maintenance, the
animation will be shown about how to dismantle the whole group or parts of it. And six buttons
160

connect with six pipes, they can also click them like the previous wall. It is also operated by a
panel with button on the wall (Figure 5.39).

Figure 5. 39 Second wall of panel
There are three buttons related to the animation, install, and uninstall the whole model
(Figure 5.40), replace the valve (Figure 5.41). When playing the animation, the information of
pipe group and tools the facility managers need to use will be shown next to the animation to
give more guidance.

Figure 5. 40 Modify whole model

161

Figure 5. 41 Modify parts model

5.5 Feedback of Volunteers
Five volunteers were asked to use this software separately; they only followed the
instruction in the welcome panel, and they are free to move in Watt Hall to experience the
function in AR-FM (Figure 5.42) (Figure 5.43) (Figure 5.44). They gave comments about the
time it took for them to be familiar with the application, the best and worst parts of the
application, and suggestions for improvement. The following are their feedback (Table 5.1)
(Table 5.2) (Table 5.3) (Table 5.4) (Table 5.5).

Figure 5. 42 Volunteer using panel on the wall
162

Figure 5. 43 Volunteer using panel on the wall

Table 5. 1 Feedback of Volunteer #1

163

Table 5. 2 Feedback of Volunteer #2

Table 5. 3 Feedback of Volunteer #3

164

Table 5. 4 Feedback of Volunteer #4

Table 5. 5 Feedback of Volunteer #5

According to the feedback above, the animation of showing the installation or
uninstalling of the pipe group would be the best part in AR-FM and better rendering needs to be
added to make the model beautiful. In some cases, users who cannot arrive Watt Hall maybe not
have enough space to move in the model, it will be easier to them moving inside the model by
using analog stick. And some important components could have a navigation system to guide the
users to find the specific location in the model.
165

5.6 Summary
In this chapter, how to use the application AR-FM is discussed based on the real building
(USC Watt Hall). If users want to create an AR application like AR-FM, they first need to build
the model in Revit, then, process and modify the model in 3ds Max, if it is necessary, creating
the animations. The next work is in Unity 3D, visualize the model and create a user interface
panel to achieve function. Finally, package it up and download in devices (Figure 5.45) (Figure
5.46).

Figure 5. 44 Workflow of creating AR-FM

Figure 5. 45 Features of AR-FM
166

All the components are editable, and users can see it in different sizes and view as they
want. The animation will guide users to find the methodology of repair the inside comprehensive
structure group. By asking for 5 volunteers using AR-FM, this tool can provide some
convenience for facility management process. In conclusion, AR-FM can be as a prototype to
enable to visual the model and make it editable.

167

Chapter 6
Introduction
In this chapter, the research background and methodology will be summarized. Then, the
process of creating the application AR-FM will be analyzed, including the useful functions for
FM, some limitations and tricky problems that were solved. In addition, future enhancements for
the functions and performance of this application will be introduced. These improvements
contain both short-term target and long-term work; the former includes some easy updates and
small problems that could be solved or changed easily. As for the later, some challenging plans
and technology will be involved. And in the end, the potential development in the future will be
discussed.
6.1 Research Background and Methodology
In Chapter 1, the basic concept of Building Information Augment Reality (AR), Virtual
Reality (VR), Mixed Reality, Modeling (BIM) and Facility Management (FM) were introduced.
With the rise of AR technology, more industries begin to develop applications of this technology
and apply it to practical work. The same is true in the field of architecture, where augmented
reality could be an asset. Augmented reality in architecture and construction can show the
building model before the building process. It can also show the designer’s idea about how to
improve the building, which will be helpful to save time and cost and enhance the whole
efficiency of whole building process (Zlatanova, 2002). As for BIM technology, it is an
integrated workflow based on coordinated and reliable information about a project from design,
construction to operation. BIM is very mature in the field of construction. Through BIM
technology, the accurate building virtual model is digitally constructed (Salman, 2009). The
combining of BIM and AR technology is being gradually adopted in the AEC field; it has the
168

potential for improvement over traditional technologies in FM which could ensure the quality
and efficiency of building operation (Yang, 2019). By this application, BIM+AR can effectively
decrease faults in construction.

In Chapter 2, the previous research based on BIM, AR, or BIM+AR for FM were
discussed. According to these research cases, AR technology has been applied in the field of
architecture, including architectural design, facility management, and the entire life cycle for
architecture. However, the combination of AR technology and BIM technology still has certain
defects, especially the tracking difficulty. Although the combination of BIM and AR technology
needs further development, their use in the construction industry continues to grow. It is still a
new technology with great potential to transform architecture and building products and
processes.

To achieve the integration of multi-functional facility management, an application named
AR-FM was created that enables users to view components and guide facility managers in their
FM work like watching animation of installing or uninstalling a grouping of components. The
method and features in case studies were introduced in Chapter 3 and Chapter 5 respectively.
The whole process can be summarized in three steps (Figure 6.1):

1. In Revit, create the model and add the MEP system if not already present.
2. In 3ds Max, modify and process the model, keeping the texture and creating animation
3. In Unity 3D, visualize model building and create UI scene to accomplish human-
computer interaction, and package the application

169

Figure 6. 1 Enire Workflow of Creating AR-FM
In Chapter 4, the methodology of how to create the function of AR-FM were present in
the form of listing code and specific methods (Figure 6.2) (Table 6.1). In this process, Unity 3D
was used as a platform to combine the Watt Hall 3D model created from Revit and modified in
3ds Max, the mp4 file of animation, the scripts from Visual Studio, the plugins from the inherent
elements in Unity 3D or outside the Unity 3D, and the UI panel created from Fairy GUI.

The UI panel gives a Welcome Panel and operation scene to AR-FM; after creating the
scene and buttons from Fairy GUI, it can export to Unity 3D project (Figure 6.3). The scripts in
Visual Studio were attached into Unity 3D as an inserting component to added into the
component (Figure 6.4). The plugins were from the “Assert Store” (Figure 6.5) and “Package
Manager” (Figure 6.6). The videos and images were all imported into Unity 3D, putting them in
the sub-files of the corresponding components (Figure 6.7), and through the code settings, when
170

the user clicks the UI interface button to select the editing function, the corresponding video or
picture will be activated.

Figure 6. 2 Scripts to realize the functions in AR-FM
Table 6. 1 Scripts created for AR-FM
File name Description
WelcomePanelUI.cs
(MonoBehaviour)
A script that enables the users choose to start to load the Watt Hall trest model
or turn off the application AR-FM
EditorObjectSelection.cs
(ScriptableObject)
A script that the target widget selected by the user will be identified, and then
the following script functions will be triggered:
1. By default, the moving function is automatically enabled for the component,
that is, the 3D coordinate axis that can be moved appears.
2. Users can choose to rotate, zoom, highlight or hide the component through
the buttons.
3. If target item is the wall with components inside, a UI panel will be active,
users can use buttons on panel to select the functions, if choosing component,
the button will relate to the target component, if playing animation or images,
it will automatically read the mp4 file or the jpg file which were imported
under the component file.
Highlighter.cs
(MonoBehaviour)
A script that once active the target object will have a glowing effect. This
effect is given to each widget and can be turned on and off through the
functions in EditorObjectSelection.cs
UIController.cs
(MonoBehaviour)
A script that can link the component inside the wall with buttons on the UI
panel on the wall. It can realize that the users click the button on the panel to
select the corresponding component on the wall.
171

Figure 6. 3 Import UI panel from Fairy GUI to Unity 3D

Figure 6. 4 Attach codes in Visual Studio to Unity 3D
172

Figure 6. 5 Import exterior plugins in Unity Assert Store

Figure 6. 6 Manage Unity 3D inherent plugins in Package Manager
173

Figure 6. 7 Import Videos to AR-FM project
Chapter 5 first gave user directions of how to use AR-FM in the real work, including the
steps of using AR-FM.

The features of this tool were explained, and the methodology introduced for the users if
they would like to create an AR application like AR-FM (Figure 6.8). After the theory is
introduced, a case study in Watt Hall in USC School of Architecture was shown. Five volunteers,
who are from different backgrounds of study, were invited to use AR-FM by themselves; they
were only allowed to read the users’ direction to help them understand how to use this tool.

174

Finally, the time they were able to become familiar with this tool and their evaluation and
suggestions were collected, which was helpful in checking the convenience and limitations of
this tool and provide ideas for the future work.

Figure 6. 8 List of features in AR-FM
6.2 Limitations and Future work
In this section, based on the evaluation of the five volunteers and the challenges during
the process of creating AR-FM, many limitations that were discovered are discussed. The
location system for AR-FM is the most important element that needs to be improved. More
future work relates to the function and some operations of the application. These improvements
contain the short-term target and long-term work; the former contains some easy functions and
small problems that could be solved or changed easily. As for the latter, some challenging plans
and technology will need to be involved.

175

6.2.1 Limitation of the Animation
The volunteers thought that not only would it be difficult to make the animations, but that
the definition of the test animation was low and its content of the animation relatively simple.
Since many facilities have small components such as screws, it is difficult to clearly show the
user the more delicate dismantling process, such as the rotation process of the dismantling screw.
At the same time, for the animation, there is no pause function, so if the user does not clearly
understand a certain part of the animation, they can only wait for the animation to play the next
time in a loop to watch it again, which would be very time-consuming.

6.2.2 Limitation for Performance of AR-FM
Regarding the performance of AR-FM, it should be acknowledged that it can indeed have
a certain guiding role in the process of facility management. However, in terms of details, first,
in the rendering of the model, the texture in Revit is completely saved after being modified by
3ds Max. However, for non-professionals, it is difficult for them to understand the model, so if
more description can be added to the components or rendering with a more realistic model will
better assist them in their operation. On the other hand, the entire building model is relatively
large, and it is easy for users who enter Watt Hall for the first time to get lost, and the entire
model consists of more than 2,000 individual components, and they can easily lose the
positioning of the target components, because the set highlight widget feature does not lead them
to find widgets. Therefore, guidance on the specific positioning of the core building blocks is a
drawback in AR-FM. Finally, in the process of editing the component, it is achieved by dragging
the 3D coordinate axis, which is uncomfortable for many users, so there should be a joystick to
operate the movement of the component to provide more choices.
176

6.2.3 Filter Unimportant Elements
In AR-FM, the problem of too many components causes the colliders of many
components to intersect together in some spatial areas. Therefore, when the user clicks to select
this area, the system sometimes has difficulty in determining which area to use. A collider is
selected. Therefore, a better solution is to divide core components (such as load-bearing walls,
elements with other components inside, and MEP systems) and non-core systems, that is,
ordinary components in AR-FM, into two component groups. At the same time, adding a
function button is to filter non-important components. When this button is activated, all elements
of the non-important component group will be hidden, so that the overall AR model will be
greatly simplified, which is convenient for users to operate.

6.2.4 Navigation System to AR-FM
Based on the volunteers’ suggestion, the too many individual components and large space
this model, for facility managers, even they can totally understand this model, in real world, they
will be hard to find some specified target location. And for the users who are not the major in
architectural field, it would be a serious challenge for them to find a location by themselves.
Therefore, it’s necessary to add a navigation system into AR-FM. For the detail, this function can
be realized by UI panel. A list of the core components can be shown and selected on the
operation scene. Once the component is selected, the virtual route will be shown on the screen,
guiding the users to the specific location, and the target component is automatically selected,
users can do the operation on it.

177

6.2.5 Short-Term Plan
Initial short-term improvements and new functionality additions include visual changes to
the model components. For example, models with components inside the walls can be made
transparent to achieve visual functions. Secondly, it is further optimized for the whole model.
Because there is a certain deviation between the model and the building in the real world, only
the alignment of the target components is changed with the real world at present. The last point is
the optimization of the entire model content. Because the whole model is too large, useability
was an issue. Therefore, some unnecessary components are sketched, and transparent or linear
models are displayed in the entire operation interface to reduce the operating burden of the
equipment.

6.2.6 Long-Term Plan
For longer term goals, this will become more challenging and more practical. For
example, in the current application, the existing model rendering cannot be changed, but can
only be marked. In the future, the rendering of the whole model will be changed during the
operation of the device, such as adding more textures, so that users can customize the beauty in
use. Another important point is about components, more accurate positioning system in detail,
every time a component is selected, when once again chosen another member can automatically
measure the distance between the two components, at the same time, whenever the key
components such as water pipe has been moved, and the distance between the initial position will
be prompted to come out. In addition, it would be useful for users to make changes in AR that
can be sent back to the building information model and for the app to provide a map to the area
that needs working on. These would need further programming.
178

6.3 Discussion of Results
AR-FM realizes the combination of BIM and AR technology, which makes it possible to
assist FM in existing buildings, which includes editing the components based on matching the
model with the real components and provide more guidance for facility managers by displaying
material information, showing pictures of the internal structure of the wall, and animations that
demonstrate the disassembly and installation process of complex component groups. And all of
these features above could be achieved by the buttons in UI panel that integrate the functions
together (Figure 6.9). It is possible that an entire building information model displayed in the
form of human-computer interaction through AR technology could enhance the accuracy and
efficiency for facility management work.

Figure 6. 9 Diagram of UI panel for AR-FM
6.4 Analyze the Whole FM Process by Using AR-FM
In this section, a whole process from the facility manager gets the requirement of doing
FM work and then, learns the information of the target building, finally, finish fixing the
179

component will be described and give the feedback to the FM system. And then, the entire
workflow will be compared with the functions in AR-FM, showing the parts that AR-FM can
accomplish or cannot to do (Figure 6.10). At the same time, AR-FM is brought into a complete
FM workflow for analysis.

Figure 6. 10 Workflow of FM
6.4.1 Understanding the FM Task
In this section, Watt Hall will be as the target building and the process of FM will be
thought as with the support of AR-FM. When the FM department gets the information, they need
firstly get to know the information of the building needs for FM work.

This process contains researching target buildings from their difficulty of work, the
budget of the entire work, the building, material, and target components information. The
requirement of the maintenance workers and technology would also be taken into consideration.
180

During this workflow, AR-FM can help the FM manager to get the material information and
giving the information and what they need for the work process (Figure 6.11).

Figure 6. 11 AR-FM help FM managers know the requirement of work

6.4.2 Planning for the FM Task
After understanding the information and requirements of the work, the FM department
needs to give a great plan to make sure there will be no mistake in this work, The plan show
should contain the information of the real work like the date, process, workers’ arrangement and
the scope. Besides these, some potential hazards also need to be paid more attention to. Then, the
workers should do some simulation in the company, this is what they can realize from AR-FM.

6.4.3 Acting for Real Work
Once the research and planning process is finished, the next step is to come to Watt Hall
to do the FM work. Once FM workers arrive Watt Hall, they can first start AR-FM at the start
point to make sure the AR model can match the real world. If it is possible, the navigation
system will help them find the target component more quickly. And using the functions in AR-
181

FM like edit the model, check the animation to know more about the specific location of the
components inside the wall and know the work process of how to uninstall the comprehensive
structure group. Then, with the completed preparation, FM workers can finish the work with
high efficiency. And finally, they can restore the disassembled structure. In this part, AR-FM can
help direct the workers for some of the work, like they can operate the visual model and know
the steps of uninstalling the whole model group by checking the animation in AR-FM.

6.4.4 Giving Feedback to System
When the work is finished, the information of the new component would be renewed in
the BAS, Revit, and Unity 3D system. Currently the AR-FM cannot save the edited file so the
information of the model needs to be renewed in the Unity 3D system. But for future work, the
automatic data transmission will be accomplished. And in the system, comments will be added
into the components to show that they were changed. The work completed statement should be
published and FM department will hold a meeting to discuss the experience in this work.
However, in this part, AR-FM can hardly give the support, this is the main aims for AR-FM to
be developed in the future.
6.5 Conclusion
A BIM-based augmented reality software model was created, enabling single-item data
transfer from BIM to AR. The user can simply position the device's camera at a pre-set initial
point and hit the start button to overlay the virtual model on the real element. At the same time, it
is applied and tested in the actual case analysis (Figure 6.12). The results show that all the
functions added in AR-FM can run normally in software, realize a variety of functions, and
realize the whole process of equipment management. The most beneficial function in the entire
182

AR-FM to help the FM process is to display the tools needed for the entire operation process in
the form of text when playing the animation of the dismantling inside the wall. However, this is
not automatically done within the software and would be a major future goal. This is the most
intuitive help for the FM manager. It gives all the information in the preparation stage and the
operation stage, which greatly improves the efficiency of the entire FM process.

Figure 6. 12 Using AR-FM in case study
With AR-FM, users can clearly locate important components. Even if the components are
inside the wall, just click on the marked wall to display the internal components. At the same
time, functions such as rotation, movement, and size change after selecting components can also
help users to see the structure of the components more clearly from all angles.
183

6.6 Summary
In this chapter, the research background, methodology, limitations, impact, and future
work were discussed. For now, AR-FM can realize some visual functions including aligning the
AR model with the real world, view components, view an animation, and guide users for FM
work.

184

REFERENCES
Aruanno, B., & Garzotto, F. (2019). MemHolo: Mixed reality experiences for subjects with
alzheimer’s disease. Multimedia Tools and Applications, 78(10), 13517-13537.
Aryani, A. L., Brahim, J., & Fathi, M. S. (2014). The development of building information
modeling (BIM) definition. Applied Mechanics and Materials, 567, 625-630.
doi:10.4028/www.scientific.net/AMM.567.625
Azuma, R. T. (1997). A survey of augmented reality. Presence: Teleoperators & Virtual
Environments, 6(4), 355-385.
Bargstädt, H. (2015). Challenges of BIM for construction site operations. Procedia
Engineering, 117, 52-59.
Berryman, D. R. (2012). Augmented reality: A review. Medical Reference Services
Quarterly, 31(2), 212-218.
Cai, S., Liu, E., Yang, Y., & Liang, J. (2019). Tablet‐based AR technology: Impacts on students’
conceptions and approaches to learning mathematics according to their self‐efficacy. British
Journal of Educational Technology, 50(1), 248-263. doi:10.1111/bjet.12718
Chen, J. Y., & Fragomeni, G. (2018). Virtual, augmented and mixed reality: Interaction,
navigation, visualization, embodiment, and simulation: 10th international conference,
VAMR 2018, held as part of HCI international 2018, las vegas, NV, USA, july 15-20, 2018,
proceedings, part I Springer.
Chen, K., Chen, W., Li, C. T., & Cheng, J. C. P. (2019). A BIM-based location aware AR
collaborative framework for facility maintenance management. Journal of Information
Technology in Construction, 24, 360-380. doi: 10.36680/j.itcon.2019.019
Chen, K., & Tsai, M. (2021). Conversation‐based information delivery method for facility
management. Sensors (Basel, Switzerland), 21(14), 4771. doi:10.3390/s21144771
Chi, H., Kang, S., & Wang, X. (2013). Research trends and opportunities of augmented reality
applications in architecture, engineering, and construction. Automation in Construction, 33,
116-122.
Coates, P., Arayici, Y., Koskela, L., Kagioglou, M., Usher, C., & O'Reilly, K. (2010). The
limitations of BIM in the architectural process.
Coogan, C. G., & He, B. (2018). Brain-computer interface control in a virtual reality
environment and applications for the internet of things. IEEE Access, 6, 10840-10849.
Danker, F., & Jones, O. (2014). Combining augmented reality and building information
modelling-an industry perspective on applications and future directions.
Dey, A., Billinghurst, M., Lindeman, R. W., & Swan, J. (2018). A systematic review of 10 years
of augmented reality usability studies: 2005 to 2014. Frontiers in Robotics and AI, 5, 37.
185

Diao, P., & Shih, N. (2019). BIM-based AR maintenance system (BARMS) as an intelligent
instruction platform for complex plumbing facilities. Applied Sciences, 9(8), 1592.
doi:10.3390/app9081592
Eastman, C. M., Eastman, C., Teicholz, P., Sacks, R., & Liston, K. (2011). BIM handbook: A
guide to building information modeling for owners, managers, designers, engineers and
contractors John Wiley & Sons.
Gerstweiler, G., Vonach, E., & Kaufmann, H. (2016). HyMoTrack: A mobile AR navigation
system for complex indoor environments. Sensors, 16(1), 17.
Gheisari, M., Foroughi Sabzevar, M., Chen, P., & Irizzary, J. (2016). Integrating BIM and
panorama to create a semi-augmented-reality experience of a construction site. International
Journal of Construction Education and Research, 12(4), 303-316.
doi:10.1080/15578771.2016.1240117
Grubert, J., & Grasset, R. (2013). Augmented reality for android application development (1st
ed.). Birmingham, U.K: Packt Pub.
Guo, M., & Hsiao, T. (2019). The research of VR and AR technology on cultural
communication IEEE. doi:10.1109/ICACEH48424.2019.9041932
Harborth, D. (2017). Augmented reality in information systems research: A systematic literature
review.
Honic, M., Kovacic, I., Sibenik, G., & Rechberger, H. (2019). Data-and stakeholder management
framework for the implementation of BIM-based material passports. Journal of Building
Engineering, 23, 341-350.
Huang, Z. (2020). Application research on BIM+AR technology in construction safety
management IOP Publishing. doi:10.1088/1742-6596/1648/4/042019
Hyman, P. (2012). In honor of alan turing. Communications of the ACM, 55(9), 20-23.
Jahn, G., Newnham, C., & Beanland, M. (2018). Making in mixed reality. holographic design,
fabrication, assembly and analysis of woven steel structures.
Jiao, Y., Zhang, S., Li, Y., Wang, Y., & Yang, B. (2013). Towards cloud augmented reality for
construction application by BIM and SNS integration. Automation in Construction, 33, 37-
47.
Kensek, K. (2015). BIM guidelines inform facilities management databases: A case study over
time. Buildings, 5(3), 899-916.
Kim, S. L., Suk, H. J., Kang, J. H., Jung, J. M., Laine, T. H., & Westlin, J. (2014). Using unity
3D to facilitate mobile augmented reality game development. Paper presented at the 2014
IEEE World Forum on Internet of Things (WF-IoT), pp. 21-26.
Kuula, T., Piira, K., Seisto, A., Hakkarainen, M., & Woodward, C. (2012). User requirements for
mobile AR and BIM utilization in building life cycle management. Paper presented at
the Proceedings of 12th International Conference on Construction Applications of Virtual
Reality (CONVR 2012), pp. 203-211.
186

Li, X., Yi, W., Chi, H., Wang, X., & Chan, A. P. (2018a). A critical review of virtual and
augmented reality (VR/AR) applications in construction safety. Automation in
Construction, 86, 150-162.
Li, X., Yi, W., Chi, H., Wang, X., & Chan, A. P. (2018b). A critical review of virtual and
augmented reality (VR/AR) applications in construction safety. Automation in
Construction, 86, 150-162.
Mirshokraei, M., De Gaetani, C. I., & Migliaccio, F. (2019a). A web-based BIM–AR quality
management system for structural elements. Applied Sciences, 9(19), 3984.
Mirshokraei, M., De Gaetani, C. I., & Migliaccio, F. (2019b). A web-based BIM–AR quality
management system for structural elements. Applied Sciences, 9(19), 3984.
Mirshokraei, M., De Gaetani, C. I., & Migliaccio, F. (2019c). A web-based BIM-AR quality
management system for structural elements. Applied Sciences, 9(19), 3984.
doi:10.3390/app9193984
Park, C., Lee, D., Kwon, O., & Wang, X. (2013). A framework for proactive construction defect
management using BIM, augmented reality and ontology-based data collection
template. Automation in Construction, 33, 61-71. doi: 10.1016/j.autcon.2012.09.010
Reddy, K. P. (2011). BIM for building owners and developers : Making a business case for using
BIM on projects. Hoboken, N.J: John Wiley and Sons.
Sacks, R., Koskela, L., Dave, B. A., & Owen, R. (2010). Interaction of lean and building
information modeling in construction. Journal of Construction Engineering and
Management, 136(9), 968-980.
Setti, A., Bosetti, P., & Ragni, M. (2016). ARTool—Augmented reality human-machine
interface for machining setup and maintenance. Paper presented at the Proceedings of SAI
Intelligent Systems Conference, pp. 131-155.
Shalabi, F., & Turkan, Y. (2017). IFC BIM-based facility management approach to optimize data
collection for corrective maintenance. Journal of Performance of Constructed
Facilities, 31(1), 04016081.
Shirowzhan, S., Sepasgozar, S. M., Edwards, D. J., Li, H., & Wang, C. (2020). BIM
compatibility and its differentiation with interoperability challenges as an innovation
factor. Automation in Construction, 112, 103086.
Tulubas Gokuc, Y., & Arditi, D. (2017). Adoption of BIM in architectural design
firms. Architectural Science Review, 60(6), 483-492. doi:10.1080/00038628.2017.1383228
Wang, K., & Hu, Z. (2009). Application study of virtual reality in real estate industry. Paper
presented at the Proceedings of the 3rd WSEAS International Conference on COMPUTER
ENGINEERING and APPLICATIONS, pp. 247-251.
Wang, X., & Love, P. E. (2012). BIM + AR: Onsite information sharing and communication via
advanced visualization IEEE. doi:10.1109/CSCWD.2012.6221920
187

Wang, X., Truijens, M., Hou, L., Wang, Y., & Zhou, Y. (2014a). Integrating augmented reality
with building information modeling: Onsite construction process controlling for liquefied
natural gas industry. Automation in Construction, 40, 96-105.
Wang, X., Truijens, M., Hou, L., Wang, Y., & Zhou, Y. (2014b). Integrating augmented reality
with building information modeling: Onsite construction process controlling for liquefied
natural gas industry. Automation in Construction, 40, 96-105.
Xu, J., Lu, W., Xue, F., & Chen, K. (2019). ‘Cognitive facility management’: Definition, system
architecture, and example scenario. Automation in Construction, 107, 102922.
Yam, R., Tse, P. W., Li, L., & Tu, P. (2001). Intelligent predictive decision support system for
condition-based maintenance. The International Journal of Advanced Manufacturing
Technology, 17(5), 383-391.
Zhang, X. (2015). Green real estate development in china: State of art and prospect agenda—A
review. Renewable and Sustainable Energy Reviews, 47, 1-13.
Zima, K., Plebankiewicz, E., & Wieczorek, D. (2020). A SWOT analysis of the use of BIM
technology in the polish construction industry. Buildings, 10(1), 16.
Meža, S., Turk, Ž, & Dolenc, M. (2014). Component based engineering of a mobile BIM-based
augmented reality system. Automation in Construction, 42, 1-12.
Quincas Moreira, (2022) How Augmented Reality Improves Your Construction Projects.
https://www.youtube.com/watch?v=miZpOfxZrsc
Höbart, K. (2020). No title. AR-Schulungs-Anwendung Und-Editor Für 3D-BIM-Visualisierung
Im Bauingenieurwesen
Piekarski, W., & Thomas, B. (2002). ARQuake: The outdoor augmented reality gaming
system. Communications of the ACM, 45(1), 36-38.
Reardon, C., Lee, K., Rogers, J. G., & Fink, J. (2019). Augmented reality for human-robot
teaming in field environments. Paper presented at the International Conference on Human-
Computer Interaction, pp. 79-92.
Arth, C., Grasset, R., Gruber, L., Langlotz, T., Mulloni, A., & Wagner, D. (2015). The history of
mobile augmented reality. arXiv Preprint arXiv:1505.01319,
Zlatanova, S. (2002). Augmented reality technology. GISt Report no.17, Delft, 2002, 72
Abdelaziz, M. (2021). Virtual and augmented reality in construction field.
Linowes, J. (2015). Unity virtual reality projects Packt Publishing Ltd.
Theoktisto, V. (2015). Enhancing detailed haptic relief for real-time interaction.
Seeley, I. H. (1987). Building maintenance Macmillan International Higher Education.
Göçer, Ö, Hua, Y., & Göçer, K. (2015). Completing the missing link in building design process:
Enhancing post-occupancy evaluation method for effective feedback for building
performance. Building and Environment, 89, 14-27.
188

Porter, M. E., & Heppelmann, J. E. (2017). Why every organization needs an augmented reality
strategy. Hbr’s 10 Must, 85
Patacas, J., Dawood, N., Vukovic, V., & Kassem, M. (2015). BIM for facilities management:
Evaluating BIM standards in asset register creation and service life planning. Journal of
Information Technology in Construction, 20(10), 313-318.
Stowe, K., Zhang, S., Teizer, J., & Jaselskis, E. J. (2015). Capturing the return on investment of
all-in building information modeling: Structured approach. Practice Periodical on Structural
Design and Construction, 20(1), 04014027.
Chi, H., Kang, S., & Wang, X. (2013). Research trends and opportunities of augmented reality
applications in architecture, engineering, and construction. Automation in Construction, 33, 116-
122.
Calori, C., & Vanden-Eynden, D. (2015). Signage and wayfinding design: A complete guide to
creating environmental graphic design systems John Wiley & Sons.
Xiang, Y., Mottaghi, R., & Savarese, S. (2014). Beyond pascal: A benchmark for 3d object
detection in the wild. Paper presented at the IEEE Winter Conference on Applications of
Computer Vision, pp. 75-82.
Aarni, 2020. Retrieved September 24, 2021, from https://aec-business.com/fologram-and-all-
brick-introduce-an-entirely-new-way-of-building-in-ar/)
Posada, J., Toro, C., Barandiaran, I., Oyarzun, D., Stricker, D., De Amicis, R., et al. (2015).
Visual computing as a key enabling technology for industrie 4.0 and industrial internet. IEEE
Computer Graphics and Applications, 35(2), 26-40.
Azhar, S. (2011). Building information modeling (BIM): Trends, benefits, risks, and challenges
for the AEC industry. Leadership and Management in Engineering, 11(3), 241-252.
doi:10.1061/(ASCE)LM.1943-5630.0000127

Becerik-Gerber, B., Jazizadeh, F., Li, N., & Calis, G. (2012). Application areas and data
requirements for BIM-enabled facilities management. Journal of Construction Engineering and
Management, 138(3), 431-442.

Abstract (if available)

Abstract Building information modeling (BIM) is critical in the architecture, engineering, and construction industry, especially for 3D modeling and the addition of geometric and non-geometric elements. Virtual reality (VR) and, more recently, augmented reality (AR) technology are being developed from a specialized computer technology software to applications used in many areas. Based on the contributions and challenges discussed in many research papers in recent years, it is apparent that the integration of BIM and AR elements for positioning wall components in existing buildings is feasible and can facilitate the maintenance and repair of components in the future. A workflow and user interface were developed to track a Revit model in a Unity workspace for an augmented reality overlay.
The model building chosen was the third floor of School of Architecture office building in University of Southern California. Revit, 3ds Max, Unity 3D, and other software were used as experiments to model the elements in the wall based on the AR foundation and matching existing models with real-world architecture. Different from the tracking location method in other research, a start location was created to make it easier to match the digital with the real world. Then, users can click to choose the wall, the floor or the roof and hide it to show the inside components. And the components can also be moved by the button on device to realize the human-computer interaction. In addition to AR imaging, a demonstration animation was added to show the optimal disassembly method of a complex structure group or complex components in the wall. As a result, AR-FM app realized the transferring of BIM model to Unity 3D and visualize the MEP system and architecture system. In future, more function will be added to make the tool more comprehensive and help the work process during facility management.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Using building information modeling with augmented reality: visualizing and editing MEP systems with a mobile augmented reality application

PDF

Building information modeling based design review and facility management: Virtual reality workflows and augmented reality experiment for healthcare project

PDF

Streamlining sustainable design in building information modeling: BIM-based PV design and analysis tools

PDF

A BIM-based visualization tool for facilities management: fault detection through integrating real-time sensor data into BIM

PDF

Data visualization in VR/AR: static data analysis in buildings

PDF

Automating fire code compliance using BIM and Revit plug-ins

PDF

Building bridges: filling gaps between BIM and other tools in mechanical design

PDF

Visualizing thermal data in a building information model

PDF

Building information modeling: guidelines for project execution plan (PxP) for India

PDF

Streamlining precast back-frame design: automated design using Revit plugin

PDF

CFD visualization: a case study for using a building information modeling with virtual reality

PDF

Office floor plans generation based on Generative Adversarial Network

PDF

Multi-domain assessment of a kinetic facade: determining the control strategy of a kinetic façade using BIM based on energy performance, daylighting, and occupants’ preferences; Multi-domain asse...

PDF

Planning in advance for rehabilitation and restoration using BIM and seismic simulation to record and analyze the Japanese House in Los Angeles

PDF

MM Electrical Tool: a tool for generating electrical single line diagrams in BIM

PDF

Visualizing architectural lighting: creating and reviewing workflows based on virtual reality platforms

PDF

Interoperability between building information models (BIM) and energy analysis programs

PDF

Quantify human experience: integrating virtual reality, biometric sensors, and machine learning

PDF

Carbon accounting tool (CAT) in BIM: an embodied carbon plug-in for revit

PDF

Bridging the gap: a tool to support bim data transparency for interoperability with building energy performance software

Asset Metadata

Creator Zheng, Ruisong (author)

Core Title BIM+AR in architecture: a building maintenance application for a smart phone

School School of Architecture

Degree Master of Building Science

Degree Program Building Science

Degree Conferral Date 2022-05

Publication Date 04/27/2022

Defense Date 03/09/2022

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag animation,augmented reality (AR),building information modeling (BIM),facility management (FM),OAI-PMH Harvest

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Kensek, Karen (committee chair), Choi, Joon-Ho (committee member), Schiler, Marc (committee member)

Creator Email a15953115677@gmail.com,ruisongz@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC111117822

Unique identifier UC111117822

Document Type Thesis

Format application/pdf (imt)

Rights Zheng, Ruisong

Type texts

Source 20220427-usctheses-batch-933 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu