University of Southern California Dissertations and Theses

Impact of virtual reality (VR)-based training on construction robotics remote-operation

(USC Thesis Other)

Impact of virtual reality (VR)-based training on construction robotics remote-operation

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content

Impact of Virtual Reality (VR)-based Training on Construction Robotics Remote-Operation

by

Pooya Adami

A Dissertation Presented to the
FACULTY OF THE USC VITERBI SCHOOL OF ENGINEERING
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
CIVIL ENGINEERING

May 2023

Copyright 2023 Pooya Adami

ii

Dedication
To my parents, sister, and close friends, who have been my unwavering pillars of support
throughout my Ph.D. journey, this thesis is dedicated with heartfelt gratitude.

iii

Acknowledgements
This material is supported by the National Science Foundation under Grant No. 1822724 and the
U.S. Army Research Office under Grant No. W911NF2020053. I am very thankful to my Ph.D.
advisors, Dr. Burcin Becerik-Gerber and Dr. Lucio Soibelman, for their continuous support and
guidance. Also, special thanks to the research team that supported and contributed to the studies,
Dr. Yasemin Copur-Gencturk, Dr. Gale Lucas, Dr. Peter Woods, Dr. Rashmi Singh, and Dr.
Tenzin Doleck. I would also like to thank Dr. Najmedin Meshkati for his guidance and constructive
comments on this thesis. I am also thankful to iLab members, especially Patrick Rodrigues,
Mohamad Awada, Dr. Runhe Zhu, and Dr. Ashrant Aryal, who partially supported the research
presented in this thesis. Finally, I am very thankful to my family and close friends for their support
and kind helps all these years. The help of Mike Martin, Michael Peschka, and the Brokk company
throughout this research study is greatly appreciated. Any opinions, findings, conclusions, or
recommendations expressed in this material are those of the authors and do not necessarily reflect
the views of NSF or the University of Southern California, and no official endorsement should be
inferred.

iv

Table of Contents
Dedication ....................................................................................................................................... ii
Acknowledgements ........................................................................................................................ iii
Table of Contents ........................................................................................................................... iv
List of Tables ................................................................................................................................ vii
List of Figures .............................................................................................................................. viii
Abstract .......................................................................................................................................... vi
Chapter 1. Problem Definition and Motivation .............................................................................. 1
Chapter 2. Literature Review and Research Gaps .......................................................................... 4
2.1 Virtual Reality-based Training ............................................................................................. 4
2.2 Human-Robot Interaction ..................................................................................................... 6
2.2.1 Trust in the Robot and Robot Operation Self-efficacy .................................................. 6
2.2.2 Mental Workload ........................................................................................................... 7
2.2.3 Situational Awareness .................................................................................................... 8
Chapter 3. Research Objectives and Questions ............................................................................ 10
3.1 Research Objective 1 .......................................................................................................... 10
3.2 Research Objective 2 .......................................................................................................... 10
3.3 Research Objective 3 .......................................................................................................... 10
Chapter 4. An Immersive Virtual Reality (VR)-based Training for Construction Robotics
Remote-Operation ......................................................................................................................... 11
4.1 Background ......................................................................................................................... 11
4.1.1 Existing VR-based Training Programs ........................................................................ 11
4.1.1.1 Visualization Tools and Controllers .................................................................... 11
4.1.1.2 Navigation Methods ............................................................................................. 11
4.1.1.3 Learning Scenarios............................................................................................... 12
4.1.1.4 Virtual Environments ........................................................................................... 12
4.1.2 Adult Learning Theory ................................................................................................ 12
4.2 Use-case Robot ................................................................................................................... 13
4.3 System Setup and Configuration ........................................................................................ 14
4.4 Learning Modules ............................................................................................................... 15
Chapter 5. Effectiveness of VR-based Training on Improving Construction Workers’
Knowledge, Skills, and Safety behavior in Robotic Remote-operation ....................................... 21

v

5.1 Methodology ....................................................................................................................... 21
5.1.1 In-person Training ....................................................................................................... 21
5.1.2 Participants ................................................................................................................... 22
5.1.3 Experimental Procedure ............................................................................................... 23
5.1.4 Analysis........................................................................................................................ 25
5.1.4.1 Quantitative Analysis ................................................................................................ 25
5.1.4.2 Qualitative Analysis .................................................................................................. 26
5.2 Results ................................................................................................................................. 26
5.2.1 Quantitative analysis: VR-based training’s impact on workers’ knowledge, operational
skills, and safety behaviors ................................................................................................... 26
5.2.1 Qualitative analysis: Observations on workers’ performance ..................................... 31
5.3 Discussion ........................................................................................................................... 32
5.3.1 Knowledge Acquisition ............................................................................................... 32
5.3.2 Operational skills & safety behavior............................................................................ 33
5.4. Limitations ......................................................................................................................... 35
5.5. Conclusion ......................................................................................................................... 35
Chapter 6. Impact of VR-Based Training on Human–Robot Interaction for Remote Operating
Construction Robots...................................................................................................................... 37
6.1 Methodology ....................................................................................................................... 37
6.1.1 Experimental Procedure ............................................................................................... 37
6.1.2 Analysis............................................................................................................................ 38
6.2 Results ................................................................................................................................. 39
6.3 Discussion ........................................................................................................................... 45
6.3.1 Trust in the Robot and Robot Operation Self-Efficacy ............................................... 46
6.3.2 Situational Awareness and Mental Workload.............................................................. 47
6.4. Limitations ......................................................................................................................... 47
6.5. Conclusion ......................................................................................................................... 48
Chapter 7. Participants Matter: Effectiveness of VR-based Training on the Knowledge, Trust in
the Robot, and Self-Efficacy of Construction Workers and University Students......................... 50
7.1 Methodology ....................................................................................................................... 50
7.1.1 Participants ................................................................................................................... 50
7.1.2 Study Procedure ........................................................................................................... 51
7.1.3 Analysis........................................................................................................................ 52

vi

7.2 Results ................................................................................................................................. 52
7.3 Discussion ........................................................................................................................... 55
4.3.1 Knowledge Acquisition ............................................................................................... 55
7.3.2 Trust in the robot & self-efficacy................................................................................. 56
7.4. Limitations ......................................................................................................................... 57
7.5. Conclusion ......................................................................................................................... 57
Chapter 8. Limitations and Future Work ...................................................................................... 59
Chapter 9. Conclusion ................................................................................................................... 60
Publications to Date ...................................................................................................................... 62
Peer-Reviewed Journal Papers (Published) .............................................................................. 62
References ..................................................................................................................................... 63
Appendices .................................................................................................................................... 73
Appendix A. Knowledge Assessment....................................................................................... 73
Appendix B. Trust in the robot and robot operation self-efficacy survey ................................ 86
Appendix C. Performance Assessment ..................................................................................... 87
Appendix D. Situational Awareness Survey ............................................................................. 89
Appendix E. Mental Workload Survey ..................................................................................... 90
Appendix F. System Usability Survey (SUS) ........................................................................... 91

vii

List of Tables
Table 1. Targeted operational skills and safety behavior in each learning module ……….…… 18
Table 2. Demographics of participants based on training types ……….…….………....……… 23
Table 3. Means and standard deviations of knowledge assessment based on individual
Differences .…….…....……..…….………………………………………………….……...….. 26
Table 4. Means and standard deviations of safety behavior assessment based on
Individual differences….………....……………………………………………...……….…….. 30
Table 5. Means and standard deviations of operational skills assessment based on individual
differences ……….…….………....…………….…………………………………………...….. 32
Table 6. Themes, examples, and numbers of participants who got positive and negative
comments based on their training type in the qualitative analysis ………….….………....…… 39
Table 7. Background Indicators of Workers in the Two Conditions …………….………....…. 40
Table 8. Means and standard deviations (SD) of trust in the robot based on individual
differences ……….…….………….…………………………...…………………………....…. 42
Table 9. Means and standard deviations of robot operation self-efficacy based on individual
differences ……….…….………....…………….…………………………………………...….. 44
Table 10. Means and standard deviations (SD) of SA assessment based on individual
differences ……….…….………….………………………………………………………….... 50
Table 11. Means and standard deviations (SD) of MWL assessment based on individual
differences ……….…….………....……………………………………………………....…….. 53

viii

List of Figures
Fig. 1. The actual and the simulated Brok110 ……………………………………..…….……. 14
Fig. 2. VR-based training system setup …………………….…….………....………….….….. 15
Fig. 3. VR-based training learning modules ……….…….………....…………….….…….….. 19
Fig. 4. Workers during an in-person training session ……….……...……....…………...…….. 22
Fig. 5. Performance evaluation ……….…….………....…………….……………….………... 25
Fig. 6. Workers’ average score on the knowledge assessment ……….…….………...……..… 28
Fig. 7. Mean rating of safety behavior assessment ……….…….………....…………..………. 29
Fig. 8. Mean rating of operational skills assessment ……….…….………....……….….…….. 31
Fig. 9. Participants’ average score on the knowledge assessment ……….……..…….…......… 53
Fig. 10. Participants’ average score on the trust in the robot survey ……….…….………….... 54
Fig. 11. Participants’ average score on the robot operation self-efficacy survey …….….……. 55

vi

Abstract
Despite the growing interest in automation and the widespread use of robots in the construction
industry, the dynamic and unstructured nature of construction sites requires construction workers
to acquire new skills and upgrade their existing skills to prepare for the future of work. However,
there are limited opportunities for construction workers to practice and develop skills such as trust,
self-efficacy, situational awareness, and resistance to increasing mental workload in working with
robots. Virtual Reality (VR)-based training has emerged as a viable and cost-effective method that
offers a safe way for workers to become familiar with hazardous tasks without exposing them to
actual safety risks associated with traditional training methods such as hands-on training, lecture-
based training, and apprenticeship programs. Given these challenges, this proposal aims to define
the research objectives as follows.
1. To understand the impact of VR-based training on construction workers’ knowledge
acquisition, operational skills, and safety behavior compared to the traditional in-person
training method,
2. To understand the impact of VR-based training on human-related factors in Human-Robot
Interaction (HRI), such as trust in the robot, robot operation self-efficacy, situational
awareness, and mental workload during robotic remote operation compared to the traditional
in-person training method,
3. To empirically compare the effect of VR-based training in construction workers vs. graduate
construction engineering students on knowledge acquisition, trust in the robot, and robot
operation self-efficacy.
The structure of this proposal is organized as follows: chapter 1 provides the broad background
and motivation for the research efforts. Chapter 2 presents a thorough literature review of the
proposal scope, and research gaps are identified. Research objectives and questions are described
in chapter 3.
Chapter 4 presents the Virtual Reality (VR)-based training studied throughout this dissertation.
This training simulates the remote operation of a construction robot in order to support training in
worker-robot teamwork on construction sites. The VR-based training presented in this chapter
utilizes a remote-operated demolition robot named Brokk. This chapter describes the use-case
robot (Brokk), system setup and configuration, and the learning scenarios. In this training, the
trainee learns about the robot's various components, safety management, the robot's control box
functions, and demolition instructions used during common demolition tasks. Significantly, this
training is developed based on adult learning theories in general—and andragogy principles in
particular. In addition, VR-based training has been augmented with useful features from existing
VR-based training programs. The impact of VR-based training on construction workers will be
investigated in subsequent chapters.
To address research objective 1, chapter 5 presents the first study in this proposal. The VR-based
training program was developed for a remote-operated Brokk demolition robot. This VR-based
training provides immersive, interactive, scenario-based training that promotes “learning by
doing” through utilizing adult learning theories within dynamic construction work environments
that include completion of multiple tasks, engagement with virtual workers who share the same

vii

space, exposure to varied environmental conditions (e.g., uneven terrain, dust, rain), and
experience of both positive and negative consequences of operational actions. Moreover, the actual
controller of the robot is programmed and connected to VR-based training, which provides a tactile
and memory-based opportunity to use the same levers and buttons to move, position, and operate
the equipment in VR as in real settings. Fifty construction workers were randomly assigned to
complete either VR-based or in-person training for remote-operating a demolition robot.
Quantitative and qualitative data analyses have been used to answer our research questions. Our
results indicate that VR-based training was associated with a significant increase in knowledge,
operational skills, and safety behavior compared to in-person training. Our findings suggest that
VR-based training provides a viable and effective option for future training pro- grams and a
valuable option for construction robotics safety and skill training.
The second study of this proposal presented in chapter 6, which used the same sample of
construction workers and the same VR-based training, strived to understand the impact of VR-
based training on human-related factors in Human-Robot Interaction (HRI) such as trust in the
robot, robot operation self-efficacy, situational awareness, and mental workload during robotic
remote-operation compared to the traditional in-person training method. Improving HRI can
increase the adoption of robots on construction sites; for example, increasing trust in robots could
help construction workers to accept new technologies. Confidence in operation (or self-efficacy),
mental workload, and situational awareness are among other key factors that help such workers
remote operate robots safely. However, construction workers have few opportunities to practice
with robots to build trust, self-efficacy, situational awareness, and resistance to increasing mental
workload before interacting with them on job sites. Virtual reality (VR) could afford a safer place
to practice with the robot; thus, it is tested if VR-based training could improve these four outcomes
during the remote operation of construction robots. In an experimental study where construction
workers remote-operated a demolition robot, trust in the robot, self-efficacy, mental workload, and
situational awareness are measured. Fifty workers were randomly assigned to either VR-based or
traditional in-person training led by an expert trainer. Results show that VR-based training
significantly increased trust in the robot, self-efficacy, and situational awareness, compared to
traditional in-person training. Our findings suggest that VR-based training can significantly
increase beneficial cognitive factors over more traditional methods and has substantial
implications for improving HRI using VR, especially in the construction industry.
Although researchers have developed VR-based training for construction workers, some have
recruited students rather than workers to understand the impact of their VR-based training.
However, students are different from construction workers in many ways, which can threaten the
validity of such studies. Therefore, our third study in chapter 7 aimed to investigate the extent to
which the findings of a VR-based training study are contingent on whether students or construction
workers were used as the study sample. This chapter strives to compare the impact of VR-based
training on university students’ and construction workers’ knowledge acquisition, trust in the
robot, and robot operation self-efficacy in the remote operation of a construction robot. Twenty-
five construction workers and twenty-five graduate construction engineering students were
recruited to complete VR-based training for remote operating a demolition robot. Quantitative
analyses have been used to answer our research questions. Our study shows that the results depend
on the target sample in that students gained more knowledge, whereas construction workers gained
more trust in the robot and more self-efficacy in robot operation. These findings suggest that the
effectiveness of VR-based training on students may not necessarily associate with its effectiveness

viii

on construction workers. Chapter 8 discusses the limitations of the study, including areas where
further research is needed. Additionally, it identifies potential directions for future research that
could build upon the findings of this study. In chapter 9, the conclusions drawn from the study are
presented, summarizing the key findings and their implications for the research field. Finally,
chapter 10 acknowledges the contributions and support of those who have helped in the completion
of this PhD thesis.

1

Chapter 1. Problem Definition and Motivation
The construction industry is a significant contributor to the global economy, representing 6% of
the world's GDP [1]. However, it faces significant challenges, including safety issues, a shortage
of skilled labor, and low productivity rates. In particular, the construction industry has the highest
number of fatal and non-fatal injuries compared to other industries [2]. According to the U.S.
Bureau of Labor Statistics, in 2019, the U.S. construction industry had 1,061 occupational
fatalities, which was the highest number among all industries [3]. Additionally, the construction
industry has a significantly higher rate of non-fatal injuries compared to other industries, and
approximately half of the workplace injuries go unreported each year, according to the
Occupational Safety and Health Administration (OSHA) [4]. The construction industry also
suffers from severe labor shortages, particularly in skilled labor, with 266,000 unfilled jobs in the
U.S. reported in February 2021 by the U.S. Bureau of Labor Statistics [5]. Moreover, the
construction industry's labor productivity growth has been lower than that of manufacturing and
the entire economy, with a yearly increase of only 1% between 1995 and 2014, compared to 2.7%
for the total economy and 3.6% for manufacturing on average [6].
The construction industry has shown a growing interest in robotics and automation as a solution
to address safety concerns, labor shortages, and stagnant productivity. Over the last two decades,
the scientific community and industry professionals have focused on developing and deploying
robots to construction sites [7], leading to increased production of scientific research and the
anticipated deployment of over 7000 new construction robots to sites by 2025 [8]. Automation and
robotics have the potential to address the shortcomings of the construction industry by improving
productivity rates and enhancing safety [9]. On-site robotic systems can perform repetitive and
tedious tasks, allowing human workers to concentrate on more complex tasks [10]. Robots can
also help reduce project costs by enabling construction in adverse weather conditions and
mitigating labor shortages [11]. Additionally, automation and robotics can increase workforce
diversity by enabling underrepresented groups to join the workforce, such as women and disabled
workers who may be unable to perform heavy tasks [2]. Construction robots can execute hazardous
and labor-intensive tasks, such as demolition, and improve the industry's dangerous work climate
by preventing injuries and fatalities [12].
Successful adoption of construction robotics necessitates exploring the Human-Robot Interaction
(HRI) field. New automation technologies may not be welcomed by construction workers who
view them as a potential threat to their jobs [13]. Furthermore, due to construction sites' constantly
changing and unpredictable nature, workers may prefer traditional methods and may not trust
robots [14]. To build trust between workers and new robotic systems, workers must feel safe
around robots, accept robot-provided information or decisions, and be willing to work with robots
in the future [15, 16]. However, workers have limited opportunities to build trust with robots before
remotely operating them on job sites. While robotics can increase safety on construction sites by
removing workers from hazardous environments, interacting with new technologies can pose new
safety risks [17]. Unlike in other industries where robots and workers are separated, robots operate
alongside construction workers in constantly changing environments. Therefore, ensuring the
safety of humans working alongside robots is essential for successfully adopting construction
robotics. Workers’ safety behavior, mental workload (MWL), and situational awareness (SA) are

2

crucial factors that impact the safe remote operation of construction robots. MWL and SA are
critical objects of interest in cognitive engineering as they refer to the cognitive loads imposed on
operators during task execution when robots and other intelligent systems are involved. MWL
relates to the cognitive capacity necessary for a given task [18], while SA indicates how operators
perceive and comprehend the task and its environment [19]. Despite their significance in ensuring
the safe remote operation of construction robots, workers have limited opportunities to optimize
their MWL and build SA before remotely operating robots on-site.
In order to ensure the safe and effective use of new technologies, it is crucial to provide
construction workers with training in upskilling (new skills directly related to their current roles)
or reskilling (new skills necessary for different roles) to interact with and control these
technologies. The McKinsey Global Institute emphasizes the importance of upskilling and
reskilling the workforce to increase productivity and prepare for the future of work in the
construction industry [6, 9]. A range of training strategies has been proposed and implemented,
including passive strategies (such as lectures, pamphlets, presentations, and videos) and active
strategies (such as computer-based or learner-centered instruction, apprenticeship models, and
hands-on demonstrations). Research has shown that more engaging training strategies can improve
worker behavioral performance in safety and health, as well as increase knowledge acquisition and
reduce accidents and injuries [20]. However, some construction tasks may not be suitable for more
engaging training strategies due to technical, economic, safety, and ethical constraints. For
example, simulating real-world scenarios during training can be difficult due to construction sites'
dynamic, unconstrained, and hazardous nature, and the costs associated with acquiring the
necessary equipment and materials for training may be too high. Additionally, training on-site may
create disturbances to the job site that cannot be justified.
The use of Virtual Reality (VR)-based training is being proposed as a solution to address concerns
about providing construction workers with in-person training experiences in hazardous situations
without exposing them to actual safety risks. VR-based training has already been applied in various
construction-related areas, including construction safety [21–24], ergonomic behavior [25,26],
operating construction equipment [27–30], and performing construction tasks [31,32]. In the field
of HRI, VR-based training can be used to build trust in automation and construction robots, as well
as promote safer interaction between humans and robots by reducing mental workload and
increasing SA. However, there is limited research on whether VR-based training can effectively
provide construction workers with the necessary knowledge, skills, safety behavior, trust, self-
efficacy, and SA in construction robotics operations. The development of effective methods for
training workers to engage with robotics in a VR environment is complex, as it requires accurate
simulation of robots with complex control mechanisms and interactions among various agents.
Additionally, it is challenging to evaluate skill transfer from VR-based training to real-world
settings [33]. Considering the unique demographics of construction workers, such as potentially
low English proficiency and low literacy, developing effective VR-based training is even more
challenging [34]. It is essential to investigate the impact of VR-based training on various aspects,
such as knowledge acquisition, operational skills, safety behavior, trust in the robot, self-efficacy,
SA, and MWL, compared to a more traditional, comparable in-person pedagogical model,
particularly for ethnic minorities such as Hispanic/Latino construction workers who may have
lower education levels and language barriers.
Through empirical research, numerous VR-based training programs have been developed and
compared to traditional training methods such as lecture-based, video-based, and hands-on

3

training. The majority of these studies have focused on safety training, particularly hazard
identification [21-24], although there are also studies on VR-based training for task execution,
equipment operation [27–30], and ergonomic behavior [25,26]. These studies have demonstrated
various benefits of VR-based training over traditional methods. However, using students as
research participants in these studies raises concerns about the generalizability of the findings to
the target population, namely construction workers [23,31,35]. Despite acknowledging this
limitation, previous research has not explored how the effectiveness of VR-based training differs
between these two groups. Therefore, it is crucial to investigate whether the impact of VR-based
training on learning and other relevant outcomes varies between construction workers and graduate
construction engineering students through direct comparison. Such research is necessary to
accurately estimate the effectiveness of VR-based training for construction workers and inform
the development of future training programs.

4

Chapter 2. Literature Review and Research Gaps
2.1 Virtual Reality-based Training
Since the 1990s, the potential of Virtual Reality (VR) has been recognized in various research
fields, and its usage has been increased in training applications [36]. In the following decades,
numerous research studies have investigated and advocated the value of integrating VR in training
and learning [37]. Researchers have found positive results in exploring the efficacy of vocational
training utilizing VR in different industries such as manufacturing, aviation, robotic surgery, and
mining. Generally, these studies have found that VR-based training results in higher knowledge
acquisition [38-42], safety behavior [43-45], and skillsets than traditional training approaches such
as lecture, text, and digital media-based training.
Furthermore, VR can potentially provide a valuable tool for training workers in the construction
industry. In the past two decades, research communities in the construction industry have studied
the effectiveness of VR-based training in various aspects, such as hazard identification and safety
training [21-24], with a more limited number of studies in ergonomic behavior training, equipment
operation, and task execution training [25-30]. These studies have found that VR-based training
has provided an adequate simulation of construction sites enabling significant knowledge
acquisition and retention compared to traditional pedagogical methods (i.e., lectures, text-based
education, or 2D visual guides) [22]. Besides promoting knowledge, VR-based training has also
been shown to enhance self-efficacy in identifying hazards on construction sites [46]. It has also
been associated with improving safety behavior significantly among novices than experienced
workers, indicating its potential to motivate and prepare the incoming generation of construction
workers [46]. This is a crucial finding since the construction industry is challenged with labor
shortages and attracting young workers to fulfill many open job positions on construction sites.
The education research field has also shown that VR can help workers successfully apply the skills
and knowledge developed in simulations in real-world settings as effectively or more effectively
than workers learning through other means [22]. In this context, in-person training in real-world
contexts does not provide a significant advantage over learning these skills in virtual contexts [47].
Delving into the reasons behind the success of VR-based training over traditional training methods;
researchers have come up with multiple points. The first reason comes from the fact that interacting
with the actual construction tools on job sites would often prove dangerous for trainees. Therefore,
VR-based training has the potential to provide hands-on experience without imposing actual safety
risks on the trainees, even if they fail at training tasks [48,49]. Moreover, the simulation tool allows
for designing different scenarios representing potential conditions they may face on construction
sites, which is a distinct advantage over real-world experiences [50]. To shed light on this point,
Schank [50] argues that designers can create simulations that align with research into cognitive
development, thus ensuring learning to a higher degree. Eiris et al. [46] also recognize that VR can
make normally invisible hazards (such as electricity) visible, situating VR as a robust learning
context beyond real-world experience. The second reason behind the success of VR-based training
is its potential to increase construction workers' concentration and engagement because they are
involved in interactive experiences beyond the usual process of passively digesting information
through audio, text, or images in more traditional pedagogical methods [22]. Simulations provide
workers with the affordance of immediate feedback (including haptic feedback) rather than waiting
for an individual to comment on their work [47,48]. However, this point does not imply that VR-

5

based training should replace teachers/trainers or learning materials (e.g., textbooks). Combined
with the on-demand nature of VR, VR presents a potentially democratizing tool in vocational
training within the construction industry, one that may allow more workers in more places to
develop much-needed technical skills. Together, VR simulations provide a highly effective means
for developing safety behaviors, knowledge, and skills related to construction tasks without the
added stressors of potentially injuring someone or chaotic environments. However, researchers
and professionals cannot reach this goal without carefully considering how to design realistic VR
simulations integrated with interactive learning scenarios. Researchers have mentioned the need
for interdisciplinary partnerships between educators, industry professionals, and software
engineers to develop effective VR-based training.
Despite the increased body of research on the effectiveness of VR-based training in preparing
construction workers over the past two decades, there are many unanswered questions when it
comes to training construction workers to work and/or interact with remote-operated robots in the
construction industry using VR-based environments. Research into construction training programs
has overlooked the potential of using VR-based training to improve workers' knowledge,
operational skills, and safety behavior during human-robot interaction on construction sites. In
addition, even in the studies that have addressed the use of VR-based environments to train
construction workers in the operation of construction machines, the transfer of the acquired skills
and safety behavior from the virtual environment to real-world applications using the actual
machines has not been investigated. In response to this gap, the study in chapter 5 investigates the
impact of VR-based training on construction workers' knowledge, safety behavior, and operation
skills in interacting with remote-operated construction robotics. Chapter 5 also explores whether
simulation activities that target key concepts and skills by leveraging the benefits of adult learning
theory (andragogy) improve workers' knowledge and skill during VR- based training.
Despite the extent of research and empirical findings on the effectiveness of VR-based training in
the construction industry, while some researchers have evaluated the impact of VR-based training
on construction workers as the core population of their experiments, others have recruited students
as participants [23, 51, 52]. Even though the research is intended to conclude training for
construction workers- about how much this training could help the construction worker
population- researchers usually do not recruit their samples from this population. The findings of
some of these studies have shown that knowledge gain for students in VR-based training was
higher than video-based or lecture-based training, but the differences between the two training
methods were not significant. In contrast, Dzeng et al. found that students who participated in VR-
based hazard identification training scored significantly higher than those participating in
traditional lecture-based training with a large effect size (d = 2.2) [52]. Similarly, results from
Jeelani and colleagues reveal that students who participated in personalized VR-based safety
training can identify significantly more hazards after the training with a very large effect size (t
(52) = 20.02, p < 0.01, d = 2.76) [51]. As some researchers have noted as limitations of their
studies, these findings with students may not be applicable and generalizable to the professional
labor population. VR-based training may have a different impact on construction workers'
knowledge acquisition rather than the level of knowledge gained by university students. In this
regard, it is essential to investigate and understand the difference in knowledge acquisition
between students and construction workers with VR-based training methods. Chapter 7 compares
the impact of VR-based training on knowledge acquisition in university students versus
construction workers.

6

2.2 Human-Robot Interaction
2.2.1 Trust in the Robot and Robot Operation Self-efficacy
Advancements in automation have allowed workers to collaborate with robots on various job sites;
nevertheless, construction sites' dynamic, unstructured nature has yielded hurdles in deploying
robots on job sites [53]. Construction sites are inherently unpredictable, and construction workers
and robots work alongside each other rather than separately and in an organized environment as
they do in other industries (i.e., manufacturing). In addition, since robots are often designed to
perform more dangerous tasks than humans in collaborative teams of humans and robots, human's
trust in the robot to fulfill the task safely and effectively plays a more pivotal role in high-risk
environments, such as construction sites. For instance, surveys indicate that workers might prefer
traditional methods over technological solutions due to the unpredictable and dynamic nature of
construction sites [54]. They often feel unsafe working around robots. Accordingly, it is crucial
to build and enhance trust in the automation or robotic system among construction workers.
Building this trust among human operators or collaborators produces an increased sense of safety,
a willingness to accept robot-provided information or decisions, and an inclination to work with
robots in the future.
Lee and See [55] define trust as "the attitude that an agent (e.g., automation, a robot, or a human)
will help achieve an individual's goals in a situation characterized by uncertainty and
vulnerability." The level of human’s trust depends on the characteristics of the trustee (e.g., culture,
age, gender, personality), the trustor (e.g., features of the automation, capabilities of the
automation), and the context of the interaction between them (e.g., team collaboration, tasks) [56-
58]. Trust in human interaction with automation can be challenged by disuse and misuse. Disuse
relates to the situation when humans do not accept technology and reject using it, while misuse
refers to over-trusting automation excessively and inappropriately [57]. While trust in automation
and trust in robots have similar fundamental characteristics, the human–robot trust may differ from
the human-automation trust since robots have different characteristics than other forms of
automation [59]. In this regard, researchers have been investigating factors that influence trust in
a robot [60]. Existing studies indicate that trust in a robot can be influenced by the characteristics
of humans, robots, and the surrounding environment [61], with the characteristics of the robot
being regarded as more significant than the characteristics of humans and the environment in the
development of trust [59]. On many occasions, however, there are mismatches between the
perceptions of humans on the robot's characteristics and capabilities and the robot's actual
characteristics and capabilities, which can lead to trust failures. For that, training the humans
involved in interactions with robots has been presented as a key strategy to promote trust by
reducing the differences between the expectations of humans toward the robot's capabilities [59]
and the actual robot's capabilities and to recover trust after trust failures resulting from incorrect
user expectations toward the robot or user unintentional failures during the interaction [62].
Most commonly, trust is assessed subjectively with the help of questionnaires based on Likert
scales in which the subjects indicate their levels of trust in their ability to properly interact with
the robot (self-efficacy) and/or the ability of the robot to achieve the task goals. Examples include
proposed trust scales that account for various factors that influence HRI, such as team
configuration, team process, context, task, and system [63], and trust scales that assess the overall
perception of the subjects on the robot's capabilities using repeated measures analysis [64]. In one
of the few attempts to measure trust in a robot objectively, researchers proposed a model that

7

determines an overall trust score based on the human task allocation decision behavior, risk, and
robot behavior and found that as robot competency decreases, the mission time and the user
interventions increase [65]. Based on the proposed formulation, the authors also proposed an
analytical methodology that allows the comparison of the trust behavior of the operators to the
expected behaviors of an expert, providing direct feedback on the operator's training needs relative
to trust behavior. The model proposed is based on the correlation between trust in automation and
self-confidence or self-efficacy. Robot-use self-efficacy is a human-related characteristic
correlated with trust in a robot. Self-efficacy refers to an individual's belief about his/her
performance skills in a given situation. Specifically, robot-use self-efficacy refers to the workers'
beliefs about their ability to use robots [66]. However, self-efficacy does not equal efficacy; a
person may be able to perform a task successfully, but he/she may not believe that they have the
power to produce the desired effect [67].
VR-based training has been used to study and enhance trust in automation in different fields,
including drivers' and pedestrians' trust in autonomous vehicles [68-71]. However, in construction
applications, the study of trust in HRI is rare and limited to the study of perceived safety in HRI
teams because of the physical separation between workers and robots and its impacts on promoting
team identification and trust [72]. Since the development of trust in the robot and robot operation
self-efficacy is crucial for adopting construction robotics, chapter 6 investigates VR-based
training's impact in enhancing the factors mentioned above in construction workers compared to
traditional in-person training. Furthermore, while the research community in the construction
industry is striving to understand construction workers' trust and self-efficacy in interaction with
robots and equipment, some researchers have recruited students in their experiments to understand
construction workers' perceived safety in HRI. Investigating the difference in the effect of VR-
based training on students' and workers' trust in the robot and robot operation self-efficacy might
shed light on the generalizability of the findings to the professional population in future studies.
Hence, chapter 7 investigates the effectiveness of the same VR-based training on construction
workers' and university students' trust in the robot and robot operation self-efficacy.
2.2.2 Mental Workload
Since more than 70% of all accidents in the construction industry are related to workers'
movements and activities, it is crucial to mitigate human-related factors influencing the safety
conditions in this industry [73]. Construction workers' capability to perceive hazards can help them
to avoid dangerous conditions and prevent accidents. Among the human-related factors that relate
to hazard perception, especially in interaction with robots, is mental workload (MWL) [74-76].
One of the most accepted definitions of the MWL associated with a task is "the level of attentional
resources required to meet objective and subjective performance criteria, which task demands,
external support, and experience may mediate” [77].
The study of MWL has become a topic of interest due to the increasing cognitive demand
requirements resulting from deploying more complex human-machine and human–robot systems
in diverse fields, including aviation, surgery, manufacturing, and construction. In many studies,
MWL has been recognized as a critical factor that affects an operator's performance during human-
machine and human–robot interactions [78-82]. Most commonly, these studies have shown that
decreasing the cognitive loads imposed on the operator during task execution usually results in
improved performance. Although most studies focus on mental overload, when task requirements
overcome operator capabilities, mental underload is another situation that leads to reduced

8

performance. As presented by Young et al. [83], instead of trying to remove the operator from as
many tasks as possible when deploying automated systems, the designer should try to optimize the
design of the tasks to take advantage of both the technology and the operator's skills, which can be
accomplished through the use of adaptive interfaces and dynamic task allocation. In such cases,
human factors such as the operator's workload and levels of fatigue, and physiological data, such
as heart rate variability, can be used to dynamically allocate tasks to the humans and robots
involved in the interaction to alleviate the adverse effects of workload, fatigue, and stress [84, 85].
Various methods can be used to assess MWL during task execution, including subjective measures
[e.g., NASA-Task Load Index (TLX) and the subjective workload assessment technique (SWAT)],
physiological measures (e.g., heart rate, eye-gazing, electrodermal response), and objective
measures based on task performance (primary and/or secondary tasks). Developed by the Ames
Research Center [86,87], the NASA-TLX is a standard, questionnaire-based, subjective measure
of the overall workload experienced by a human working in a human-machine or human–robot
system. It is one of the most used measures of task load and considers six subscales: mental
demand, physical demand, temporal demand, level of performance, effort, and frustration [88-91].
Even though various physiological measures have been used to predict MWL in many domains,
subjective assessments alone have been preferred in many studies, mainly due to their simplicity
of application and non-intrusive nature. Also, for MWL specifically, existing studies show that
while most of the physiological measures used in MWL research can detect changes in MWL
levels, the validity of these measures depends on the application at hand, requiring a proper
selection of the physiological measures for each task scenario [92, 93].
In construction applications, some of these techniques have been used, sometimes combined, to
assess the levels of MWL that workers experience when working alongside machines and robots
[94], assess the reliability of using physiological data to predict MWL [95], or adjust robot
behavior during the interaction [96]. Current efforts to understand the implications of VR-based
training on MWL have shown significant differences between the levels of MWL experienced by
the subjects when operating simulated drones and real drones, being the MWL higher in the
simulated condition [95]. Nevertheless, it is still not clear whether the same results can be obtained
when using VR-based training to train construction workers on the operation of more complex
construction machines and robots, given the requirements of longer training sessions, more
unstructured environments, and the relatively more complex control interfaces and mechanisms
found in these machines/robots.
Despite the increasing body of research on the cognitive impacts of the deployment of intelligent
systems and robotics on-site, the impacts of VR-based training on the mental loads experienced
by construction workers during the actual remote operation of a construction robot have not yet
been thoroughly explored. Chapter 6 investigates the mental workloads experienced by two groups
of construction workers with VR-based versus in-person training are measured using NASA-TLX
and compared to evaluate the effectiveness of VR-based training in reducing MWL during the
remote operation of a construction robot.
2.2.3 Situational Awareness
Another critical human-related factor in human-robot interaction is Situational Awareness (SA),
which, according to Johnson et al. [97], forms the basis for decision-making and performance in
the operation of complex systems. As with the MWL, current studies have increasingly focused
on SA to investigate new human-machine systems design and training programs in various fields

9

[98]. The most accepted definition of SA centers on the operator's "perception of the elements of
the environment within a volume of time and space, the comprehension of their meaning, and the
projection of their status in the near future" [99]. This definition presents three phases in an
operator acquiring SA: perception, comprehension, and projection. These three phases are defined
in the hierarchical model of SA in decision-making proposed by Endsley et al. [100], which defines
Level 1 SA (lowest level) as the perception of the environment and its elements, Level 2 SA as the
holistic comprehension of these elements, and their implications for the task goals, and Level 3
SA (highest level) as the projection of the future states of these elements in the environment.
Various methods and metrics have been proposed to assess workers' SA, which include process
measures, performance measures, and direct SA measures, which are further differentiated among
Situation Awareness Rating Technique (SART), Situation Awareness Global Assessment
Technique (SAGAT), and Situation Present Assessment Technique (SPAM) [98]. Among these,
SAGAT is one of the most used techniques for measuring SA and involves randomly freezing the
task simulation and asking the subject questions about the current situation as a means to determine
his/her knowledge about the situation considering the three levels of SA (perception,
comprehension, and prediction) [99, 100]. After multiple queries occur at various moments during
the simulation, a composite SAGAT score is calculated. It represents an objective measure of SA
because the perceptions of the operator (as represented by his/her answers to the queries) are
compared to the actual conditions of the simulation [99].
In the construction research field, SA has commonly been studied from the perspectives of hazard
identification and/or operating performance of complex machines and equipment, especially
cranes and excavators [101-104]. Existing results show that increasing an operator's SA with the
help of an assistance system based on visual cues, for example, can improve the overall operator's
safety and task performance [102]. Relative to the use of VR-based training to increase
construction workers' SA, it claimed that current VR-based simulators for construction operation
training put too much emphasis on the development of photo- and physics-realistic scenarios and
less emphasis on the development of context-realistic scenarios, limiting the ability of the trainees
to increase their SA and skills [105]. As is the case with the operation of actual construction
equipment, increasing the worker's SA during training in a simulated environment can also
improve the worker's safety behavior and help workers to visualize potential risks associated with
their actions after the training sections [101]. Many studies show that physical and mental loads
and environmental and task requirements also affect the worker's SA and, consequently, the ability
of these workers to identify safety hazards during task execution. Task complexity, for example,
has been associated with reduced performance and SA and increased mental workload, which may
require specific training scenarios to mitigate the reduction of the operator's SA levels during more
complex tasks. Finally, for similar levels of task complexity, construction workers' SA is
significantly affected by different levels of MWL, with SA decreasing for higher levels of MWL
[106].
Although construction sites represent one of the most hazardous working environments and there
have been an increased number of robots deployed on construction sites, there is still a lack of
research into the potential of VR-based training to enhance the workers' SA during the remote
operation of actual construction robots. Thus, chapter 6 investigates the impact of VR-based
training on construction workers' SA compared to traditional in-person training.

10

Chapter 3. Research Objectives and Questions
3.1 Research Objective 1
To investigate the impact of VR-based training on knowledge acquisition, operational skills, and
safety behavior while working with the robot compared to a more traditional, comparable in-
person pedagogical model.
• Research Question 1.1: How does VR-based training impact knowledge acquisition for
construction workers, compared to the traditional training method?
• Research Question 1.2: How does VR-based training impact construction workers’ safety
behavior, compared to the traditional training method?
• Research Question 1.3: How does VR-based training impact construction workers’
operational skills, compared to the traditional training method?
3.2 Research Objective 2
To explore the effectiveness of VR-based training on construction workers’ mental workload and
situational awareness, as well as their development of trust in robots and ability to use the robot
(robot-use self-efficacy), compared to a more traditional, comparable in-person pedagogical
model.
• Research Question 2.1: How does VR-based training impact mental workload for
construction workers, compared to the traditional training method?
• Research Question 2.2: How does VR-based training impact situational awareness for
construction workers, compared to the traditional training method?
• Research Question 2.3: How does VR-based training impact trust in the robot and robot
operation self-efficacy for construction workers, compared to the traditional training
method?
3.3 Research Objective 3
To empirically compare the effect of VR-based training in construction workers vs. graduate
construction engineering students on knowledge acquisition, trust in the robot, and robot operation
self-efficacy.
• Research Question 3.1: How does VR-based training impact knowledge acquisition for
construction engineering students, compared to construction workers?
• Research Question 3.2: How does VR-based training impact construction engineering
students’ robot operation self-efficacy, compared to construction workers?
• Research Question 3.3: How does VR-based training impact construction engineering
students’ trust in the robot, compared to construction workers?

11

Chapter 4. An Immersive Virtual Reality (VR)-based Training for
Construction Robotics Remote-Operation
This chapter presents a Virtual Reality (VR)-based training that simulates the remote operation of
a construction robot in order to support training in worker-robot teamwork on construction sites.
The VR-based training presented in this chapter utilizes a remote-operated demolition robot named
Brokk. This chapter describes the use-case robot (Brokk), system setup and configuration, and the
learning scenarios. In this training, the trainee learns about the robot's various components, safety
management, the robot's control box functions, and demolition instructions used during common
demolition tasks. Significantly, this training is developed based on adult learning theories in
general—and andragogy principles in particular. In addition, the VR-based training has been
augmented with useful features from existing VR-based training programs. The impact of VR-
based training on construction workers will be investigated in subsequent chapters.
4.1 Background
4.1.1 Existing VR-based Training Programs
Several VR-based training programs have been developed in the construction industry. However,
most of them have focused exclusively on hazard identification and safety training, with a more
limited number of studies in ergonomic behavior training, equipment operation, and task
execution training. Moreover, several VR-based training programs for robot-incorporated work
sites are designed for various industries, including mining, manufacturing, shipbuilding, and
healthcare. A novel VR-based training should consider existing training programs' limitations and
useful characteristics. VR-based training includes features such as visualization tools and
controllers, navigation methods, learning scenarios, and virtual environments simulated in training
programs.
4.1.1.1 Visualization Tools and Controllers
One of the main features of existing VR-based training programs is the medium through which
users can visualize the simulated environment. Many initial programs have used PC monitors to
visualize the environment and keyboard and mouse as the user interface. However, monitors do
not provide immersive learning experiences. Others have used a power wall as the visualization
method to provide immersive and realistic experiences. However, power walls only allow a third-
person view of the trainee. Recent VR-based training programs have used Head-Mounted Displays
(HMDs) as the visualization tool. HMDs can offer first- and third-person views, providing a sense
of presence in the simulated environment; however, only a few researchers have utilized first- and
third-person views in their VR-based training programs. When the trainees experience both views
in training, they can be more aware of their interactions with other workers and/or machines.
4.1.1.2 Navigation Methods
Many robot-incorporated VR-based training programs require the trainee to be stationary during
the training. Usually, handheld controllers are used to navigating in the virtual environment, which
can inhibit the natural interaction between humans and the robot. Other VR-based training
programs allow the trainee to walk around the environment; however, the walking boundaries are

12

often limited. Therefore, a controller-free navigation method should be considered in the design
of VR-based training to allow trainees to freely remote-operate with the robot.
4.1.1.3 Learning Scenarios
Many researchers in the construction industry have studied VR-based training in the format of
tutorial-like games which provide pre-determined sequences of conditions. These learning
scenarios lack interactivity since the trainee has no opportunities for collaborative interactions with
other workers and equipment. However, some of the robot-included VR-based training in other
industries have developed interactive learning scenarios that promote “learning while doing.” This
is one of the valuable features that should be appropriated for developing novel VR-based training
for the remote operation of robots in the construction industry.
4.1.1.4 Virtual Environments
Most existing VR-based training programs have not incorporated other virtual workers in the
simulated environment, thus, limiting the trainees’ interactions with the robot. Since construction
sites are dynamic spaces within which multiple interactions occur between multiple workers, VR-
based training should simulate other construction workers in the virtual environment. Besides,
many existing programs are confined to and simulate indoor job sites without simulating outdoor
operating environments. Crucially, since construction sites are mostly outdoors, VR-based training
should provide realistic elements and conditions common to a construction site, including uneven
surfaces, dust, realistic shadows, and various weather conditions.
To summarize, many researchers have studied the application of VR-based training in various
industries. However, the effectiveness of VR-based training for the remote operation of robots is
underexplored in the construction industry. Moreover, most of the VR-based training programs in
the construction industry have focused on hazard identification and safety training. Furthermore,
many recent training programs require using a handheld controller as the navigation tool while the
user is stationary. Existing programs providing first- and third-person views are few and far
between. Many programs have used tutorial-like games format, failing to provide interactive
learning scenarios. Most of them have followed traditional sequences for the learning material
without offering opportunities for learning by doing. In addition, many existing programs are cast
in indoor environments without simulating more common construction scenarios—outdoor
remote-operating scenarios.
4.1.2 Adult Learning Theory
Pedagogy is defined by Knowles (1980) [107] as the "art and science of helping children learn."
In contrast, andragogy is a theory of adult learning proposed by Knowles (1968), based on five
assumptions. It is essential for training targeting upskilling/reskilling adult learners to incorporate
principles and assumptions of adult learning theories in the design. In this area, researchers have
tried to recognize adult learners' characteristics and attributes in the development of learning
programs. The five assumptions of Andragogy are as follow [107]:
1. An adult has an independent self-concept who can direct his or her learning.
2. An adult has a reservoir of life experiences, which can be a rich resource for learning.
3. An adult has learning needs closely related to changing social roles.
4. An adult is problem-centered and interested in the immediate application of knowledge.

13

5. An adult is motivated to learn by internal rather than external forces.
Although there is some debate about the definitions of andragogy and pedagogy, andragogy
continues to serve as the base for learning programs developed for adult learners. One ongoing
concern is how andragogy's assumptions can be applied to adults. For example, some adults may
not prefer learning mapped along the self-directed learning approach (andragogy); instead, they
may prefer the teacher-directed learning approach (pedagogy) [108]. Knowles addressed this issue
by proposing a continuum ranging from pedagogy to andragogy [107]. Therefore, while VR-based
training may be driven by principles and assumptions of andragogy, it may not be wholly based
on the pedagogy or andragogy principles.
4.2 Use-case Robot
The VR-based training in this study is developed for a remote-operated demolition robot
manufactured by Brokk Inc (Fig. 1). A remote-operated demolition robot is selected based on the
industry acceptance trends, technology development level, use frequency in construction projects,
and potential impact on enhancing construction productivity and safety. Construction workers are
exposed to more challenging and hazardous work conditions (e.g., extreme weather conditions,
dust, collapses, radioactivity contamination) in demolition tasks than other construction jobs.
Handheld demolition tools are associated with an average of 32 missed days for workers due to
fractures, injuries, and the effects of excessive vibration and strain. Therefore, remote-operated
demolition robots are employed faster than other types of robots on construction sites to enhance
safety and productivity. This type of robot constitutes about 90% of the total construction robotics
market [109]. Remote-operated demolition robots can access spaces out of reach or hazardous for
workers (e.g., nuclear sites) as workers can remotely execute demolition tasks safer and faster-
using demolition robots. Brokk has been used for various demolition tasks from the destruction of
a concrete building to the delicate tasks of building renovation. Due to its size, the robot can be
operated in confined spaces where other construction equipment cannot go. Since the operator has
the most control over the robot on the construction sites, it is critical to train operators to ensure
the safety and efficiency of the working environment. For instance, we need to teach operators the
safe distance to operate the robot, and they need to learn the emergency tasks to execute when a
construction worker enters the danger zone. Hence, the effectiveness of VR-based training
utilizing a remote-operated robot has been studied. Since a human worker controls the robot
directly, the human role in this interaction is to be the operator. Since one operator interacts with
one demolition robot, the team composition is one human to one robot. The communication
between humans and robots is based on digital codes through the robot’s controller (buttons and
joysticks). Hence, the interaction type is physical and synchronous since the operator and the robot
work simultaneously.

14

Fig. 1: The actual and the simulated Brok110.
4.3 System Setup and Configuration
The developed VR-based training is developed using the Unity3D game engine platform. The
training, which takes about 120 minutes to complete, occurs in a four-floor building and a
simulated construction site, modeled in Revit and imported FBX files into Unity3D using the
PiXYZ plugin. Robot’s model has been simulated through physics simulation in the Unity3D game
engine. Brokk110’s technical specifications, such as mass, drag, angular drag, and mesh colliders
of various components, are used to model the rigid body properties of the robot in the VR
environment. Additionally, multiple joints of the 5-degrees-of-freedom (DOF) robot (e.g., fixed,
hinge, and configurable joints) have been modeled to provide an accurate movement similar to the
actual robot. Connected bodies, anchors, break force, and torque is assigned based on
specifications acquired from the robot’s manufacturing company. Additionally, various robot
components’ movement and rotation (considering relative axis and speed) are simulated using
scripts written in C# programming language. The virtual model of the robot has been tested and
verified by an expert trainer with more than 15 years of experience in training workers. Moreover,
the simulated construction site includes various work conditions such as dust, rainy and sunny
weather conditions, and uneven terrain to deliver trainees a realistic experience of an actual job
site. In this regard, a set of built-in 3D models are added to the virtual environment from the
Unity3D asset store to simulate construction sites.
The training system consists of VR-based training on a PC with an NVIDIA GeForce GTX 1080
graphics card. The trainee must wear a head-mounted display (HMD) as the visualization tool. The
trainee uses a VR controller to interact with the VR-based training (e.g., going to the next/previous
step in the learning scenario, replaying the narrative voice, and interacting with objects in the VE).
While the HMD gives the trainee a first-person view, the headphone connected to the HMD
provides sound effects. Two base stations track the HMD and VR controller. In addition to the VR
equipment, the trainee uses the demolition robot’s actual controller unit to remote operate the
simulated robot in the VR-based training environment. The robot’s controller is programmed and
connected to the computer using Arduino Pro micro serial connection.
Since the trainee needs to use the robot’s controller during the training, it is essential to use a
controller-free navigation method so that the trainee does not need a controller to walk within the
virtual environment. Therefore, the locomotion technique used in this VR-based training is a walk-
in-place treadmill. Virtuix Omni is used as the VR treadmill, designed to allow participants to walk
within the VR-based training environment without boundary since they are walking on a treadmill,

15

as opposed to a room-scale VR environment that would limit the participants to the boundary of
the room that the experiment takes place. The treadmill has a bowl-shaped surface that requires
the user to wear low-friction shoes for movement. Using inertial sensors, the simulator can track
the trainee’s position, speed, and stride length.

Fig. 2: System setup, Trainee on a VR treadmill
4.4 Learning Modules
Multiple steps are taken to develop the learning modules of the VR-based training so that the
program's content targets the critical content and skills workers need to remote-operate the robot
safely and effectively. Since the median age of the labor force in the North American construction
industry is 42.9 years [110], learning modules are developed based on andragogy learning theory
principles, one of the significant adult learning theories. A focus group interview is run with six
expert trainers to ensure that learning modules are generalizable across trainers. This has been a
vital step to ensure that workers receive essential learning related to the changing state of
construction sites, as mentioned in the third principle of adult learning theory. In the next step,
learning modules for the VR-based training are developed using manuals of the demolition robot
provided by the manufacturer, along with analyzing the data collected through focus groups and
in-person training sessions. Before starting the experiment, a pilot study was run to prevent
potential technical issues, such as the malfunction of the hand-held VR controller and the accuracy
of the simulated demolition robot's controller. In the final step, an expert trainer verified the
developed learning modules for the experiment.
The VR-based training studied in this research project consists of seven learning modules in two
languages (English and Spanish). As recommended, the learning content is displayed in a text box
on a canvas that trainees can interact with using the VR controller. Since adult learning theory
assumes that adults are independent self-concepts who can direct their learning direction, trainees
are able to go back and forth at their own pace in the learning scenarios. If trainees cannot read the
text, they can listen to the narration by using the controller. At the end of each learning module,
the trainee is evaluated by means of some tasks in the virtual environment. Following this, the

16

training program gives feedback based on the trainees' performance, this feature can help trainees
who do not have all the characteristics of an adult learner and need direction throughout the
training. A detailed discussion of each of the seven learning modules is presented in the following
paragraphs, and table 1 presents the targeted operational skills and safety behavior in each learning
module.
In the first module, basic information about the robot is presented. The trainee starts the scenario
in a construction site and can see the robot in front of him/her. The learning materials are displayed
on a board. First, the purpose and applications of the robot are introduced. Then, the trainee learns
about the different components of the robot. Various components are highlighted, and animations
illustrate the range of movement for each component. The introduction of the robot in this module
embraced the notion that workers are motivated to gain new skills and knowledge in their field of
expertise to adapt themselves to the advancement of the construction industry. Brokk has various
models, so the differences and commonalities between them are summarized for the trainee. After
these steps, multiple boxes appear under the board, each representing the names of the robot's
components (Fig. 2(a)). If the trainee points to these boxes, each component's detailed information
will be presented on the board along with the animation of the highlighted elements. The trainee
has the freedom to walk around the robot, examine different parts of the robot, and watch the
movements and detailed information to get familiar with the robot. This is aligned with the self-
direction approach in training adult learners. At the end of the module, for assessment, the trainee
is tasked with answering questions about the components by pointing the controller to the
components mentioned in the question. Using the ray-casting and event-triggering features, the
trainee is automatically provided feedback.
In the second module, the trainee learns how to safely remote operate the robot. First, information
about how to safely power the robot is presented. Then, the trainee learns power cable
management, that is, the cable: should not be on a wet surface or sharp objects should be behind
the robot and should not be close to the outriggers. As assumed in adult learning theory,
construction workers have a reservoir of experiences, such as managing power cables, different
interactions on job sites, and demolition procedures, which can be used as a rich resource for
learning. In the learning scenario, the cable is initially in a dangerous position. However, with the
guidance of the VR-based training program, the trainee learns how to move it step by step to a safe
position by picking up the cable and positioning it via the controller. Thus, the trainee learns power
cable management by doing, which is one of the crucial principles of adult learning theories. In
the next section, the trainee learns about safe operator positioning. Trainees are exposed to the
concept of the danger zone around the robot. The boundary of this zone depends on a number of
factors, such as the height of the robot's arm. These boundary conditions are introduced to the
trainee through Unity 3D's animation feature (Fig. 2(b)). If the trainee enters the danger zone, the
screen turns red, and the trainee is signaled to exit the zone. In addition, a virtual worker enters the
danger zone during the learning scenario to illustrate the consequences of violating the danger
zone boundary. This point is aligned with the assumption that adult learners are problem-centered
and motivated to learn the actual scenarios and immediate applications of knowledge and skills
they might use on construction sites. Since a box-collider component is attached to the user's
avatar, if the trainee walks into the danger zone, he/she will be hit by the robot's arm. Considering
that many demolition scenarios require positioning the robot on an inclined surface, the robot
operator should be cognizant of the threshold for safely positioning the robot (i.e., 30 degrees limit
for the safe operation on inclined surfaces). This is illustrated by contrasting two types of ramps,

17

one with a slope lower than the safe limit and the other with a slope higher than the safe limit. The
trainee can see that the robot topples on the ramp with the steeper slope allowing him/her to learn
about the 30 degrees limit for safe operation on inclined surfaces. In the last section of this module,
the trainee learns the most critical points about workplace inspection; for instance, whether or not
the robot can be maneuvered to fit in the workplace and assess other workplace conditions such as
overhead (ceiling) and the lighting condition. For the assessment section of this module, two key
areas are targeted: (1) positioning the cable in a safe condition and (2) knowledge of safety issues
for operation.
In the third learning module, the trainee learns how to use the control box to operate the robot.
This module is not in an immersive virtual environment since the user needs to see the control box.
After getting familiar with the control box, the trainee wears the HMD in the following learning
modules without needing to see the controller. This design approach has considered the importance
of not watching the controller and focusing on the safety points during remote-operation in
dynamic construction sites, which aligns with the immediate application of the skills after the
training. The trainee learns how to start the control box and perform the functions of each button
step by step. The trainee is able to use the control box to move the robot in the virtual environment
visualized on the screen. In the assessment section of this module, the trainee is assessed on their
ability to use the control box to perform various operations with the robot.
In the fourth module, the trainee learns how to start the robot using the control box using a pre-
startup checklist. Before starting the robot, some key steps need to be followed: check the hydraulic
oil level, ensure no oil leakage, and inspect for loose objects on the robot. Also, the emergency
stop button of the control box should be checked. In this module's assessment, the trainee is asked
to execute the pre-startup checklist and mention the issues that must be addressed. The module's
design has considered the problem-centered assumption of adult learners during the training.
In the fifth module, the operating positioning of the robot is introduced. It is essential to correctly
position the robot to facilitate the demolition and prevent unnecessary damage to the machine. The
demolition robot should not be too close to the work object; the distance between the robot and
other objects in the demolition site must also be considered. Moreover, the robot's arms should not
be fully extended. Besides, the angles between the arms should be within an acceptable limit. The
trainee moves the robot to a safe operating condition with the guidance of the VR-based learning
program. The practice tasks designed in this module have considered the actual scenarios that
construction workers might encounter on construction sites. In the assessment part of this module,
the trainee is tasked with positioning the robot in a safe operating condition.
In the sixth module, trainees are provided opportunities to learn how to move the robot. Now that
the trainee has learned the control box functions, the trainee practices moving the robot and its
different parts through three activities—the first activity centers around using the robot to kick a
soccer ball (Fig. 2(c)). A virtual soccer ball is given to the trainee, and he/she is asked to place the
ball in a specified zone by using the robot. This exercise provides practice opportunities around
different activities: moving the robot, using the outriggers to position and stabilize the robot, and
moving the arm system to kick the ball. The tasks design in this module targets the interest of
workers as adult learners to learn through solving problems. Trainees experience the consequences
of their decisions while remote operating the robot in the virtual environment without causing any
damage to themselves or the actual robot. For example, if trainees do not put the arm system of
the demolition robot in a safe position and stretch the arms while moving, the robot will tilt, and
trainees will experience the consequence of their unsafe behavior. The next activity asks the trainee

18

to move the robot through a pipe. This task is designed to help the trainee learn how to navigate
the robot through confined spaces. In the last activity of this module, the trainee is tasked with
positioning the robot on an inclined surface. Since demolition scenarios frequently happen on
sloping surfaces, the trainee needs to practice positioning the robot on these surfaces. This
consideration in the design is aligned with the necessity to provide the immediate applications of
the skills they learn for their careers. Given the practice-oriented nature of this module, there is no
corresponding assessment.
The final module presents the general recommendations for demolishing concrete slabs, floors,
free-standing walls, beams, and columns (Fig. 2(d)). The direction of the demolition tool and the
demolition process is vital to learn. Not only is the demolition sequence important, but also the
starting points of the demolition. These general recommendations are exhibited to the trainee by
highlighting and animating the robot and the work objects. After the learning scenario, the trainee
can practice demolishing objects using various strategies to experience the consequences of each
approach. Trainees have the opportunity to practice with the demolition robot in the simulated
construction site and execute tasks using different strategies based on the learning content and their
prior experience and background knowledge. This design feature considers the self-direction
characteristics and having reservoir of experiences of construction workers as adult learners. In
the assessment of this module, the trainee is asked to demolish an object to evaluate the trainee's
performance in following the expected rules.

Table 1
Targeted operational skills and safety behavior in each learning module
Mod. # Targeted Operational Skills Targeted Safety Behavior

1
Introduction to the robot, its purpose, different
applications, different components, and their
detailed explanation
Introduction to the range of motion for each
component

2
Operator positioning to have the best view and
control during operation & how to move the
robot (e.g., whether or not the robot can be
maneuvered to fit in the workspace)
Cable safety management (e.g., the cable should not
be on wet surfaces, near sharp objects, outriggers, and
cable should be undamaged and behind the robot).
Definition of operating zone and risk zone and
boundary conditions for the risk zone. Workplace
inspection (e.g., keep robot out of dust and flying
rocks, be aware of personnel; turn off the robot in the
event people enter the operating zone)

3
How to use the control unit (e.g., controller’s
setting, how to use each lever/button, etc.)

How to use the controller levers smoothly to move
the robot’s components safely and at a controllable
pace.

4
How to start the control unit and the robot Safety checks before starting the robot (e.g., check the
hydraulic fluid level, ensure that there is no oil
leakage, check if power and control cables are
connected, inspect for loose objects on the robot, and
check the emergency stop button of the control unit)

19

5
How to position the robot (e.g., demolition
robot should not be too close to the object, the
distance between the robot and other objects
must be considered, optimum operating
position for the arm system)
How to position the robot safely (e.g., robot's arms
should not be fully extended; angles between
cylinders should be within an acceptable limit)

6
How to move the robot, use the outriggers to
position and stabilize the robot, and move the
arm system via different simulated activities
Safety concerns during movement of the robot (e.g.,
avoiding the danger of tilting the robot; robot must be
secured if there is a risk of collapsing/tilting)

7
How to demolish concrete slabs, floors, walls,
beams, and columns effectively (e.g., the
direction of demolition tools and demolition
process and sequence, demolition starting
points, demolition in one direction and
sections, demolish the entire section within
the working zone before moving the robot,
etc.)

How to demolish concrete slabs, floors, walls, beams
and, columns safely (e.g., positioning of the hammer
to prevent harmful bounces, levers movement speed
to have the best control on delicate demolition tasks,
working using sight and hearing, etc.)

(a) (b)

(c) (d)
Fig: 3: (a) Highlights and animations illustrating the range of each component’s movement (Module 1), (b)
Illustration of risk zone boundary and consequence of violating it (Module 2), (c) Trainee practices using the
control unit by kicking a soccer ball, moving the robot through a pipe, and positioning the robot on an inclined
surface (Module 6), (d) Demolition of concrete slabs (Module 7)

20

21

Chapter 5. Effectiveness of VR-based Training on Improving Construction
Workers’ Knowledge, Skills, and Safety behavior in Robotic Remote-operation
This chapter addresses the first research objective by investigating the impact of VR-based training
on construction workers’ knowledge acquisition, operational skills, and safety behavior compared
to the traditional training for the remote-operation of construction robots. The study’s
methodology, including in-person training, participants, experimental procedure, and analysis
methods, are presented in section 5.1. Also, the results and findings are discussed in sections 5.2
and 5.3. Finally, sections 5.4 and 5.5 presents the limitations, summary, and directions for future
research steps.
5.1 Methodology
5.1.1 In-person Training
As stated in section 3, the first research objective was to understand the impact of VR-based
training compared to traditional in-person training, a common practice in the construction industry.
Therefore, in-person training with similar content and duration was provided to workers. The
sessions were held with around six to seven workers, one expert trainer, and the actual demolition
robot (Brokk), lasting about two hours (Fig. 3). The location of the training was the outdoor area
of about 700 sqft at the Department of Civil & Environmental Engineering at the University of
Southern California. The robot size in both VR-based and in-person training was the same. While
the learning content in both pieces of training was the same and in parallel, the training
methodology was different. For example, in-person training was not designed based on adult
learning theory (andragogy). The expert trainer started in-person training by verbally introducing
the robot, its history, and its applications. Similar to VR-based training, in module 1 (Table 1), the
expert trainer presented a detailed introduction to the robot's various components. Then, he
described how to manage safety points while demonstrating them with the actual robot. For
instance, how to manage the power cable or what the robot's operation zone is (VR-based training;
module 2). Similar to the content of the VR-based training, module 4, he presented the instructions
for starting the robot and its controller, including checking pre-start safety points. In the next
section of the training, the trainer demonstrated different controller functions to move the robot
(VR-based training; module 3). He also taught the trainees how to move the robot safely,
preventing tilting (VR-based training, module 6). He showed how to position the robot for
demolition (VR-based training, module 5), stabilize it using its outriggers, and the safe and
effective strategies to demolish a concrete block, like the last learning module of the VR-based
training (VR-based training, module 7). After demonstrating the robot and a Q&A session, trainees
had the opportunity to work with the actual controller and the demolition robot under the trainer's
supervision. However, since there was one robot and multiple trainees, they had limited time to
remote-operate the robot. Besides, trainees were limited in various maneuverings in remote
operations since they may cause damage to the actual robot and themselves. As the trainees
practiced remote-operating the robot, the trainer provided feedback to trainees about their
operational skills and safety behavior.

22

Fig. 4: Workers during an in-person training session
5.1.2 Participants
The inclusion criteria for the study participants were construction workers over 18 years old.
Therefore, 50 construction workers over 18 years old (48 males and two females) were recruited
for the experiment from on-campus construction projects at the University of Southern California.
They were with varying degrees of experience in the construction industry. They were randomly
assigned to complete either VR-based or in-person training (25 participants for each group). It is
vital to mention that research members have used data from 49 participants in analyses since one
VR-based training participant quit the study due to the inability to use the VR-based equipment.
Table 2 presents the demographic information of the participants based on their age, language,
education level, and years of experience in the construction industry. None of the recruited
participants had previous experience with the demolition robot used in the experiment or VR-based
training. Only one participant from the in-person training group had a previous experience with a
demolition robot (not the demolition robot used in this study); however, the answer to this question
was not an exclusion criterion for the study.
Based on the analyses of the collected demographic information, no statistically significant
relationships were found between worker’s gender and race and the training to which they were
assigned, 
2
(1, N = 49) = 0.31, p = .576 for gender, and 
2
(1, N = 49) = 1.06, p = .302 for race.
Participants in these two conditions were also not statistically different in terms of their age group

2
(1, N = 49) = 0.98, p = .808, experience in the construction 
2
(1, N = 48) = 0.47, p = .792, and
experience with using a demolition robot 
2
(1, N = 49) = 0.98, p = .322. In addition, workers in
each training condition had similar levels of prior experience with VR 
2
(1, N = 49) = 1.18, p =
.277. Both groups also had similar levels of initial trust in the robot (Mdiff = −.22, SD = .17, p =
.20), and self-efficacy (Mdiff = −0.14, SD = .29, p = .628). Hence, we can confidently state that,
taken altogether, randomization was successful and workers in both training programs were similar
regarding their backgrounds, demographics, and baseline trust and beliefs.

23

Table 2
Demographics of participants based on training types
Demographics

VR-based training In-person training

Language

English 12 (24%) 12 (24%)
Spanish

13 (26%) 13 (26%)
Age groups
18-29 9 (18%) 7 (14%)
30-39 7 (14%) 7 (14%)
40-49 2 (4%) 4 (8%)
50-69

7 (14%) 7 (14%)
Education levels
Less than a high school diploma degree 9 (18%) 10 (20%)
High school diploma degree 12 (24%) 12 (24%)
College degree

4 (8%) 3 (6%)
Construction experience
Less than 5 years 12 (24%) 10 (20%)
5-10 years 6 (12%) 8 (16%)
More than 10 years 7 (14%) 6 (12%)

5.1.3 Experimental Procedure
Before starting the training, all the participants were asked to complete a written informed consent.
Also, they completed surveys measuring their backgrounds and demographics. Specifically,

24

participants were asked to report their gender, age group, race, and the language they were
comfortable speaking. The demographic survey also measured participants' education level,
employment status, experience in the construction industry, and any experience using VR
technology or demolition robots. After answering the demographic survey, participants were asked
to complete a knowledge assessment survey including 32 multiple-choice questions measuring
participants' knowledge about remote-operating the demolition robot safely. Specifically,
questions were about all aspects of the robot, including components of the robot, risk zone, power
cable management, workplace inspection, safety checks, controller functions, starting up the
controller and the robot, arm positioning requirements, actions to be taken when demolishing,
safety precautions and actions. The content of the items was validated by receiving feedback from
an expert trainer to ensure that the critical knowledge needed to perform the robot safely was being
assessed. Cronbach’s alpha for the knowledge assessment was 0.82. This indicates a high level of
consistency among the items in the measure and suggests that the items are measuring the same
underlying construct. The knowledge assessment survey can be found in Appendix A.
Once participants completed the knowledge assessment, they were randomly assigned to one of
the two conditions: 25 participants were asked to complete the VR-based training, while the other
25 were asked to complete the in-person training. After both groups completed their training, they
were asked to retake the knowledge assessment survey to record the variations in their knowledge
of remote-operating the demolition robot.
In the final step of the experiment, both groups of participants were asked to take a performance
assessment. During this assessment, participants' operational skills and safety behavior were
evaluated while remote-operating the actual robot (Fig. 4). An expert trainer evaluated the content
of the assessments to ensure that all the crucial points were included. Participants' operational skill
performance and safety behavior were rated based on the criteria by the expert trainer on a scale
from 1 to 3 (1 = failed, 2 = done to an extent, 3 = perfectly done). The expert trainer also
qualitatively measured participants' performance in remote-operating the robot. The trainer wrote
a brief observation of each assessment that described the positive and negative aspects of the
participants' safety behavior and operational skills in working with the robot.
Participants were first asked to start the robot and run pre-start-up safety checks (e.g., hydraulic
oil level, oil leakage, cable position). After starting the controller and the robot, they moved it in
the trajectory indicated on the ground. Different objects were put in the environment to challenge
participants' skillset and safety behavior in moving the robot. Participants have to move the robot
while keeping a safe distance from surrounding objects and safely orient the arm system to avoid
the robot's tilting. One of the safety concerns is that the worker should look at the robot during
remote operations to prevent accidents. Therefore, the worker must know the controller's function
without needing to look at it. At the end of the route, participants were then asked to demonstrate
how to position the robot and its arm system to demolish a concrete block based on the operational
skills and safety points they had learned during their training. There were steel plates simulated as
a concrete block so participants could position the robot and demonstrate the demolition process.
The smoothness of using the controller's levers is another critical factor in remote operation. The
worker needs to push the levers smoothly to move the robot's arm system faster. After the
demonstration, participants were asked to move the robot in reverse to the start. The reliability for
the measure of safety behavior in remote-operation was 0.81 while the reliability for operation
skills was 0.77 in Cronbach’s alpha, both indicating acceptable consistency among measure items.

25

Fig. 5: Performance evaluation
5.1.4 Analysis
To answer research questions, the analyses relied on quantitative and qualitative approaches that
were dominant by the former methods. Johnson et al. define quantitative dominant mixed methods
research as “the type of mixed research in which one relies on a quantitative, post-positivist view
of the research process, while concurrently recognizing that the addition of qualitative data and
approaches are likely to benefit most research projects” [111]. Hence, while the analysis relies
heavily on the quantitative data collected from the experiment, the qualitative data primarily
provides valuable context for the findings.
5.1.4.1 Quantitative Analysis
The quantitative data collected were used to understand the impact of VR-based training on the
traditional in-person training method on three dependent variables: knowledge acquisition, safety
behavior, and operational skills while remote-operating the demolition robot. For knowledge
acquisition assessment, a 2 × 2 mixed factorial ANOVA with time (pre-vs. post-training) as the
within-subject factor and training type (VR-based training vs. in-person training) as the between-
subject factor is conducted. A mixed factorial ANOVA test compares the mean differences of a
dependent variable (e.g., knowledge level) between groups that have been split into two factors
(i.e., independent variables), each with two levels (training type: VR-based vs. in-person training,
time: pre- vs. post-training). This test aims to understand the effect of the two independent
variables by calculating the probability (p-value) of incorrectly rejecting the null hypothesis. The
null hypothesis in this test is that there is no statistically significant difference in terms of the
change in the dependent variable (e.g., knowledge) from pre-to-post training between VR-based
training participants and in-person training participants. It is vital to mention that normalized gain
has been selected in comparing the change in dependent variables from pre-to-post training. Since
the scores have upper limits, using raw changes does not account for the fact that the group with
lower pre-test ratings have more to gain than the group with higher pre-test rating. Therefore, the
normalized gain, independent of training type and pre-test ratings, provides a less biased
comparison between VR-based and in-person trainee’s knowledge level changes.
Also, independent sample t-tests with training type (VR-based training vs. in-person) as the
independent variables for each operational skill and safety behavior outcome are conducted. This

26

statistical test provides the probability of incorrectly rejecting the null hypothesis (no statistically
significant differences in measures between VR-based and in-person trainees). In quantitative
studies, it has been recommended to report both substantive significance (effect size) and statistical
significance (p-value). Therefore, this study reported both the p-value and effect size. It is critical
to provide effect sizes, i.e., the magnitude of the differences between groups because merely
providing p values can only inform whether an effect exists or not but will not reveal the size and
strength of the effect. Additionally, it has been argued that statistically significant results can be
achieved using a large sample size, whereas effect size is independent of sample size; thereby, it's
useful to show the size of a difference between two measures. Cohen’s (1988) guidelines have
been used in this study, which suggest (d =0.2) as small, (d = 0.5) as a medium, and (d ≥ 0.8) as a
large effect size. Cohen’s d is an appropriate effect size for the comparison between two means
which considers both the deviations and variations.
Moreover, additional tests are conducted to check for moderation by demographic factors: in
separate mixed ANOVAs, moderation by 1) language (Spanish vs. English), 2) age, 3) level of
education, and 4) experience in the construction industry are tested.
5.1.4.2 Qualitative Analysis
The analysis of the qualitative data collected in this study (specifically, the written observations
from the expert trainer) relied on a grounded theory approach articulated by Glaser and Strauss
[112]. Because grounded theory intentionally situates all findings and theory within collected data
(as opposed to applying theoretical frameworks from other sources), this methodological approach
produced a detailed understanding of the impacts of training methods on participants’ performance
in this specific context [113]. Once the qualitative data was collected, each team member coded
the whole data separately through an iterative and open process. This process began by employing
a descriptive coding technique to produce a set of themes that the expert trainer discussed across
the entire collection of observations [114]. After this first coding round, researchers engaged a
pattern coding strategy to condense the initial themes into analytical units and further illustrate any
patterns within the data [115]. To ensure Inter-Rater Reliability (IRR), two research team members
have coded the qualitative data separately and the agreement was measured using kappa statistics.
The IRR value obtained was greater than 0.81, indicating a satisfactory level of agreement.
5.2 Results
5.2.1 Quantitative analysis: VR-based training’s impact on workers’ knowledge,
operational skills, and safety behaviors
As indicated in Table 3, workers’ knowledge in both training programs increased after completing
the training. The average knowledge gain was greater among those who completed the VR-based
training (60.22%) than those who completed in-person training (39.55%) (F(1,47) = 18.36, p <
0.001, Cohen’s d > 1.0) (Fig. 5). Of all the background indicators, only the moderating effect of
age approached significance (F (3,43) = 10.18, p = 0.08; all other were nonsignificant Fs < 0.31,
ps > 0.58).
Table 3
Means and Standard Deviations (SD) of knowledge assessment based on individual differences

27

Measures VR-based training

In-person training
Before After Before After
Language

English 9.26 (8.57) 74.74 (9.37) 8.11 (9.20) 55.73 (16.69)
Spanish

10.16 (9.98) 65.11 (15.24) 14.75 (14.63) 46.87 (17.44)
Age groups

18-29

5.99 (6.10)

75.70 (10.21)

9.84 (10.11)

60.71 (15.18)
30-39 12.63 (11.28) 72.32 (11.04) 9.93 (14.75) 50.45 (13.55)
40-49 10.93 (11.06) 67.19 (17.56) 21.98 (16.03) 53.91 (14.39)
50-69

11.46 (10.21) 59.37 (3.95) 9.04 (9.73) 40.63 (15.73)
Education levels

Less than a high school diploma degree

12.54 (10.45)

64.45 (14.75)

9.78 (13.17)

40.94 (13.61)
High school diploma degree 9.20 (8.83) 69.27 (10.48) 13.87 (13.15) 56.25 (18.08)
College degree

5.56 (7.11) 82.82 (12.10) 8.43 (9.68) 64.59 (4.77)
Experience groups

Less than 5 years 12.01 (10.31) 69.27 (15.41) 7.26 (6.02) 44.69 (20.63)
5-10 years 6.39 (6.47) 73.44 (12.77) 11.00 (10.18) 57.03 (11.66)
More than 10 years 8.41 (8.83) 67.71 (10.39) 21.50 (19.03) 57.81 (13.65)

28

Fig. 6: Workers’ average score on the knowledge assessment
Analyses of the participants’ safety behavior assessment are presented in Table 4. Results indicate
that VR-based training participants (mean rating: 2.60) have significantly better safety behavior in
operating the robot than in-person training participants (mean rating: 2.30) (t(47) = 3.985, p <
0.001, Cohen’s d > 1.0) (Fig. 6). Similar to the knowledge assessment, of all the background
indicators, only the moderating effect of age approached significance (F(1,49) =, p < 0.001; all
others were nonsignificant Fs < 0.29, ps > 0.36).

Table 4
Means and Standard Deviations (SD) of safety behavior assessment based on individual differences
Measures

VR-based training In-person training
Language

English 2.62 (0.16) 2.40 (0.29)
Spanish

2.57 (0.25) 2.20 (0.28)
Age groups

18-29

2.60 (0.16)

2.39 (0.21)
30-39 2.55 (0.18) 2.36 (0.36)
9.70
69.92
11.58
51.12
0
10
20
30
40
50
60
70
80
Mean Rating of Knowledge Assessment
VR-based training Hands-on training
Before Experiment
After Experiment

29

40-49 2.73 (0.11) 2.35 (0.21)
50-69

2.58 (0.32) 2.11 (0.31)
Education levels

Less than a high school diploma degree

2.53 (0.26)

2.19 (0.28)
High school diploma degree 2.61 (0.20) 2.37 (0.30)
College degree

2.68 (0.08) 2.36 (0.30)
Experience groups

Less than 5 years 2.57 (0.21) 2.19 (0.33)
5-10 years 2.62 (0.19) 2.40 (0.30)
More than 10 years 2.61 (0.25) 2.37 (0.20)

Fig. 7: Mean rating of safety behavior assessment

Analyses for the operational skills assessment are presented in Table 5. Similar to safety behavior,
VR-based training participants (mean rating: 2.65) have showed significantly better skill sets than
2.60
2.30
0.00
0.50
1.00
1.50
2.00
2.50
3.00
Mean Rating of Safety Behavior
VR-based training Hands-on training

30

in-person training participants (mean rating: 2.29) (t(47) = 5.11, p < 0.001, Cohen’s d > 1.0) (Fig.
7). None of the demographic variables significantly moderated this effect (all Fs < 1.15, ps > 0.29).

Table 5
Means and Standard Deviations (SD) of operational skills assessment based on individual differences
Measures

VR-based training In-person training
Language

English 2.63 (0.15) 2.40 (0.30)
Spanish

2.66 (0.16) 2.21 (0.27)
Age groups

18-29

2.62 (0.15)

2.39 (0.22)
30-39 2.59 (0.15) 2.37 (0.34)
40-49 2.75 (0.07) 2.34 (0.23)
50-69

2.63 (0.18) 2.10 (0.31)
Education levels

Less than a high school diploma degree

2.65 (0.15)

2.19 (0.28)
High school diploma degree 2.63 (0.19) 2.39 (0.31)
College degree

2.60 (0.09) 2.37 (0.31)
Experience groups

Less than 5 years 2.65 (0.12) 2.19 (0.34)
5-10 years 2.65 (0.19) 2.40 (0.31)
More than 10 years 2.66 (0.20) 2.36 (0.19)

31

Fig. 8: Mean rating of operational skills assessment

5.2.1 Qualitative analysis: Observations on workers’ performance
Qualitative results from the expert trainer’s observations on participants’ performance during
remote operation are presented in Table 6. The analysis results identified three themes in the
trainer’s observations: pre-start-up checks, using the controller, and moving the robot during
remote operation. Table 6 presents each theme with examples of the trainer’s comments and the
number of positive and negative comments participants received based on their training type. The
trainer did not comment on some of the themes for a few participants.
The “pre-start-up checks” theme relates to the sequence of safety points the participant needs to
check before starting the robot. It was the first task that participants had to do regarding safety
behavior during the performance assessment. The trainer commented on how participants
remembered the steps confidently and retained knowledge about the checklist. The results indicate
that VR-based training participants received more positive comments from the trainer in
remembering and performing the safety checklist than the in-person training participants. Many
VR-based training participants performed safety checks without hesitation, or the trainer prompted
them.
The “using the controller” theme relates to participants’ knowledge of the controller’s functions
and smoothly using the controller (levers and buttons) without looking at it. The results show that
VR-based training had a better performance in remembering the controller’s functions and using
it without hesitations than in-person training participants. The trainer’s feedback indicates that VR-
based training participants smoothly moved the controller’s levers and used the functions without
hesitation.
The theme of “moving the robot” refers to participants’ performance in moving and orienting the
robot effectively and safely. Similarly, VR-based training participants showed better performance
following safety guidelines and operational skills in moving the demolition robot than in-person
2.65
2.29
0
0.5
1
1.5
2
2.5
3
Mean Rating of Operational Skills
VR-based training Hands-on training

32

training participants. Based on the trainer’s comments, most of the VR-based training participants
remembered the details of the safety points and positioned themselves for better vision during
remote operation, while some other participants made a few mistakes and needed coaching.
Two research team members independently coded for the three identified themes in the trainer’s
observations. Interrater reliability was assessed using the Kappa statistic before both research
members reconciled coding. The Kappa statistics calculated for both members were > 0.81, a
nearly perfect size, suggesting that the identified themes have high reliability.

Table 6
Themes, examples, and numbers of participants who got positive and negative comments based on their
training type in the qualitative analysis
Themes

Example
VR +
VR -
in-person +
in-person -

Pre-start-up checks
- Participant did excellent in
remembering all the pre-start checks
- Participant at first could not remember
any of the pre-start checks

16

5

11

9

Using the controller
- Never looked at controller,
remembered controls pretty much w/o
hesitation
Repeatedly could not remember the
functions

16

6

12

6

Moving the robot
- Remembered the details of the safety
points mostly w/o hesitation.
- Same mistakes again and again;
Doesn’t understand the cautions and
doesn’t listen well to corrections

15

7

11

7

5.3 Discussion
5.3.1 Knowledge Acquisition
Based on the analyses and results from the experiment, the success of VR-based training regarding
knowledge acquisition can be attributed to two factors: the effective use of learning theory within
the curriculum design and the nature of the knowledge being learned. The study's findings support
previous studies that the use of VR-based training for learning heavily relies on visual and spatial
information, and remote operating a construction robot also depends on this information [116-
119]. Specifically, effective and safe remote operation of a demolition robot relies on workers

33

seeing where the robot is and its various elements in relation to other elements on the job site. VR-
based training provides a safe and effective way to present essential information about the robot's
components, movement range, risk zones, workplace inspection, safety management, and
demolition strategies. Due to safety, cost, and maintenance concerns, the trainer has limitations in
illustrating all the critical information in the in-person training. Therefore, workers only verbally
acquire part of their knowledge in the traditional in-person training. Communication skills between
trainers and trainees limit the effectiveness of transferring knowledge verbally. VR-based training
can overcome the limitations of in-person training, such as language barriers and the need for
trainers and actual robots in all training sessions. However, developing VR-based training can be
time-consuming and costly, and running the training may require substantial computing power. As
VR technology becomes more common, the costs and limitations of VR-based training are likely
to decrease, making it an increasingly attractive option for a wide range of learning situations.
On the other hand, the alignment between the knowledge of remote-operating a demolition robot
and VR-based training as a pedagogical model does not guarantee a successful training process.
According to Wikens, Chavez, and Bayona, the design of the VR program is the primary
determinant of its success [120,121]. To this end, the VR-based training in this study was heavily
influenced by adult learning theory (andragogy) and extant research in VR education to create this
specific set of learning modules (based on the exact contents of in-person training) [122].
Regarding the adult learning theory, the study's focus was solely on integrating the theory of
andragogy in VR-based training without modifying the existing in-person training method in the
industry. For instance, Trainees can learn at their own pace in the self-directed VR-based training,
review learning materials, and perform tasks multiple times. The trainees are also requested to
perform tasks at the end of each module to review the learning module if they did not pass the test.
This represents a distinct benefit over in-person training, where individuals must work at the
instructor's pace. Additionally, construction workers as adult learners have a wealth of life
experiences, which serve as a resource for designing the tasks in VR learning modules. For
example, asking the worker to manage the power cable based on the learning content helps the
trainee draw on his/her previous experiences to improve the learning process. Workers require
learning needs that are closely related to their job goals due to the advancement of automation on
construction sites. Adult learners in the construction industry are problem-centered and are
interested in directly applying the acquired skills and knowledge. The VR-based training,
therefore, provides skills and knowledge that are immediately applicable to the remote operation
of robots. Moreover, immediate feedback based on their performance can help trainees who do not
have all the characteristics of an adult learner and need direction throughout the training. To this
end, the success of VR in this study is attributed to the use of research on VR education and adult
learning theory (andragogy) in the design of this training program. As a result, it is recommended
that future design efforts draw from a similar research base.
5.3.2 Operational skills & safety behavior
In accordance with previous research on virtual reality (VR) use in vocational training, particularly
in the construction industry, this study's results strongly support the effectiveness of VR-based
training for improving construction workers' operational skills and safety behavior while working
with remote-operated construction robotics. Compared to those who received in-person training,
VR-based training participants demonstrated significantly better operational skills and safety
behavior in remote-operating robots. Therefore, VR is a promising alternative for future training

34

programs and has the potential to enhance workplace safety. In addition, VR-based programs could
assist in designing and developing construction robotics by enabling end-users to interact with
several prototypes in a simulated environment. Moreover, VR training has several advantages over
in-person training, such as reducing the barriers associated with cost, scheduling, and accessibility.
VR training also promotes consistency, quality, and reliability across trainers, providing all
trainees with the same learning content, which is not guaranteed in the in-person training [123-
125]. Additionally, VR-based training minimizes the need for a specialized training workforce and
can reach trainees in locations where trainers and/or equipment are unavailable. Thus, VR is a
valuable option for the future of construction safety and skill training due to its consistency,
scalability, and accessibility.
The study's results provide findings on the success of VR-based training on two affordances
associated with VR. Firstly, the training's alignment with previous research on learning within VR
has shown that participants are more comfortable taking risks and making mistakes, and “failing”
without the fear of severe consequences. This makes VR-based training different from in-person
training, even though the learning content is the same. Secondly, training with a demolition robot
is one of the primary reasons for using VR, as learning with the actual robot can be hazardous,
costly, and stressful. The VR system, however, provides a less dangerous and stressful learning
environment, and it holds just as much motivation for the learner as working with the machine
(assuming that the motivation involves getting to work with new technology in general and not a
specific machine). The alignment between learning with the demolition robot and VR-based
training models applies to nearly all construction robots since most rely on visual/spatial
information, and most are dangerous or expensive to operate. As a result, these findings position
VR-based training as an effective tool for enhancing operational skills and safety behavior for
construction workers who want to learn how to interact with construction robots.
The hesitation and smoothness associated with working with the actual robot, a finding associated
with the theme of "using the controller" and "moving the robot" in robotic remote operation that
emerged within the qualitative analysis, points to this affordance. Since VR trainees did not feel
as nervous working within this context, they could focus on skill and safety behavior development
without worrying about damaging the robot. Second, the developed VR-based training program
allowed workers to learn with the actual controller they would use. As Bhoir & Esmaeili [126]
contend, the hesitation associated with adopting VR technology within training programs often
stems from the trainer's assumption that VR does not provide a realistic experience for trainees.
However, since the construction site was in various hazardous working scenarios and the actual
controller with realistic maneuvering of the robot was simulated, these points help improve that
experience and more strongly align VR with real-world experiences involving the robot, leading
to a high rate of transfer between contexts. The findings of this study also build on Mekacher's
[127] assertion that VR-based training, while covering the exact contents of in-person training,
allows for a certain amount of experimentation and opportunities for "failure" not available in in-
person experiences. Similarly, the program also relied on the established idea within VR education
that trainees can experience the consequences of their decisions without causing irreparable
damage or harm. For example, they could move the robot onto a dangerous steep surface and
experience the robot tilting and falling without breaking the actual robot. This further builds on
extant research into safety training for construction workers that uncovered the connection
between knowledge acquisition and the ability to "fail" provided by VR [128, 129]. Although VR-
based training allows the trainee to practice with the robot in the VR environment, this training

35

can have physical side effects on trainees, such as dizziness, eyestrain, or nausea. In order to
prevent participants from experiencing motion sickness, they were asked to take off the HMD after
each learning module (for 5–10 min), and they did not move in the VR environment without
physically moving themselves using the treadmill, as both of these are standards for minimizing
motion sickness.
5.4. Limitations
While this study presents VR-based training implications for knowledge, operational skills, and
safety behavior development in construction robotic remote operation, some limitations do exist.
Although VR-based training has advantages in terms of safety, scalability, and overcoming
language barriers, developing and running VR-based training requires significant effort and
computing power. Besides, using VR-based training may have physical side effects. In future
studies, researchers might compare the costs of implementing VR-based training compared to in-
person training. While in-person training has costs such as potential workers injuries, potential
damage to the robot, robot maintenance, hiring professional trainers, and disturbance of the work
on construction site, VR-based training also has costs such as developing the virtual environment,
required computational power, required hardware, and potential physical side effects (e.g., fatigue,
dizziness, motion sickness).
Moreover, due to the limited resources in recruiting construction workers, renting the actual robot,
and hiring a professional trainer for the experiment during the pandemic, only one professional
trainer was hired to evaluate trainees’ performance with the actual robot, both quantitatively and
qualitatively. Using a second evaluator can ensure having inter-rated agreement on the quantitative
scores and qualitative assessment in future studies. Besides, participants’ perceptions of VR-based
training could not be collected due to the time limitation. It is recommended future studies collect
feedback from participants on their experience with VR-based training.
5.5. Conclusion
The study in this chapter makes an essential contribution to existing research on VR-based training
within the construction industry. It is built on previous studies that examined the use of VR within
safety training for construction workers by shifting the context toward technology and robotics
training for the same population. Findings from this study indicate that VR-based training in this
context was associated with a more significant increase in knowledge acquisition, operational
skills, and safety behavior compared to in-person training with the machine. In doing so, VR-based
training can be positioned as a valuable tool in developing workers’ knowledge, ability, and safety
behavior to implement robotics in construction. In addition, this study positions VR-based training
as an equally effective pedagogical model compared to hands-on or in-person training, an insight
that produces multiple (and substantial) implications for improving human-robot interaction using
VR, especially in the construction field. First, VR-based training reduces the risk to workers and
machinery associated with in-person training since trainees cannot hurt themselves or damage the
robot if they make a mistake. Second, VR-based training can potentially significantly reduce the
costs associated with training. While VR technology may not be universally accessible at this point
(both in terms of physical access and cost), in-person training requires (at the very least) rental,
transportation, and trainer fees for every training session. Since the cost of VR technology
continues to decrease, this approach to training provides an inexpensive, on-demand, and

36

individualized alternative to traditional approaches to training. To this end, VR represents a safe
and accessible format for construction training, one that the industry should further develop as the
field increasingly adopts robots in real-world applications.
While this study holds several significant implications for knowledge, operational skill, and safety
behavior development in human-robot interaction with a specific focus on construction research,
some limitations exist. This study relies on data generated by a limited number of participants.
While the differences between training conditions were quite large, the effects were significant
even with the small sample size; this study might have been underpowered to test for moderation
(e.g., language, age groups, educational level). Besides, it is established that knowledge of robotic
remote operation was improved more by VR-based training than in-person training. However,
knowledge acquisition was studied by measuring it before and immediately after the training.
Future studies are needed to investigate whether workers are maintaining such knowledge gains in
the long run and the effectiveness of VR-based training on knowledge acquisition, operational
skills, and safety behavior while working with robots with a larger sample size. Beyond
generalizability issues, using a larger sample size also allows for more detailed investigations into
individual differences to more accurately determine when and why VR works as a pedagogical
model.

37

Chapter 6. Impact of VR-Based Training on Human–Robot Interaction for
Remote Operating Construction Robots
The study in this chapter answers to the second research objective by investigating the impact of
VR-based training on human-related factors in Human-Robot Interaction (HRI) such as trust in the
robot, robot operation self-efficacy, Situational Awareness (SA), and Mental Workload (MWL) in
the construction industry domain. The study compares the results between construction workers
who have completed a VR-based training and workers who have completed a traditional in-person
training with the same and in-parallel content. The study’s methodology, including in-person
training, participants, experimental procedure, and analysis methods, are presented in section 6.1.
Also, the results and findings are discussed in sections 6.2 and 6.3. Finally, sections 6.4 and 6.5
presents the limitations, summary, and directions for future research steps.
6.1 Methodology
6.1.1 Experimental Procedure
The participants of this study were the same as the study in the previous chapter. Fifty construction
workers, all over 18 years old with varying degrees of experience in the construction industry,
were recruited. participants’ backgrounds and demographics were measured by a set of survey
items. Specifically, participants were asked to report their gender, age group, race, and the
language they were comfortable speaking. Moreover, the survey measured participants’ education
level, employment status, and experience in the construction industry. Participants also reported if
they have any experience in using VR or demolition robots.
Participants were randomly assigned to one of the two conditions: 25 participants were asked to
complete the VR-based training, while the other 25 were asked to complete the in-person training.
The in-person training in this study is the same training method from the previous chapter. There
was one experiment with several parameters to study. While the previous chapter focused on
knowledge acquisition, safety behavior, and operation skills, this study has focused on human-
related factors in HRI. Before starting either training, participants were required to complete two
surveys that measured trust in the robot and robot operation self-efficacy. The measure of trust in
the robot was modified from the automated trust scale to measure participants’ attitudes toward
interaction with the robot specifically. The modified survey used in this study has used items and
words proposed in the automated system scale [130]. Modifications were made to adapt the survey
to the demolition robot. The modified survey consists of 21 sentences about participants’ trust in
the robot's reliability, integrity, and safety and their beliefs about the robot’s influence on their
careers. Participants rated the sentences on a 5-point Likert scale that ranges from completely
disagree to completely agree. For example, participants were asked to rate the sentences such as
“I can trust the robot,” “The robot is reliable,” and “The robot provides safety/security” with a
number from 1 to 5 indicating their disagreement (1) or agreement (5) with each sentence. The
robot operation self-efficacy survey was modified from the validated robot-use self-efficacy scale.
It consisted of two sentences (“I am confident in the robot” and “I feel confident around the robot”)
measuring participants’ self-efficacy and confidence in their ability to remote operate the robot.
As with the trust in the robot survey, participants rated the sentences on a 5-point Likert scale
ranging from completely disagree to completely agree.

38

Once the surveys were completed, participants began their assigned training. After completing
their training, both groups were asked to retake the trust in the robot and robot operation self-
efficacy surveys. Then, participants were asked to complete a performance assessment, remote
operating the actual robot, in which each worker’s SA and MWL were assessed. First, they had to
start the robot, running the sequence of pre-start-up safety checks (e.g., hydraulic oil level, oil
leakage, cable position). After starting the controller and the robot, participants moved the robot
in the direction indicated on the ground. They had to use the controller’s function and follow the
safety guidelines to move the robot efficiently and safely. Participants then demonstrated the
demolition position of the robot’s arm system on a simulated concrete block. After showing the
demolition process, participants were asked to move the robot in reverse to the starting position
and go through the complete shutdown procedure.
To measure situational awareness, a modified version of the SAGAT was employed. While
moving the actual robot to the simulated concrete block in the performance assessment session,
participants were asked to pause the remote operation and answer the SA survey. This survey
consisted of eight questions evaluating the trainee’s perception, comprehension, and projection. In
the perception section, participants answered questions related to the perception of the cable’s
location relative to the robot, the outriggers, and sharp edges (e.g., “Is the cable behind the robot?”,
“Is the cable close to the outriggers?”, and “Is the cable close to sharp objects?”). In the
comprehension section, the trainer asked participants if the robot had sufficient distance from
various objects and if the angles between the arms were in the correct range (e.g., “Is the distance
between the robot and the element to be demolished sufficient for a proper operation?”, “Are the
angles between the arms of the machine in the correct position?”). Finally, in the projection section,
participants discussed whether the robot proceeded to the correct position, and the trainer observed
whether the arm trajectory hit the operator or any objects (e.g., “Is the robot proceeding to the right
position?”, “Will the arm trajectory hit the operator?”, “Will the arm trajectory hit any objects?”).
The expert trainer rated the participants’ answers. Finally, to measure participants’ MWL,
participants were asked to complete a NASA-TLX survey. After the remote operation of the actual
robot, participants were asked to complete the MWL survey. In this survey, participants reported
their mental demand, physical demand, temporal demand, performance, effort, and frustration
level while remote operating the robot based on a Likert scale that ranges from very low to very
high (e.g., “How much mental activity was required to perform your job (thinking, deciding,
calculating, remembering, looking, searching, etc.)?”). The NASA- TLX asks the subject to use a
rating between 0 and 100 for a group of questions in each of these subscales, and these ratings are
used to determine the weights during the comparisons of the level of importance the subject
assigned to each subscale [86, 87].
6.1.2 Analysis
The data collected, both pre-training and post-training, were used to understand the impact of VR-
based training compared to in-person training on four dependent variables: trust in the robot, robot
operation self-efficacy, SA, and MWL. For each of the first two outcomes, a 2 × 2 mixed factorial
ANOVAs is conducted with time (pretraining versus post-training) as the within-subject factor
and training type (VR-based training versus in-person training) as the between-subject factor.
Additionally, independent sample t-tests with training type (VR-based training versus in-person)
as the independent variables for each of the latter two outcomes are conducted. Moreover,
additional tests were run to check for moderation by demographic factors: in separate mixed

39

ANOVAs, moderation by (1) language (Spanish versus English), (2) age, (3) level of education,
and (4) experience in the construction industry were tested.
6.2 Results
Analyses for the trust ratings (range: 0-5) are presented in Table 7. The time (pre- and post-
training) by training interaction is statistically significant for trust in the robot (F(1,47) = 25.94, p
< 0.001, Cohen’s d > 1.0), with the VR-based training increasing trust more (1.38) than the hands-
on training (0.62). The reliability of this scale (Cronbach’s alpha) was .91. None of the
demographic variables significantly moderated this effect (all Fs < 1.15, ps> .29).

Table 7. Means and Standard Deviations (SD) of trust in the robot based on individual differences
Measures VR-based training
Mean (SD)
Hands-on training
Mean (SD)
Before After Before After
Language

English 2.79 (0.37) 4.33 (0.39) 2.95 (0.27) 3.45 (0.34)
Spanish

2.83 (0.31) 4.06 (0.49) 2.80 (0.39) 3.35 (0.39)
Age groups

18-29

2.78 (0.32)

4.27 (0.40)

3.04 (0.32)

3.39 (0.24)
30-39 2.83 (0.35) 4.23 (0.44) 2.77 (0.24) 3.50 (0.25)
40-49 2.83 (0.25) 4.20 (0.83) 2.75 (0.58) 3.58 (0.61)

40

50-69

2.83 (0.45) 4.04 (0.63) 2.88 (0.26) 3.20 (0.36)
Education levels

Less than a high school diploma
degree

2.82 (0.37)

4.05 (0.50)

2.75 (0.44)

3.47 (0.43)
High school diploma degree 2.91 (0.28) 4.25 (0.54) 2.99 (0.23) 3.32 (0.32)
College degree

2.49 (0.32) 4.32 (0.48) 2.86 (0.25) 3.49 (0.29)
Experience groups

Less than 5 years 2.76 (0.38) 4.11 (0.46) 2.96 (0.40) 3.49 (0.49)
5-10 years 2.95 (0.25) 4.31 (0.76) 2.86 (0.28) 3.40 (0.26)
More than 10 years 2.78 (0.35) 4.25 (0.34) 2.84 (0.31) 3.44 (0.28)

The analyses for the robot operation self-efficacy ratings (range: 0-5) are presented in Table 8. The
time by training interaction is statistically significant for self-efficacy (F(1,47) = 10.43, p < 0.002,
Cohen’s d > 1.0), with VR-based training increasing self-efficacy more (1.62) than the hands-on
training (0.74). The reliability of this scale (Cronbach’s alpha) was .69. Again, none of the
demographic variables significantly moderated this effect (all Fs < 3.22, ps> .14).

Table 8. Means and Standard Deviations (SD) of robot operation self-efficacy based on individual
differences

41

Measures VR-based training
Mean (SD)
Hands-on training
Mean (SD)
Before After Before After
Language

English 2.96 (0.66) 4.50 (0.56) 3.08 (0.42) 3.50 (0.60)
Spanish

2.63 (0.71) 4.33 (0.75) 2.58 (0.91) 3.61 (0.62)
Age groups

18-29

3.05 (0.63)

4.55 (0.40)

2.86 (0.85)

3.42 (0.45)
30-39 2.71 (0.56) 4.59 (0.44) 2.64 (0.85) 3.79 (0.39)
40-49 3.00 (0.10) 4.50 (0.83) 2.63 (0.83) 3.88 (0.85)
50-69

2.42 (0.92) 4.33 (0.63) 3.07 (0.19) 3.79 (0.69)
Education levels

Less than a high school diploma
degree

2.56 (0.50)

4.31 (0.50)

2.45 (0.44)

3.65 (0.67)

42

High school diploma degree 2.88 (0.77) 4.41 (0.82) 3.08 (0.23) 3.50 (0.60)
College degree

3.00 (0.82) 4.62 (0.48) 3.00 (0.25) 3.50 (0.50)
Experience groups

Less than 5 years 2.79 (0.58) 4.45 (0.49) 2.90 (0.70) 3.45 (0.68)
5-10 years 3.08 (0.66) 4.25 (0.98) 2.93 (0.42) 3.43 (0.41)
More than 10 years 2.50 (0.89) 4.50 (0.63) 2.83 (0.93) 4.00 (0.54)

Analyses for situational awareness measurement (range: 0-1) are presented in Table 9. The results
reveal that VR-based training participants (mean SA rating = 0.98) have significantly greater
situational awareness compared to participants who have completed hands-on training (mean SA
rating = 0.86) (t(47) = 3.449, p < 0.001, Cohen’s d > 1.0). None of the demographic variables
significantly moderated this effect (all Fs < 1.15, ps > .29).

Table 9. Means and Standard Deviations (SD) of situational awareness assessment based on individual
differences
Measures

VR-based training
Mean (SD)
Hands-on training
Mean (SD)
Language

English 0.99 (0.04) 0.85 (0.22)

43

Spanish

0.97 (0.06) 0.87 (0.09)
Age groups

18-29

0.98 (0.04)

0.89 (0.09)
30-39 0.98 (0.05) 0.91 (0.06)
40-49 1.00 (0.00) 0.91 (0.06)
50-69

0.96 (0.06) 0.75 (0.27)
Education levels

Less than a high school diploma
degree

0.95 (0.06)

0.86 (0.09)
High school diploma degree 0.99 (0.03) 0.85 (0.22)
College degree

1.00 (0.00) 0.88 (0.13)
Experience groups

Less than 5 years 0.97 (0.05) 0.80 (0.22)

44

5-10 years 0.97 (0.05) 0.92 (0.06)
More than 10 years 0.97 (0.05) 0.88 (0.13)

Finally, the analyses for the MWL during the teleoperation of the robot are presented in Table 10.
Although VR-based training participants (mean MWL rating (range: 0-100) = 45.20) have shown
lower mental workload than hands-on training participants (mean MWL rating = 53.73), there was
no significant difference between VR-based and hands-on training (t(1,47) = 1.77, p = 0.915,
Cohen’s d > 1.0). Cronbach’s alpha for this scale was .77, indicating good reliability. Again, none
of the demographic variables significantly moderated this effect (all Fs < 3.22, ps > .14).

Table 10. Means and Standard Deviations (SD) of mental workload assessment based on individual
differences
Measures

VR-based training
Mean (SD)
Hands-on training
Mean (SD)
Language

English 41.04 (21.49) 46.39 (10.76)
Spanish

49.38 (8.28) 60.51 (19.51)
Age groups

18-29

47.13 (10.76)

41.07 (10.39)
30-39 40.83 (27.83) 54.99 (5.79)

45

40-49 39.17 (2.36) 55.83 (18.27)
50-69

49.45 (8.00) 63.93 (23.65)
Education levels

Less than a high school diploma
degree

48.33 (13.51)

58.25 (22.46)
High school diploma degree 44.44 (19.01) 51.94 (12.70)
College degree

41.25 (16.42) 45.83 (13.09)
Experience groups

Less than 5 years 47.78 (8.36) 51.25 (10.30)
5-10 years 38.33 (21.63) 55.83 (19.31)
More than 10 years 46.96 (23.50) 58.89 (23.40)

6.3 Discussion

The study in this chapter aimed to understand the impact of VR-based training on construction
workers’ trust in the robot, robot operation self-efficacy, SA, and mental workload as compared
in-person training approach. The results showed that VR-based training had significantly impacted
the first three measures compared to traditional in-person training. This section provides a
discussion of the significance of these findings.

46

6.3.1 Trust in the Robot and Robot Operation Self-Efficacy
The present study has provided evidence that using virtual reality (VR) technology for training
construction workers in remote operating robots can improve workers' trust in the robot and their
self-efficacy during robotic remote operation significantly compared to in-person training. One of
the key factors contributing to this success can be attributed to the immersive nature of the VR
environment, which allows trainees to familiarize themselves with the robot's functions and work
with it in various scenarios. This helped humans to gain trust in the robot by managing humans’
expectations of the robot's actions [131]. These results confirm that VR-based training can help
trainees focus on the information relevant to the training to gain confidence in using new
technology. Additionally, the accurate representation of the consequences of different strategies in
VR-based training enhances workers' self-efficacy, as seen in a prior study by Koppula et al [132].
However, developing VR-based training requires significant effort, time, computing power, and
cost, as it involves creating accurate simulations of the robot and various scenarios. Nonetheless,
these negative factors can be mitigated with the increasing availability and advancement of VR-
based applications. Additionally, developing VR-based training is a one-time effort, unlike
traditional in-person training, which requires an actual robot and a professional trainer for each
training session, as noted in previous chapters.
According to Autor et al., while some middle-skill jobs are at risk of full automation, others will
require workers to acquire a range of tasks to adapt to emerging technologies. The study findings
suggest that VR-based training has the potential to help construction workers overcome their fears
about using robots in the industry. Many workers are apprehensive that new robotic systems will
replace their jobs, particularly in the case of demolition robots that will substitute manual labor in
demolition sites. However, the study results indicate that VR-based training can increase workers'
trust in robots and their self-efficacy in robot operation, leading to greater acceptance of new
robots, including demolition robots, in the construction industry. This has significant implications
for enhancing HRI using VR.
The use of VR in training can motivate and attract construction workers to develop their vocational
skills and adaptability for future work in the construction industry. The VR-based training
environment provides various scenarios that present the abilities that a demolition robot provides
to the worker. The effectiveness of implementing robots in dangerous tasks, while delivering the
same learning outcomes as in-person training, affects the workers' trust in the robot. This limitation
is attributable to the fact that in-person training does not allow workers to practice dangerous tasks
with the robot due to ethical, financial, and safety concerns. Furthermore, given that construction
robots are not yet widespread, the training for the safe and effective use of these robots is not
standardized and varies significantly among instructors. Nonetheless, VR-based training presents
a consistent, efficient, and scalable approach to training in the construction industry.
This study supports Lee and See's suggestion that providing essential information about the
purpose and methods of implementing new technology in interactive settings can enhance trust in
the new technology [133]. Unlike in-person training, which restricts trainees from interacting with
the actual robot, VR-based training enables trainees to learn about the adoption of the robot in an
interactive context. During interaction with the robot, workers can observe its behavior and gain
knowledge of the underlying processes, which enhances their mental model of the robot and fosters
trust in automation. As a result, workers' trust in the robot and their self-efficacy in robot operation
increases significantly more in VR-based than in in-person training.

47

6.3.2 Situational Awareness and Mental Workload
The comparison of SA scores between VR-based and in-person training conditions demonstrates
that the VR-based training group had significantly higher SA than the in-person training group
when remote operating the demolition robot. Consistent with the findings on trust in the robot and
robot operation self-efficacy, the results suggest that the VR environment offers trainees greater
opportunities that contribute to higher SA scores. Unlike in-person training, where remote robot
operation is limited mainly due to safety concerns, VR-based training allows trainees to remote
operate the robot in various scenarios and apply different strategies. This advantage provides
opportunities for trainees to experience different situations and learn the consequences of incorrect
decisions while operating the robot remotely. For example, trainees can experience the impact of
ignoring power cable management during robot operation by losing the cable by putting it in
hazardous locations. Moreover, they can learn the consequences of improper positioning of the
demolition robot's arm system while moving the robot, which could result in tilting and failure. As
a result, VR-based training participants have a better understanding of the power cable position,
robot distance from surrounding objects and workers, and the demolition robot's trajectory during
remote operation. The study's results confirm that the use of immersive visualization techniques
in training can enhance workers' SA in complex and dynamic environments [134]. However, VR-
based training can also have physical side effects, such as dizziness, eyestrain, or nausea, among
its users. To mitigate these side effects, trainees can take breaks to remove the head-mounted
display, reducing the likelihood of experiencing adverse effects.
The present study also aimed to compare the mental workload experienced by construction
workers during VR-based and in-person training conditions. Although the NASA-TLX MWL
survey scores suggested that the VR-based training participants experienced a lower average MWL
than the in-person training participants, there was no significant difference between these two
conditions. The ability of VR-based training participants to practice with the robot in different
scenarios may have contributed to the lower MWL, but this was not reflected in the data. It is
important to note that VR-based training participants had not experienced the physical demands
of remote operating the demolition robot, as they were on a VR treadmill; therefore, they
experienced the physical demand and effort for the first time during the assessment, which may
have impacted their MWL. The study's small sample size may also have contributed to the lack of
significant differences in MWL between the two groups. Future studies should investigate the
impact of VR-based training on mental workload in larger sample sizes to draw more definitive
conclusions.
6.4. Limitations
While the current study highlights the implications of using VR-based training for human-related
factors such as trust in the robot, robot operation self-efficacy, SA, and MWL in robotic remote
operation in the construction industry, some limitations exist. There are differences in VR-based
and in-person training mechanisms, while some represent an important limitation of in-person
training. In-person training has practical constraints due to limited time and multiple workers in a
session, and workers cannot explore different strategies for remote operation due to safety risks.
In contrast, VR-based training offers more opportunities for workers to practice and safely explore
different aspects of operation without any risks. By providing VR equipment and computing
devices, trainees can experience the training individually and work with the robot for a more

48

extended period than the traditional training. Additionally, during in-person training, workers
cannot explore different strategies for remote operating the robot independently because it
represents a risk to the safety and the equipment. In contrast, VR-based training provides more
opportunities for workers to practice with the robot and safely explore different aspects of
operation without risk to safety or equipment. These are natural differences between the two kinds
of training and represent several reasons why VR-based training was suggested as a new training
method to study in the first place.
The goal of the current study was not to tease apart the different mechanisms by which VR-based
training has its effect but to investigate the impact of VR-based training compared to traditional
in-person training. Therefore, the limitation is that the study does not have experimental control to
test “why” [i.e., the mechanism(s) by which] VR training has better outcomes than in-person
training. Indeed, VR-based training presents possibilities for overcoming these limitations of
standard in-person training sessions, and this chapter wanted to harness the power of all these
natural differences between the two conditions. Hence, instead of having various VR conditions
that differ from in-person training on only one variable (thus would have better experimental
control), this experiment opted for only two conditions that differed in all ways VR training and
in-person training would naturally differ. Future research should investigate how VR-based
training improves outcomes over in-person training and, therefore, would need to isolate those
mechanisms experimentally. In these follow-up studies, the experimental conditions would be
better controlled (i.e., various VR conditions that differ from in-person training on only one
variable).
6.5. Conclusion
The study in this chapter investigated the impact of VR-based training on four human-related
factors (i.e., trust in the robot, robot operation self-efficacy, SA, and MWL) in remote operation
of a robot compared to traditional in-person training. While the advancement of construction
robotics can enhance productivity and safety in the construction industry, it also has brought about
new challenges. The unstructured and unpredictable nature of construction sites has hindered the
adoption of construction robotics. Moreover, sharing workspace between workers and robots in
dynamic and hazardous construction sites has introduced new safety concerns. Therefore, it is
crucial to enhance human-related factors such as trust in the robot, robot operation self-efficacy,
SA, and MWL while remote operating robots on construction sites to address new safety concerns
and facilitate the implementation of robotics in the construction industry. Despite the vast body of
research on the effectiveness of VR-based training in the construction industry, the impact of VR-
based training in building trust, self-efficacy, SA, and optimizing MWL in the remote operation
of construction robotics is not well studied.
Thus, immersive VR-based training was developed to study the impact of VR-based training on
these factors. Fifty construction workers were randomly assigned to complete either the VR-based
or in-person training. Construction workers were asked to complete trust in the robot and robot
operation self-efficacy surveys before and after completing their assigned training. In addition, a
professional trainer evaluated their SA during the remote operation of the robot. Finally, they
completed an MWL survey using the NASA-TLX measurement method immediately after the
remote operation of the actual robot.

49

The quantitative results show that VR-based training can significantly increase workers’ trust in
the robot and robot operation self-efficacy compared to a traditional training method such as in-
person training. Moreover, VR-based training participants have significantly more SA while
remote operating the construction robot. Although VR-based training participants had lower mean
ratings of MWL than in-person training participants, the analysis could not find any significant
difference in participants’ MWL between the two conditions in this study.
One of the key factors contributing to this success is the nature of the VR environment. The
accurate simulation and visualization of the robot and the construction site allowed the trainee to
work with the robot in various scenarios to understand the robot’s behavior in different tasks
clearly. VR-based training participants could find the opportunity to remote operate the robot in
different scenarios, implementing different strategies to experience the consequences without
exposure to danger. These findings produce multiple implications for improving HRI using VR,
especially in construction.

50

Chapter 7. Participants Matter: Effectiveness of VR-based Training on the
Knowledge, Trust in the Robot, and Self-Efficacy of Construction Workers and
University Students
This chapter addresses the third research objective by comparing the effectiveness of VR-based in
knowledge acquisition, trust in the robot, and robot operation self-efficacy on construction workers
versus construction engineering students. The study’s methodology, including participants,
experimental procedure, and analysis methods, are presented in section 7.1. Also, the results and
findings are discussed in sections 7.2 and 7.3. Finally, sections 7.4 and 7.5 presents the limitations,
summary, and directions for future research steps.
7.1 Methodology
7.1.1 Participants
A total of 50 individuals participated in this study. The study utilized a non-experimental design,
as there was no random assignment of participants to treatment groups. In an experimental design,
participants are randomly assigned to different treatment conditions, and the researcher
systematically manipulates the independent variables to determine their effect on the dependent
variables. Therefore, it is not possible to establish a causal relationship between the predictor and
the outcomes variables in this study. Instead, this study can be classified as an observational study
since the participants self-selected into the groups. The data from twenty-five construction workers
(24 males, one female) who experienced VR-based training in the previous chapters have been
used in this study. All the workers were over 18 years old with varying experience in the
construction industry. In addition, twenty-five graduate construction engineering students (21
males, four females) at the University of Southern California (studying either Master of Civil
Engineering with a focus on Construction Engineering or Advanced Design & Construction
Technologies) participated in the study. These students have participated voluntarily, responding
to the invitation email from the construction engineering graduate students' community. This
recruitment strategy aligns with the strategies used by empirical studies that have recruited students
as their sample population. Table 11 indicates the participants' demographic details based on their
age, language, educational attainment, and experience level in the construction industry. It is vital
to mention that recruited participants had no experience with the selected demolition robot.

Table 11
Demographics across samples
Demographics

Construction workers Construction students

Language

English 12 (24%) 25 (50%)

51

Spanish

13 (26%) 0 (0%)
Age groups
18-29 9 (18%) 25 (50%)
30-39 7 (14%) 0 (0%)
40-49 2 (4%) 0 (0%)
50-69

7 (14%) 0 (0%)

Education levels
Less than a high school diploma degree 9 (18%) 0 (0%)
High school diploma degree 12 (24%) 0 (0%)
College degree

4 (8%) 25 (50%)
Construction experience
Less than 5 years 12 (24%) 25 (50%)
5-10 years 6 (12%) 0 (0%)
More than 10 years 7 (14%) 0 (0%)

Video games experience 3 (6%) 24 (48%)
VR-based training experience 0 (0%) 5 (10%)

7.1.2 Study Procedure
The VR-based training used in this study is the same training introduced in chapter 4 and used in
experiments in chapters 5 and 6. Before receiving the training, the participants were requested to
complete a demographics survey containing questions about age, gender, preferred language,
education level, and work experience in the construction industry. This survey also contained
questions related to previous experiences with video games and VR-based training. The
demographic survey is provided in Appendix A.
After filling out the demographic survey, the participants answered a knowledge assessment
survey containing 32 questions related to the robot and the required safety checks that are needed
prior to and during the operation of the robot. These questions were validated by an expert trainer

52

and covered critical aspects of the safe and effective operation of the robot. Specifically, the
questions focused on the robot's components, controller functions and start-up, workplace
inspection, safety checks, risk zones, power cable management, robot and arm positioning, and
demolition practices. A professional trainer validated the knowledge assessment to confirm the
inclusion of the essential content for operating the robot safely and effectively. The reliability
measure of Cronbach's alpha for the knowledge measure in the student sample was 0.72, indicating
good internal consistency of the items. The knowledge assessment survey is included in Appendix
B.
After finishing the knowledge assessment survey, the participants were asked to complete two
surveys that assessed trust in the robot and robot operation self-efficacy. Trust was measured with
a modified version of the automated trust scale, which was used to assess the participant's attitudes
toward the robot. The survey contained 21 statements related to the participant's opinions on the
robot's reliability, integrity, and safety and their opinion on its impact on their careers. The survey
was based on a 5-point Likert scale ranging from completely disagree to completely agree.
Examples of the sentences in the survey include "I can trust the robot," "The robot is reliable," and
"The robot provides safety/security." The robot operation self-efficacy survey has been developed
based on modifying the validated robot use self-efficacy scale. It consisted of two sentences ("I
am confident in the robot" and "I feel confident around the robot") evaluating participants' self-
efficacy and confidence in their ability to remote operate the robot. Similar to the trust in the robot
survey, participants were requested to rate using a 5-point Likert scale ranging from completely
disagree to completely agree. The reliability measures of Cronbach's alpha for the trust in the robot
and robot operation self-efficacy measures in the construction engineering student sample were
0.78 and 0.75, respectively, indicating good internal consistency of the items in both measures.
The trust in the robot and robot operation self-efficacy surveys are provided in Appendix C.
Finally, after completing the surveys mentioned above, participants experienced VR-based
training. After completing the training, they answered the knowledge assessment and trust in the
robot and robot operation self-efficacy surveys again.
7.1.3 Analysis
For each of the outcomes, 2 x 2 mixed factorial ANOVAs with time (pre- vs. post-training) as the
within-subject factor and population type (students vs. workers) as the between-subject factor were
conducted. Additionally, separate paired sample t-tests were conducted to test whether the pre-to-
post-training change was significant in each sample. Additionally, independent sample t-tests were
conducted to test whether there is a significant difference between students’ and workers’ ratings
(e.g., knowledge, trust, self-efficacy) in each pre- and post-training condition.
7.2 Results
Table 12 presents the means and standard deviations of participants’ ratings in knowledge level,
trust in the robot, and robot operation self-efficacy assessments. First, an ANOVA revealed a
significant interaction, such that the knowledge acquisition gain from VR-based training was
significantly greater for construction students than for construction workers (F(1,47) = 5.582, p =
.022, Cohen’s d > 1.0): the average gain for students was 70.32 (out of 100), whereas, for
construction workers, it was only 60.21 (Fig. 8). Students had higher scores on the knowledge test
than workers in at both pre- (t(47)=2.718, p=0.009, Cohen’s d > 1.0) and post-test (t(47)=5.222,

53

p<0.001, Cohen’s d > 1.0). However, paired sample t-tests revealed significant improvement in
knowledge from pre- to post-test for both students and workers: from pre- (M= 16.9, SD= 9.5) to
post-test (M=87.3, SD= 9.7) for students (t(24)=32.9, p<0.001, d = 6.58) and from pre- (M=9.7,
SD=9.1) to post-test (M=69.9, SD= 13.3) for workers (t(23)=16.04, p<0.001, d =3.3).

Table 12
Means and Standard Deviations (SD) of measures based on population groups
Measures Students Workers
Pre-training Post-training Pre-training Post-training
Knowledge level 16.9 (9.5) 87.3 (9.7) 9.7 (9.1) 69.9 (13.3)
Trust in the robot 3.30 (0.55) 4.36 (0.56) 2.81 (0.37) 4.20 (0.50)
Self-efficacy 3.32 (0.69) 4.30 (0.68) 2.79 (0.69) 4.42 (0.65)

Fig. 9: Participants’ average score on the knowledge assessment

An ANOVA also revealed a significant interaction for trust in the robot, which actually increased
significantly more pre- to post-test for workers (1.38 out of 5) than for students (1.06; F(1,47) =
4.23, p = .045, Cohen’s d > 1.0) (Fig. 9). Additionally, independent sample t-tests showed that,
while students had significantly higher trust in the robot (3.30 out of 5) than construction workers
(2.81 out of 5) at pre-test (t(1,47) = 3.604, p < 0.001, Cohen’s d > 1.0), the difference between
samples was reduced to non-significant after the training (t(1,47) = 1.079, p <0.286, Cohen’s d >
1.0). However, paired sample t-tests revealed significant increases in trust from pre- to post-test
for both students and workers: from pre (M= 3.30, SD= 0.55) to post-test (M=4.36, SD= 0.56) for
16.94
87.26
9.71
69.92
0
10
20
30
40
50
60
70
80
90
100
Pre-training Post-training
Students Workers

54

students (t(24)=11.55, p<0.001, d = 2.31), and from pre- (M=2.81, SD=0.37) to post-test (M=4.20,
SD= 0.50) for workers (t(23)=10.69, p<0.001, d =2.19).

Fig. 10: Participants’ average score on the trust in the robot survey

A parallel ANOVA revealed a significant interaction for robot operation self-efficacy, which -even
more so than trust- increased significantly more from pre- to post-test for workers (1.62 out of 5)
than for students (0.98; F(1,47) = 7.634, p = .008, Cohen’s d > 1.0) (Fig. 10). Also like with trust
in the robot, independent sample t-tests showed that, while students had significantly higher self-
efficacy (3.32) than construction workers (2.79) at pre-test t(1,47) = 2.678, p = 0.010, Cohen’s d
> 1.0), the difference between samples was reduced to non-significant after the training (t(1,47) =
-.613, p = .542, Cohen’s d = 0.613). However, again, paired sample t-tests revealed significant
increases in self-efficacy from pre- to post-test for both students and workers: from pre- (M= 3.32,
SD= 0.69) to post-test (M=4.30, SD= 0.68) for students (t(24)=7.00, p<0.001, d = 1.40), and from
pre- (M=2.79, SD=0.69) to post-test (M=4.42, SD= 0.65) for workers (t(23)=8.618, p<0.001, d
=1.76).

3.30
4.36
2.81
4.20
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Pre-training Post-training
Students Workers

55

Fig. 11: Participants’ average score on the robot operation self-efficacy survey

7.3 Discussion
4.3.1 Knowledge Acquisition
The findings of this study suggest that VR-based training is effective in improving knowledge
acquisition for both students and construction workers, which might suggest a similarity between
these two population types. However, the statistical result from the ANOVA reveals that students
gained significantly more knowledge than construction workers. Additionally, looking at the
normalized knowledge acquisition reveals that construction workers gained only 66.7% of the
knowledge they could have learned, while students gained the higher 84.7% of what they could
have learned considering their base knowledge. Thus, the findings of this study show that the
effectiveness of VR-based training on construction workers' knowledge acquisition is not the same
as students. Since it is common in research to use students instead of construction workers as the
sample population to study the effectiveness of VR-based training in knowledge acquisition,
findings show that while student samples may be more convenient, accessible, and cost less than
recruiting actual construction workers, generalizing about construction workers from studies using
students can lead to under- or over-estimating the impact of VR-based training on knowledge
acquisition. Therefore, researchers must be cautious in generalizing results from one population to
another. The findings of this chapter suggest that, compared to students, training construction
workers might not improve their knowledge as much. Thus, the results from these studies [135-
138] might not be generalizable to the construction worker population. Additionally, the
independent-samples t-test indicates that students had a significantly higher base knowledge both
before and after experiencing the training than construction workers. These differences in the
knowledge levels between students and workers in pre- and post-training might emanate from
demographic variables such as students having a higher degree of education and being more
experienced in using technologies (more technology-savvy); however, it is important to note that
the sample size in this study was not powered enough to investigate how demographic variables
moderate the effect. Nevertheless, if the difference in knowledge acquisition is derived from
3.32
4.30
2.79
4.42
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
5.0
Pre-training Post-training
Students Workers

56

demographic differences, it further reinforces that the researchers should not use students as the
sample population to conclude that construction workers, and they need to be cautious in
generalizing their results. Also, the findings of this chapter are in alignment with previous research,
which has indicated that VR-based training might be beneficial for less experienced trainees, such
as students (young generations), compared to the more experienced construction workers [139].
According to the data collected from a student population, the VR-based training program had a
larger effect size compared to prior studies also conducted with students. Various factors could
explain these differences, including the characteristics of the student sample, data collection tools,
or differences in the programs studied. However, the analysis revealed that the results were
dependent on the population, even when the VR-based program, measures, and data collection
procedures were identical across different populations, including students and construction
workers. Indeed, the effect sizes obtained from paired sample t-tests indicate that the size of
knowledge gain relative to variation among the students was twice as large as the size of
knowledge gain relative to variation in the construction workers experiencing the same program.
This result suggests that although students and workers experience significant knowledge gains by
taking the training program, the magnitude of VR-based training's impact on students' knowledge
acquisition is more significant than on construction workers. Therefore, using students rather than
workers in a program aimed at targeting workers can lead to an overestimation of the program's
impact. Overall, the importance of collecting data from the targeted population to obtain a more
accurate estimate of the program's impact on end-users is highlighted in this chapter.
7.3.2 Trust in the robot & self-efficacy
The study's findings show that after VR-based training, construction workers gain more trust in
robots and robot operation self-efficacy than students. Despite initially lower trust and self-efficacy
among construction workers and students because workers and students no longer differ
significantly in trust or self-efficacy after training, the training appears to bring workers up to par
with students in terms of trust and self-efficacy. This difference in trust and self-efficacy at the
pre-test might be due to demographic differences between the two samples, especially their ages.
All students who participated were 18-29 years old and construction worker participants were from
a wide range of age groups, from 18-29 to 60 and older. These results support previous studies that
suggest younger adults, particularly those with higher education, are more willing to engage with
robots than older adults, hence may have higher trust in the robot and self-efficacy even before
experiencing the training.
Although the results show higher levels of trust in the robot and self-efficacy for students before
the training than for construction workers, importantly, the study finds a more significant increase
in trust and self-efficacy from pre- to post-training among construction workers than among
students. These findings suggest that previous research using students may have systematically
underestimated the impact of VR-based training on trust and self-efficacy outcomes for
construction workers. For instance, Song et al. studied the effectiveness of VR-based crane
operator training on technical high school students and found significant improvement in self-
efficacy. They found that their VR-based training could significantly improve students' self-
efficacy (Cohen's d = 1.88); however, they state that their results may not be applicable to workers
with significant experience. Using Cohen's d measure, the current study claims that its student
population's effect size (Cohen's d = 1.40) is similar to that of Song et al.'s findings on students
(Cohen's d = 1.88). Therefore, the current study suggests it is possible that since Song et al. found

57

significant improvement in self-efficacy in students, their findings could possibly apply to
construction workers if the differences in self-efficacy between the samples are representative of
population differences.
As robotics are increasingly used in the construction industry, understanding the obstacles to the
adoption of robotics and automation becomes more important; Therefore, human-related factors,
such as trust in robots and self-efficacy in robot operation, may become increasingly significant in
the HRI. This chapter’s findings provide insight into the generalizability of findings from student
samples to the construction worker population. If the differences in trust and self-efficacy between
the samples are indicative of population differences, given the effect of the training went from
large (in students) to even larger (in construction workers), then other researchers’ observed gains
could conceivably go from small (in students) to potentially significant (in construction workers).
Such smaller effects observed in prior among students may therefore be underrepresenting gains
in trust and self-efficacy possible with VR-based training among construction workers, at least
with well-developed VR platforms and modules.
7.4. Limitations
While this study has significant implications for the research community, some limitations do
exist. One limitation is that the sample of students in this study showed larger effects of our VR-
based training than prior studies have reported. This larger effect size among students may be
attributed to demographic differences among students, or that the design of the VR-based training
was perhaps more effective. In addition, while the effects were highly significant, the sample size
was moderate at best, and future studies with larger sample sizes may be needed to statistically
investigate how demographic variables (e.g., education level, video game experience) moderate
the effectiveness of VR-based training among students and construction workers. Regarding trust
in the robot and self-efficacy, while realistic simulation of the construction site and the robot were
included in VR-based training and the trainees were able to experience robot failures and
consequences of poor strategies, it is uncertain whether VR-based training results in the same level
of trust gains when the actual robot is used, and further studies are necessary to investigate this.

7.5. Conclusion
The present study contributes to existing research on VR-based training within the construction
industry. Previous research has explored the application of VR-based training for construction
workers, and this study examines whether the results from student samples can be generalized to
the construction worker population. The results indicate that VR-based training leads to a
significantly larger increase in knowledge acquisition for construction students compared to
workers. However, VR-based training improved trust in the robot and robot operation self-efficacy
significantly more for construction workers than students.
These findings suggest that studies based on student samples may not appropriately generalize to
construction workers, who are the intended population for the training. If a similar difference
between construction workers and students occurred with other VR-based trainings as observed in
this chapter, given that their training appeared to be less effective than this study, no effect of VR-
based training might have been observed if they had done the study with construction workers

58

instead of students. Indeed, our effect of VR-based training went from very large (in students) to
smaller (in construction workers); accordingly, it is possible that the effect of other researchers’
VR-based training could go from small (in students) to non-significant (in construction workers).
This suggests that, while they observed significant knowledge acquisition through VR-based
training among students, their findings might not replicate using a sample of construction workers.
This raises the possibility that VR-based training intended for construction workers may not
actually be able to significantly improve knowledge among workers in this population. On the
other hand, the larger gains that was observed for trust and self-efficacy among construction
workers than students suggest that prior work examining these outcomes may have underestimated
the effectiveness of VR-based training on construction workers, at least for these ancillary
outcomes. Either way, future research should be cautious -given these findings- when generalizing
from samples of construction students to populations of construction workers.

59

Chapter 8. Limitations and Future Work
While this study presents promising implications for using VR-based training to develop
knowledge, operational skills, and safety behavior in construction robotic remote operation, there
are several limitations that must be addressed in future research. One major limitation of VR-based
training is the significant effort and computing power required to develop and run the training. In
addition, there are potential physical side effects associated with using VR-based training, such as
fatigue, dizziness, and motion sickness. These issues must be considered when evaluating the costs
and benefits of VR-based training compared to in-person training. Future studies should compare
the costs of implementing VR-based training versus in-person training and take into account the
costs associated with hiring professional trainers, maintenance of the robot, and disturbance of
work on the construction site.
Another limitation of this study is the limited resources available for recruiting construction
workers and renting the actual robot during the pandemic, which resulted in only one professional
trainer being available to evaluate trainees' performance with the actual robot. In future studies, it
is recommended that a second evaluator be used to ensure inter-rater agreement on the quantitative
scores and qualitative assessment. Additionally, collecting participants' perceptions of VR-based
training would provide valuable feedback on their experience and should be included in future
studies.
While this dissertation highlights the benefits of VR-based training for human-related factors such
as trust in the robot, robot operation self-efficacy, situation awareness, and mental workload in
robotic remote operation, it is essential to note that there are differences in VR-based and in-person
training mechanisms. In-person training has practical constraints due to limited time and multiple
workers in a session, and workers cannot explore different strategies for remote operation due to
safety risks. In contrast, VR-based training offers more opportunities for workers to practice and
safely explore different aspects of operation without any risks. Future research should investigate
how VR-based training improves outcomes over in-person training and isolate those mechanisms
experimentally.
In summary, this study provides promising implications for the use of VR-based training in the
construction industry, but several limitations must be addressed in future research. By addressing
these limitations, researchers can better understand the potential benefits and limitations of VR-
based training and develop more effective training programs to improve the knowledge,
operational skills, and safety behavior of construction workers.

60

Chapter 9. Conclusion
This dissertation provides an essential contribution to the existing research on VR-based training
within the construction industry. While the advancement of construction robotics can enhance
productivity and safety in the construction industry, it also has brought about new challenges. The
unstructured and unpredictable nature of construction sites has hindered the adoption of
construction robotics. Moreover, sharing workspace between workers and robots in dynamic and
hazardous construction sites has introduced new safety concerns. Therefore, it is crucial to enhance
human-related factors such as trust in the robot, robot operation self-efficacy, situation awareness,
and mental workload while remote operating robots on construction sites to address new safety
concerns and facilitate the implementation of robotics in the construction industry.
By building on previous studies that have examined the use of VR within safety training for
construction workers and shifting the context towards technology and robotics training for the
same population, this thesis has provided valuable insights into the effectiveness of VR-based
training in enhancing knowledge acquisition, operational skills, safety behavior, and human-
related factors in HRI. The findings indicate that VR-based training in this context was associated
with a more significant increase in knowledge acquisition, operational skills, and safety behavior
than in-person training with the machine. This highlights the potential for VR-based training to be
a valuable tool in developing workers’ knowledge, ability, and safety behavior to implement
robotics in construction.
Moreover, it positions VR-based training as an equally effective pedagogical model compared to
hands-on or in-person training. This insight produces multiple (and substantial) implications for
improving human-robot interaction using VR, especially in the construction field. First, VR-based
training reduces the risk to workers and machinery associated with in-person training since trainees
cannot hurt themselves or damage the robot if they make a mistake. Second, VR-based training
can potentially significantly reduce the costs associated with training. While VR technology may
not be universally accessible at this point (both in terms of physical access and cost), in-person
training requires (at the very least) rental, transportation, and trainer fees for every training session.
Since the cost of VR technology continues to decrease, this approach to training provides an
inexpensive, on-demand, and individualized alternative to traditional approaches to training. To
this end, VR represents a safe and accessible format for construction training, one that the industry
should further develop as the field increasingly adopts robots in real-world applications. The
insights generated by this study are of significant value to the construction industry, which has
been seeking ways to enhance the safety and productivity of its workforce while reducing training
costs.
Furthermore, this thesis determined whether the results from student samples can be generalized
to the larger population of construction workers. The findings reveal that VR-based training results
in a significantly greater increase in knowledge acquisition for construction students compared to
workers. However, it also showed that VR-based training enhances trust in the robot and robot
operation self-efficacy to a greater extent among construction workers than students. These results
suggest that research based solely on student samples may not be generalizable to construction
workers, who are the target population for such training programs. However, the more significant
gains in trust and self-efficacy among construction workers compared to students suggest that
previous studies examining these outcomes may have underestimated the effectiveness of VR-
based training on construction workers, at least for these ancillary outcomes. In any case, future

61

research should exercise caution in generalizing from samples of construction students to
construction worker populations, given these findings.
However, despite the many advantages of VR-based training identified in this thesis, some
limitations exist. Studies in this dissertation rely on data generated by a limited number of
participants. While the differences between training conditions were quite large, the effects were
significant even with the small sample size; These studies might have been underpowered to test
for moderation (e.g., language, age groups, educational level). Future studies are needed to
investigate whether workers are maintaining such knowledge gains in the long run and the
effectiveness of VR-based training on knowledge acquisition, operational skills, safety behavior,
and human-related HRI factors while working with robots with a larger sample size. Beyond
generalizability issues, using a larger sample size also allows for more detailed investigations into
individual differences to more accurately determine when and why VR works as a pedagogical
model. Moreover, this thesis investigated the impact of VR-based training as a whole without
teasing apart different components of VR-based training. Future research should answer how VR-
based training improves outcomes over in-person training by isolating and studying VR-based
training mechanisms experimentally.

62

Publications to Date
Peer-Reviewed Journal Papers (Published)
1. P. Adami, P.B. Rodrigues, P.J. Woods, B. Becerik-Gerber, L. Soibelman, Y. Copur-
Gencturk, G. Lucas, Effectiveness of VR-based training on improving construction
workers’ knowledge, skills, and safety behavior in robotic teleoperation, Advanced
Engineering Informatics. 50 (2021) 101431. (Chapter 4)
2. P. Adami, P.B. Rodrigues, P.J. Woods, B. Becerik-Gerber, L. Soibelman, Y. Copur-
Gencturk, G. Lucas, Impact of VR-Based Training on Human–Robot Interaction for
Remote Operating Construction Robots, Journal of Computing in Civil Engineering. 36
(2022). (Chapter 5)
3. P. Adami, R. Singh, P.B. Rodrigues, B. Becerik-Gerber, L. Soibelman, Y. Copur-
Gencturk, G. Lucas, Participants Matter: Effectiveness of VR-based Training on the
Knowledge, Trust in the Robot, and Self-Efficacy of Construction Workers and University
Students, Advanced Engineering Informatics 55 (2023): 101837. (Chapter 6)
Referred Conference Papers
1. P. Adami, T. Doleck, B. Becerik-Gerber, Y. Copur-Gencturk, L. Soibelman, G. Lucas, An
Immersive Virtual Learning Environment for Worker-Robot Collaboration on
Construction Sites, in: 2020 Winter Simul. Conf., 2020: pp. 2400–2411. Doi:
10.1109/WSC48552.2020.9383944.

63

References
[1] P. Gerbert, S. Castagnino, C. Rothballer, A. Renz, R. Filitz, The Transformative Power of
Building Information Modeling, Digit. Eng. Constr. (2016).
https://www.bcg.com/publications/2016/engineered-products-infrastructure-digital-
transformative-power-building-information-modeling.aspx (accessed January 2, 2020).
[2] S. Brown, R.D. Brooks, X.S. Dong, Nonfatal Injury Trends in the Construction Industry,
(2020). https://www.cpwr.com/wp-content/uploads/DataBulletin-December2020.pdf.
[3] U.S. BUREAU OF LABOR STATISTICS (BLS), Number and rate of fatal work injuries, by
industry sector, (2020). https://www.bls.gov/charts/census-of-fatal-occupational-injuries/number-and-rate-of-
fatal-work-injuries-by-industry.htm.
[4] J. Davidson, Report of 10,000 severe workplace injuries might be only half the problem,
Washington Post. (2016).
https://www.washingtonpost.com/news/powerpost/wp/2016/03/18/report-of-10000-severe-
workplace-injuries-might-be-only-half-the-problem/ (accessed March 1, 2021).
[5] U.S. Bureau of Labor Statistics, Table A. Job openings, hires, and total separations by industry,
seasonally adjusted, (2021). https://www.bls.gov/news.release/jolts.a.htm (accessed April 15,
2021).
[6] F. Barbosa, J. Woetzel, J. Mischke, M.J. Ribeirinho, M. Sridhar, M. Parsons, N. Bertram, S.
Brown, Reinventing Construction: A Route To Higher Productivity (Full Report), 2017.
https://www.mckinsey.com/business-functions/operations/our-insights/reinventing-construction-through-a-
productivity-revolution.
[7] M. Tanzini, J.M. Jacinto-Villegas, M. Satler, M. Niccolini, C.A. Avizzano Embedded
architecture of a hydraulic demolition machine for robotic teleoperation in the construction sector
IEEE Int. Conf. Autom. Sci. Eng. (2018), pp. 506-513, 10.1109/COASE.2018.8560345
[8] Robotics Business Review, More than 7,000 Robots Will Work in Construction by 2025,
Report Says, (2019). https://www.roboticsbusinessreview.com/construction/more-than-7000-robots-will-work-
in-construction-by-2025-report-says/.
[9] K. Ellingrud, R. Gupta, J. Salguero, Building the vital skills for the future of work in operations,
McKinsey Co. (2020). https://www.mckinsey.com/business-functions/operations/our-
insights/building-the-vital-skills-for-the-future-of-work-in-operations (accessed March 5, 2021).
[10] Davila Delgado, J. M., L. Oyedele, A. Ajayi, L. Akanbi, O. Akinade, M. Bilal, and H.
Owolabi. 2019. “Robotics and automated systems in construction: Understanding industry-
specific challenges for adoption.” J. Build. Eng. 26 (Nov): 100868.
https://doi.org/10.1016/j.jobe.2019.100868.
[11] Kumar, V. S. S., I. Prasanthi, and A. Leena. 2008. “Robotics and automation in construction
industry.” In Proc., AEI 2008: Building Integration Solutions, 1–9. Reston, VA: ASCE.
[12] Castro-Lacouture, D. 2009. “Construction automation.” In Springer handbook, edited by S.
Y. Nof, 1063–1078. Berlin: Springer.

64

[13] Yahya, M. Y. B., Y. L. Hui, A. B. M. Yassin, R. Omar, R. O. Anak Robin, and N. Kasim.
2019. “The challenges of the implementation of construction robotics technologies in the
construction.” In Proc., MATEC Web of Conf., 5012. Les Ulis, France: EDP Sciences.
[14] Bartneck, C., D. Kulić, E. Croft, and S. Zoghbi. 2009. “Measurement instruments for the
anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots.”
Int. J. Soc. Rob. 1 (1): 71–81. https://doi.org/10.1007/s12369-008-0001-3.
[15] Freedy, A., E. DeVisser, G. Weltman, and N. Coeyman. 2007. “Measurement of trust in
human-robot collaboration.” In Proc., 2007 Int. Symp. on Collaborative Technologies and
Systems, 106–114. New York: IEEE. https://doi.org/10.1109/CTS.2007.4621745.
[16] You, S., J. H. Kim, S. H. Lee, V. Kamat, and L. P. Robert. 2018. “Enhancing perceived safety
in human–robot collaborative construction using immersive virtual environments.” Autom.
Constr. 96 (Sep): 161–170. https://doi.org/10.1016/j.autcon.2018.09.008
[17] You, S., J. H. Kim, S. H. Lee, V. Kamat, and L. P. Robert. 2018. “Enhancing perceived safety
in human–robot collaborative construction using immersive virtual environments.” Autom.
Constr. 96 (Sep): 161–170. https://doi.org/10.1016/j.autcon.2018.09.008
[18] O’Donnell, R. D., and F. T. Eggemeier. 1986. “Workload assessment methodology.” In Vol.
2 Handbook of perception and human performance, 1–49, Oxford, UK: Wiley.
[19] Endsley, M. R. 1988. “Situation awareness global assessment technique (SAGAT).” In Proc.,
National Aerospace and Electronics Conf. (NAECON), 789–795. New York: IEEE.
[20] M.J. Burke, S.A. Sarpy, K. Smith-Crowe, S. Chan-Serafin, R.O. Salvador, G. Islam Relative
effectiveness of worker safety and health training methods Am. J. Public Health., 96 (2006), pp.
315-324, 10.2105/AJPH.2004.059840
[21] I. Jeelani, K. Han, A. Albert Development of virtual reality and stereo-panoramic
environments for construction safety training Eng. Constr. Archit. Manag., 27 (2020), pp. 1853-
1876, 10.1108/ECAM-07-2019-0391
[22] R. Sacks, A. Perlman, R. Barak Construction safety training using immersive virtual reality
Constr. Manage. Econ., 31 (2013), pp. 1005-1017, 10.1080/01446193.2013.828844
[23] S. Joshi, M. Hamilton, R. Warren, D. Faucett, W. Tian, Y. Wang, J. Ma Implementing Virtual
Reality technology for safety training in the precast/ prestressed concrete industry Appl. Ergon.,
90 (2021), Article 103286, 10.1016/j.apergo.2020.103286
[24] H. Li, G. Chan, M. Skitmore Multiuser virtual safety training system for tower crane
dismantlement J. Comput. Civ. Eng., 26 (2012), pp. 638-647, 10.1061/(ASCE)CP.1943-
5487.0000170
[25] A.A. Akanmu, J. Olayiwola, O. Ogunseiju, D. McFeeters Cyber-physical postural training
system for construction workers Autom. Constr., 117 (2020), Article 103272,
10.1016/j.autcon.2020.103272
[26] J.A. Diego-Mas, J. Alcaide-Marzal, R. Poveda-Bautista Effects of using immersive media on
the effectiveness of training to prevent ergonomics risks Int. J. Environ. Res. Public Health., 17
(2020), p. 2592, 10.3390/ijerph17072592

65

[27] X. Su, P.S. Dunston, R.W. Proctor, X. Wang Influence of training schedule on development
of perceptual–motor control skills for construction equipment operators in a virtual training system
Autom. Constr., 35 (2013), pp. 439-447, 10.1016/j.autcon.2013.05.029
[28] J.C.Y. So, R.W. Proctor, P.S. Dunston, X. Wang Better retention of skill operating a simulated
hydraulic excavator after part-task than after whole-task training Hum. Factors., 55 (2013), pp.
449-460, 10.1177/0018720812454292
[29] J.R. Juang, W.H. Hung, S.C. Kang SimCrane 3D+: A crane simulator with kinesthetic and
stereoscopic vision Adv. Eng. Informatics., 27 (2013), pp. 506-518, 10.1016/j.aei.2013.05.002
[30] H. Song, T. Kim, J. Kim, D. Ahn, Y. Kang Effectiveness of VR crane training with head-
mounted display: double mediation of presence and perceived usefulness Autom. Constr., 122
(2021), 10.1016/j.autcon.2020.103506
[31] R. Barkokebas, C. Ritter, V. Sirbu, X. Li, M. Al-Hussein Application of virtual reality in task
training in the construction manufacturing industry ISARC. Proc. Int. Symp. Autom. Robot.
Constr., 36 (2019), pp. 796-803, 10.22260/ISARC2019/0107
[32] M. Wolf, J. Teizer, J.-H. Ruse Case study on mobile virtual reality construction training
ISARC. Proc. Int. Symp. Autom. Robot. Constr., 36 (2019), pp. 1231-1237,
10.22260/ISARC2019/0165
[33] P.S. Dunston, R.W. Proctor, X. Wang Challenges in evaluating skill transfer from
construction equipment simulators Theor. Issues Ergon. Sci., 15 (2014), pp. 354-375,
10.1080/1463922X.2011.624647
[34] C. Evia Localizing and designing computer-based safety training solutions for hispanic
construction workers J. Constr. Eng. Manag., 137 (2011), pp. 452-459, 10.1061/(ASCE)CO.1943-
7862.0000313
[35] F. Osti, R. de Amicis, C.A. Sanchez, A.B. Tilt, E. Prather, A. Liverani A VR training system
for learning and skills development for construction workers Virtual Reality: J. Virtual Reality
Soc. (2020), 10.1007/s10055-020-00470-6
[36] C.D. Wickens, Virtual reality and education, in: [Proceedings] 1992 IEEE Int. Conf. Syst.
Man, Cybern. 1992: pp. 842–847 vol.1. Doi: 10.1109/ICSMC.1992.271688.
[37] B. Chavez, S. Bayona Virtual Reality in the Learning Process Á. Rocha, H. Adeli, L.P. Reis,
S. Costanzo (Eds.), Trends Adv, Springer International Publishing, Cham, Inf. Syst. Technol.
(2018), pp. 1345-1356
[38] M. Fernandez Augmented-Virtual Reality: How to improve education systems High. Learn.
Res. Commun., 7 (2017), p. 1, 10.18870/hlrc.v7i1.373
[39] J.H. Kim, S.-T. Park, H. Lee, K.-C. Yuk, Virtual reality simulations in physics education,
Interact. Multimed. Electron. J. Comput. Learn. 3 (2001) 1–7.
http://imej.wfu.edu/articles/2001/2/02/index.asp.
[40] G. Makransky, S. Borre-Gude, R.E. Mayer Motivational and cognitive benefits of training in
immersive virtual reality based on multiple assessments J. Comput. Assist. Learn., 35 (2019), pp.
691-707, 10.1111/jcal.12375

66

[41] R. Nori, L. Piccardi, M. Migliori, A. Guidazzoli, F. Frasca, D. De Luca, F. Giusberti The
virtual reality Walking Corsi Test Comput. Human Behav., 48 (2015), pp. 72-77,
10.1016/j.chb.2015.01.035
[42] V. Souza, A. Maciel, L. Nedel, R. Kopper, K. Loges, E. Schlemmer The effect of virtual
reality on knowledge transfer and retention in collaborative group-based learning for
neuroanatomy students SVR (2020), pp. 92-101, 10.1109/SVR51698.2020.00028
[43] J. Bertram, J. Moskaliuk, U. Cress Virtual training: Making reality work? Comput. Human
Behav., 43 (2015), pp. 284-292, 10.1016/j.chb.2014.10.032
[44] G. Wallet, H. Sauzéon, J. Rodrigues, B. N’Kaoua Use of virtual reality for spatial knowledge
transfer: effects of passive/active exploration mode in simple and complex routes for three
different recall tasks Virtual Real Softw. Technol. (2008), pp. 175-178, 10.1145/1450579.1450616
[45] G. Wallet, H. Sauzéon, J. Rodrigues, B. N’Kaoua Transfer of spatial knowledge from a virtual
environment to reality: impact of route complexity and subject’s strategy on the exploration mode
JVRB - J. Virtual Real. Broadcast., 6 (2009) (2009), 10.20385/1860-2037/6.2009.4
[46] R. Eiris, M. Gheisari, B. Esmaeili Desktop-based safety training using 360-degree panorama
and static virtual reality techniques: a comparative experimental study Autom. Constr., 109 (2020),
Article 102969, 10.1016/j.autcon.2019.102969
[47] K.G. Kim, C. Oertel, M. Dobricki, J.K. Olsen, A.E. Coppi, A. Cattaneo, P. Dillenbourg Using
immersive virtual reality to support designing skills in vocational education Br. J. Educ. Technol.,
51 (2020), pp. 2199-2213, 10.1111/bjet.13026
[48] D. Zhao, J. Lucas Virtual reality simulation for construction safety promotion Int. J. Inj. Contr.
Saf. Promot., 22 (2015), pp. 57-67, 10.1080/17457300.2013.861853
[49] D.L. Mekacher Augmented reality (AR) and virtual reality (VR): the future of interactive
vocational education and training for people with handicap PUPIL Int. J. Teaching, Educ. Learn.,
3 (2019), pp. 118-129, 10.20319/pijtel.2019.31.118129
[50] R. Schank, Virtual learning : a revolutionary approach to building a highly skilled workforce,
McGraw-Hill, New York, 1997.
[51] I. Jeelani, K. Han, A. Albert Development of virtual reality and stereo-panoramic
environments for construction safety training Eng. Construct. Architect. Manage., 27 (2020), pp.
1853-1876, 10.1108/ECAM-07-2019-0391
[52] R. Dzeng, H. Hsueh, R. Chang, 3D game-based training system for hazard identification on
construction site, in: 2015 12th International Conference on Fuzzy Systems and Knowledge
Discovery (FSKD), 2015: pp. 2453–2458. 10.1109/FSKD.2015.7382339.
[53] Yahya, M. Y. B., Y. L. Hui, A. B. M. Yassin, R. Omar, R. O. Anak Robin, and N. Kasim.
2019. “The challenges of the implementation of construction robotics technologies in the
construction.” In Proc., MATEC Web of Conf., 5012. Les Ulis, France: EDP Sciences.
[54] Frank, M., R. Ruvald, C. Johansson, T. Larsson, and A. Larsson. 2019. “Towards autonomous
construction equipment-supporting on-site collaboration between automatons and humans.” Int. J.
Prod. Dev. 23 (4): 292–308. https://doi.org/10.1504/IJPD.2019.105496.

67

[55] Lee, J. D., and K. A. See. 2004. “Trust in automation: Designing for appropriate reliance.”
Hum. Factors 46 (1): 50–80. https://doi.org/10.1518/hfes.46.1.50.30392.
[56] Chen, J. Y. C., M. J. Barnes, and M. Harper-Sciarini. 2011. “Supervisory control of multiple
robots: Human-performance issues and user-interface design.” Part C Appl. Rev. 41 (4): 435–454.
https://doi.org/10.1109/TSMCC.2010.2056682
[57] Lee, J. D., and K. A. See. 2004. “Trust in automation: Designing for appropriate reliance.”
Hum. Factors 46 (1): 50–80. https://doi.org/10.1518/hfes.46.1.50.30392.
[58] Parasuraman, R., T. B. Sheridan, and C. D. Wickens. 2008. “Situation awareness, mental
workload, and trust in automation: Viable, empirically supported cognitive engineering
constructs.” J. Cognit. Eng. Decis. Making 2 (2): 140–160. https://doi.org/10.1518/155534308X284417.
[59] Hancock, P. A., D. R. Billings, K. E. Schaefer, J. Y. C. Chen, E. J. De Visser, and R.
Parasuraman. 2011. “A meta-analysis of factors affecting trust in human-robot interaction.” Hum.
Factors 53 (5): 517–527. https://doi.org/10.1177/0018720811417254.
[60] Parker, S. K., and G. Grote. 2020. “Automation, algorithms, and beyond: Why work design
matters more than ever in a digital world.” Appl. Psychol. https://doi.org/10.1111/apps.12241.
[61] Park, E., Q. Jenkins, and X. Jiang. 2008. “Measuring trust of human operators in new
generation rescue robots.” In Proc., JFPS Int. Symp. on Fluid Power, 489–492. Tokyo: Japan Fluid
Power System Society. https://doi.org/10.5739/isfp.2008.489.
[62] Tolmeijer, S., A. Weiss, M. Hanheide, F. Lindner, T. M. Powers, C. Dixon, and M. L.
Tielman. 2020. “Taxonomy of trust-relevant failures and mitigation strategies.” In Proc., 2020
ACM/IEEE Int. Conf. on Human-Robot Interaction, HRI ’20, 3–12. New York: Association for
Computing Machinery.
[63] Yagoda, R. E., and D. J. Gillan. 2012. “You want me to trust a ROBOT? The development of
a human-robot interaction trust scale.” Int. J. Soc. Rob. 4 (3): 235–248. https://doi.org/10.1007/s12369-
012-0144-0.
[64] Schaefer, K. E. 2016. “Measuring trust in human robot interactions: Development of the ‘trust
perception scale-HRI.” In Robust intelligence and trust in autonomous systems, edited by A.
Wagner, D. Sofge, R. Mittu, and W. F. Lawless, 191–218. Berlin: Springer.
[65] Freedy, A., E. DeVisser, G. Weltman, and N. Coeyman. 2007. “Measurement of trust in
human-robot collaboration.” In Proc., 2007 Int. Symp. on Collaborative Technologies and
Systems, 106–114. New York: IEEE. https://doi.org/10.1109/CTS.2007.4621745.
[66] Bandura, A. 2006. “Guide for constructing self-efficacy scales.” In Self-efficacy beliefs of
adolescents, edited by F. Pajares and T. C. Urdan, 307–337. Greenwich, CT: Information Age
Publishing.
[67] Pütten, A. R. V. D., and N. Bock. 2018. “Development and validation of the self-efficacy in
human-robot-interaction scale (SE-HRI).” ACM Trans. Hum. Rob. Interact. 7 (3): 1–30.
https://doi.org/10.1145/3139352.
[68] Jayaraman, S. K., C. Creech, D. M. Tilbury, X. J. Yang, A. K. Pradhan, K. M. Tsui, and P. J.
Robert Lionel. 2019. “Pedestrian trust in automated vehicles: Role of traffic signal and AV driving
behavior.” In Frontiers in robotics and AI, 117. Lausanne, Switzerland: Frontiers.

68

[69] Miller, D., M. Johns, B. Mok, N. Gowda, D. Sirkin, K. Lee, and W. Ju. 2016. Behavioral
Measurement of Trust in Automation: The trust fall. Los Angeles: SAGE.
[70] Morra, L., F. Lamberti, F. G. Prattico, S. Rosa La, and P. Montuschi. 2019. “Building trust in
autonomous vehicles: Role of virtual reality driving simulators in HMI design.” In Proc., IEEE
Transactions on Vehicular Technology, 9438–9450. New York: IEEE.
[71] Sportillo, D., A. Paljic, and L. Ojeda. 2019. “On-road evaluation of autonomous driving
training.” In Proc., 2019 14th ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI), 182–190.
New York: IEEE. https://doi.org/10.1109/HRI.2019.8673277
[72] You, S., J. H. Kim, S. H. Lee, V. Kamat, and L. P. Robert. 2018. “Enhancing perceived safety
in human–robot collaborative construction using immersive virtual environments.” Autom.
Constr. 96 (Sep): 161–170. https://doi.org/10.1016/j.autcon.2018.09.008.
[73] Chen, J., X. Song, and Z. Lin. 2016. “Revealing the ‘Invisible Gorilla’ in construction:
Estimating construction safety through mental workload assessment.” Autom. Constr. 63 (Mar):
173–183. https://doi.org/10.1016/j.autcon.2015.12.018.
[74] Gao, S., and L. Wang. 2020. “Effects of mental workload and risk perception on pilots’ safety
performance in adverse weather contexts BT.” In Engineering psychology and cognitive
ergonomics. Cognition and design, edited by D. Harris and W.-C. Li, 278–291. Berlin: Springer.
[75] Di Stasi, L. L., V. Álvarez-Valbuena, J. J. Cañas, A. Maldonado, A. Catena, A. Antolí, and
A. Candido. 2009. “Risk behaviour and mental workload: Multimodal assessment techniques
applied to motorbike riding simulation.” Transp. Res. Part F Psychol. Behav. 12 (5): 361–370.
https://doi.org/10.1016/j.trf.2009.02.004.
[76] Tevell, M., and P. C. Burns. 2000. “The effects of perceived risk on mental workload.” In
Proc., Human Factors and Ergonomics Society Annual Meeting, 682. Thousand Oaks, CA: SAGE.
[77] Young, M. S., and N. A. Stanton. 2001. “Mental workload: Theory, measurement, and
application.” In International encyclopedia of ergonomics and human factors, edited by W.
Karwowski, 507–509. London: Taylor & Francis.
[78] Dybvik, H., M. Løland, A. Gerstenberg, K. B. Slåttsveen, and M. Steinert. 2021. “A low-cost
predictive display for teleoperation: Investigating effects on human performance and workload.”
Int. J. Hum. Comput. Stud. 145 (Jan): 102536. https://doi.org/10.1016/j.ijhcs.2020.102536.
[79] Memar, A. H., and E. T. Esfahani. 2018. “Physiological measures for human performance
analysis in human-robot teamwork: Case of tele-exploration.” In Vol. 6 of IEEE access, 3694–
3705. New York: IEEE.
[80] Moore, L. J., M. R. Wilson, J. S. McGrath, E. Waine, R. S. W. Masters, and S. J. Vine. 2015.
“Surgeons’ display reduced mental effort and workload while performing robotically assisted
surgical tasks, when compared to conventional laparoscopy.” Surg. Endoscopy 29 (9): 2553–2560.
https://doi.org/10.1007/s00464-014-3967-y.
[81] O’Donnell, R. D., and F. T. Eggemeier. 1986. “Workload assessment methodology.” In Vol.
2 Handbook of perception and human performance, 1–49, Oxford, UK: Wiley.
[82] Tao, D., H. Tan, H. Wang, X. Zhang, X. Qu, and T. Zhang. 2019. “A systematic review of
physiological measures of mental workload.” Int. J. Environ. Res. Public Health 16 (15): 2716.
https://doi.org/10.3390/ijerph16152716.

69

[83] Young, M. S., and N. A. Stanton. 2001. “Mental workload: Theory, measurement, and
application.” In International encyclopedia of ergonomics and human factors, edited by W.
Karwowski, 507–509. London: Taylor & Francis
[84] Landi, C. T., V. Villani, F. Ferraguti, L. Sabattini, C. Secchi, and C. Fantuzzi. 2018.
“Relieving operators’ workload: Towards affective robotics in industrial scenarios.” Mechatronics
54 (Apr): 144–154. https://doi.org/10.1016/j.mechatronics.2018.07.012.
[85] Pini, F., M. Ansaloni, and F. Leali. 2016. “Evaluation of operator relief for an effective design
of HRC workcells.” In Proc., IEEE Int. Conf. on Emerging Technologies and Factory Automation.
New York: IEEE.
[86] Human Performance Research Group, N. A. S. A. "NASA Task Load Index (TLX) v. 1.0:
Paper and Pencil Package." Moffett Field, CA: NASA Ames Research Center (1986).
[87] Hart, Sandra G., and Lowell E. Staveland. "Development of NASA-TLX (Task Load Index):
Results of empirical and theoretical research." Advances in psychology. Vol. 52. North-Holland,
1988. 139-183.
[88] Grimmer, J., L. Simon, and J. Ehlers. 2021. “The cognitive eye: Indexing oculomotor
functions for mental workload assessment in cognition-aware systems.” In Proc., Extended
Abstracts of the 2021 CHI Conf. on Human Factors in Computing Systems, 1–6. New York:
Association for Computing Machinery. https://doi.org/10.1145/3411763.3451662.
[89] Sakib, M. N., T. Chaspari, and A. H. Behzadan. 2021. “Physiological data models to
understand the effectiveness of drone operation training in immersive virtual reality.” J. Comput.
Civ. Eng. 35 (1): 04020053. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000941
[90] Singh, G., C. P. C. Chanel, and R. N. Roy. 2021. “Mental workload estimation based on
physiological features for pilot-UAV teaming applications.” Front. Hum. Neurosci. 15: 1–20.
https://doi.org/10.3389/fnhum.2021.692878.
[91] Yauri, J., A. Hernández-Sabaté, P. Folch, and D. Gil. 2021. “Mental workload detection based
on EEG analysis.” In Artificial Intelligence Research and Development: Proc., 23rd Int. Conf. of
the Catalan Association for Artificial Intelligence (CCIA), edited by M. Villaret, T. Alsinet, C.
Fernández, and A. Valls, 268–277. Amsterdam, Netherlands: IOS
Press. https://doi.org/10.3233/FAIA210144
[92] Charles, R. L., and J. Nixon. 2019. “Measuring mental workload using physiological
measures: A systematic review.” Appl. Ergon. 74 (Jan): 221–232.
https://doi.org/10.1016/j.apergo.2018.08.028.
[93] Tao, D., H. Tan, H. Wang, X. Zhang, X. Qu, and T. Zhang. 2019. “A systematic review of
physiological measures of mental workload.” Int. J. Environ. Res. Public Health 16 (15):
2716. https://doi.org/10.3390/ijerph16152716.
[94] Akyeampong, J., S. Udoka, G. Caruso, and M. Bordegoni. 2014. “Evaluation of hydraulic
excavator human-machine interface concepts using NASA TLX.” Int. J. Ind. Ergon. 44 (3): 374–
382. https://doi.org/10.1016/j.ergon.2013.12.002.
[95] Sakib, M. N., T. Chaspari, and A. H. Behzadan. 2021. “Physiological data models to
understand the effectiveness of drone operation training in immersive virtual reality.” J. Comput.
Civ. Eng. 35 (1): 04020053. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000941.

70

[96] Liu, Y., M. Habibnezhad, and H. Jebelli. 2021. “Brainwave-driven human-robot collaboration
in construction.” Autom. Constr. 124 (Apr): 103556. https://doi.org/10.1016/j.autcon.2021.103556.
[97] Endsley, M. R. 1995a. “Measurement of situation awareness in dynamic systems.” Hum.
Factors 37 (1): 65–84. https://doi.org/10.1518/001872095779049499.
[98] Endsley, M. R. 2021. “A systematic review and meta-analysis of direct objective measures of
situation awareness: A comparison of SAGAT and SPAM.” Hum. Factors 63 (1): 124–150.
https://doi.org/10.1177/0018720819875376.
[99] Endsley, M. R. 1988. “Situation awareness global assessment technique (SAGAT).” In Proc.,
National Aerospace and Electronics Conf. (NAECON), 789–795. New York: IEEE.
[100] Endsley, M. R. 1995b. “Toward a theory of situation awareness in dynamic systems.” Hum.
Factors 37 (1): 32–64. https://doi.org/10.1518/001872095779049543.
[101] Cheng, T., and J. Teizer. 2014. “Modeling tower crane operator visibility to minimize the
risk of limited situational awareness.” J. Comput. Civ. Eng. 28 (3): 4014004.
https://doi.org/10.1061/(ASCE)CP.1943-5487.0000282.
[102] Fang, Y., Y. K. Cho, F. Durso, and J. Seo. 2018. “Assessment of operator’s situation
awareness for smart operation of mobile cranes.” Autom. Constr. 85 (Jan): 65–75.
https://doi.org/10.1016/j.autcon.2017.10.007.
[103] Hong, Z., Q. Zhang, X. Su, and H. Zhang. 2020. “Effect of virtual annotation on performance
of construction equipment teleoperation under adverse visual conditions.” Autom. Constr. 118
(Oct): 103296. https://doi.org/10.1016/j.autcon.2020.103296.
[104] Wallmyr, M., T. A. Sitompul, T. Holstein, and R. Lindell. 2019. “Evaluating mixed reality
notifications to support excavator operator awareness BT.” In Human-computer interaction—
INTERACT 2019, edited by D. Lamas, F. Loizides, L. Nacke, H. Petrie, M. Winckler, and P.
Zaphiris, 743–762. Berlin: Springer.
[105] Vahdatikhaki, F., K. El Ammari, A. K. Langroodi, S. Miller, A. Hammad, and A. Doree.
2019. “Beyond data visualization: A context-realistic construction equipment training simulators.”
In Automation in construction. New York: Elsevier.
[106] Kim, S., H. Lee, S. Hwang, J.-S. Yi, and J. Son. 2021. “Construction workers’ awareness of
safety information depending on physical and mental load.” J. Asian Arch. Build. Eng. 1–11.
https://doi.org/10.1080/13467581.2021.1908899.
[107] M.S. Knowles, The modern practice of adult education, New York:Cambridge, The Adult
Education Company, 1980.
[108] John A. EdD Henschke, "FOCUSING ON THE SIX MAJOR THEMES IN THE GLOBAL
PERSPECTIVE OF ANDRAGOGY: A JUNE 2015 UPDATE" in , IACE Hall of Fame
Repository, 2015.
[109] Association for Advancing Automation, Demolition Robots Disrupt a Labor-Intensive
Industry, A3 Robot. (2020). https://www.automate.org/a3-content/service-robots-demolition
(accessed January 15, 2021).
[110] U.S. Bureau of Labor Statistics, 18b. Employed persons by detailed industry and age, (2021).
https://www.bls.gov/cps/cpsaat18b.htm (accessed March 1, 2021).

71

[111] R.B. Johnson, A.J. Onwuegbuzie, L.A. Turner Toward a definition of fMixed methods
research J. Mix. Methods Res., 1 (2007), pp. 112-133, 10.1177/1558689806298224
[112] B.G. Glaser, A.L. Strauss, The discovery of grounded theory; strategies for qualitative
research, Aldine Pub. Co., Chicago, 1967.
[113] R.B. Johnson, M.W. McGowan, L.A. Turner Grounded theory in practice: is it inherently a
mixed method? Res. Sch., 17 (2010), p. 65
[114] J. Saldaña The coding manual for qualitative researchers (2nd ed.), SAGE, Los Angeles
(2013)
[115] M.B. Miles, A.M. Huberman Qualitative data analysis: an expanded sourcebook (2nd ed.),
Sage Publications, Thousand Oaks, Calif (1994)
[116] S. Ahmed A review on using opportunities of augmented reality and virtual reality in
construction project management Organ. Technol. Manag. Constr., 10 (2018), pp. 1839-1852,
10.2478/otmcj-2018-0012
[117] K.-T. Huang, C. Ball, J. Francis, R. Ratan, J. Boumis, J. Fordham Augmented versus virtual
reality in education: an exploratory study examining science knowledge retention when using
augmented reality/virtual reality mobile applications Cyberpsychol. Behav Soc. Netw., 22 (2019),
pp. 15-110, 10.1089/cyber.2018.0150
[118] D. Zhao, J. Lucas Virtual reality simulation for construction safety promotion Int. J. Inj.
Contr. Saf. Promot., 22 (2015), pp. 57-67, 10.1080/17457300.2013.861853
[119] J.P. Bliss, P.D. Tidwell The effectiveness of virtual reality for administering spatial
navigation training to police officers Proc. Hum. Factors Ergon. Soc. Annu. Meet., 39 (1995), p.
936, 10.1177/154193129503901412
[120] C.D. Wickens, Virtual reality and education, in: [Proceedings] 1992 IEEE Int. Conf. Syst.
Man, Cybern. 1992: pp. 842–847 vol.1. Doi: 10.1109/ICSMC.1992.271688.
[121] B. Chavez, S. Bayona Virtual Reality in the Learning Process Á. Rocha, H. Adeli, L.P. Reis,
S. Costanzo (Eds.), Trends Adv, Springer International Publishing, Cham, Inf. Syst. Technol.
(2018), pp. 1345-1356
[122] R. Schank, Virtual learning : a revolutionary approach to building a highly skilled workforce,
McGraw-Hill, New York, 1997.
[123] H.F. Moore, M. Gheisari A review of virtual and mixed reality applications in construction
safety literature Saf., 5 (2019), p. 51, 10.3390/safety5030051
[124] B. Bhavani, S. Sheshadri, R. Unnikrishnan Vocational education technology: rural India
Conf Women Comput. India. (2010), pp. 1-7, 10.1145/1858378.1858399
[125] R.S. Haluck, T.M. Krummel Computers and Virtual Reality for Surgical Education in the
21st Century Arch. Surg. (Chicago), 135 (2000) (1960), pp. 786-792, 10.1001/archsurg.135.7.786
[126] B. Esmaeili, S. Bhoir, State-of-the-Art Review of Virtual Reality Environment Applications
in Construction Safety, (2015) 457–468. Doi: 10.1061/9780784479070.040.

72

[127] D.L. Mekacher Augmented reality (AR) and virtual reality (VR): the future of interactive
vocational education and training for people with handicap PUPIL Int. J. Teaching, Educ. Learn.,
3 (2019), pp. 118-129, 10.20319/pijtel.2019.31.118129
[128] D. Zhao, J. Lucas Virtual reality simulation for construction safety promotion Int. J. Inj.
Contr. Saf. Promot., 22 (2015), pp. 57-67, 10.1080/17457300.2013.861853
[129] H.F. Moore, M. Gheisari A review of virtual and mixed reality applications in construction
safety literature Saf., 5 (2019), p. 51, 10.3390/safety5030051
[130] Jian, J.-Y., A. M. Bisantz, and C. G. Drury. 2000. “Foundations for an empirically
determined scale of trust in automated systems.” Int. J. Cognit. Ergon. 4 (1): 53–71.
https://doi.org/10.1207/S15327566IJCE0401_04.
[131] Sportillo, D., A. Paljic, and L. Ojeda. 2019. “On-road evaluation of autonomous driving
training.” In Proc., 2019 14th ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI), 182–190.
New York: IEEE. https://doi.org/10.1109/HRI.2019.8673277.
[132] Koppula, H. S., A. Jain, and A. Saxena. 2016. “Anticipatory planning for human-robot
teams.” In Experimental robotics, edited by M. A. Hsieh, O. Khatib, and V. Kumar, 453–470.
Berlin: Springer.
[133] Holmes, J. G. 1991. “Trust and the appraisal process in close relationships.” In Vol. 2 of
Advances in personal relationships: A research annual, 57–104. Oxford, UK: Jessica Kingsley
Publishers.
[134] Cheng, T., and J. Teizer. 2014. “Modeling tower crane operator visibility to minimize the
risk of limited situational awareness.” J. Comput. Civ. Eng. 28 (3): 4014004.
https://doi.org/10.1061/(ASCE)CP.1943-5487.0000282.
[135] I. Jeelani, K. Han, A. Albert Development of virtual reality and stereo-panoramic
environments for construction safety training Eng. Construct. Architect. Manage., 27 (2020), pp.
1853-1876, 10.1108/ECAM-07-2019-0391
[136] S. Joshi, M. Hamilton, R. Warren, D. Faucett, W. Tian, Y. Wang, J. Ma Implementing
Virtual Reality technology for safety training in the precast/ prestressed concrete industry Appl.
Ergon., 90 (2021), Article 103286, 10.1016/j.apergo.2020.103286
[137] R. Dzeng, H. Hsueh, R. Chang, 3D game-based training system for hazard identification on
construction site, in: 2015 12th International Conference on Fuzzy Systems and Knowledge
Discovery (FSKD), 2015: pp. 2453–2458. 10.1109/FSKD.2015.7382339.
[138] P. Wang, P. Wu, H.-L. Chi, X. Li Adopting lean thinking in virtual reality-based
personalized operation training using value stream mapping Autom. Constr., 119 (2020), Article
103355, 10.1016/j.autcon.2020.103355
[139] R. Eiris, M. Gheisari, B. Esmaeili Desktop-based safety training using 360-degree panorama
and static virtual reality techniques: a comparative experimental study Autom. Constr., 109 (2020),
Article 102969, 10.1016/j.autcon.2019.102969

73

Appendices
Appendix A. Knowledge Assessment
Instructions:
▪ Please answer each question by marking your choice.
▪ The purpose of this assessment is to measure the effectiveness of the training. Your
performance in this training will: (1) not be shared with any construction employers; (2)
not have any bearing on your future career; and, not be used for hiring purposes.
▪ If you come to a question that you do not know how to answer, please select the “I don’t
know” option.
▪ If you do not understand a question, please let the study personnel know. Someone from
our team will help you.
1) A Brokk demolition robot is constructed from different modules. The modules include which of the following parts?
a). Arm System and Drive
b). GPS, Arm System, and Tools
c). Tools, Drive, Outriggers, Arm system, and slew function
d). Arm System, GPS, Slew function, Drive, Outriggers
e). I do not know

2). Which of the following statements about power cable management is true?
a). The cable should be behind the Brokk
b). The cable should be away from the sharp edges
c). The cable should not be close to the outriggers
d). All of the above
e). I do not know

3). The screen display provides which of the following?

74

a). Tool configuration and operations
b). Error codes and operation data
c). Command panel and parameters
d). Alarms and monitoring points
e). I do not know

4). What is the preferred way to position and demolish a concrete slab using a breaker?
a). Position breaker along the concrete slab and strive to demolish inwards
b). Position breaker downwards (and near the concrete slab) and strive to demolish upwards
c). Position breaker downwards (and near the concrete slab) and strive to demolish downwards
d). Position breaker along the concrete slab and strive to demolish upwards
e). I do not know

5). Which of the following action(s) must be followed to start the control unit?

75

a). Pull up the stop button S1 and start control unit by pressing switch S0 up until LED 1 lights up
b). Push the stop button S0 and start control unit by pressing switch S2 up until LED 2 lights up
c). Push the stop button S1 and start control unit by pressing switch S0 up until LED 1 lights up
d). Pull up the stop button S0, then push up S1, the screen and LED1 both will illuminate
e). I do not know

6). When conducting demolition with Brokk, what are the recommended Brokk positioning requirements?

a). Brokk should be pointed backwards and Brokk should be moved away from the work object by extending
the arm system
b). Brokk should be pointed forwards (or straight back) and Brokk should be moved away from the work
object by extending the arm system
c). Brokk should be pointed forwards (or straight back) and Brokk should be positioned in a midrange so that
you can use the extension arm
d). Brokk should be pointed backwards and Brokk should be turned away from the work object using the
slew system
e). I do not know

7). What does number (4) indicate in the figure below?

76

a). Tools
b). Outriggers
c). Slew
d). Drive
e). I do not know

8). The risk zone around the Brokk is variable and is based on:
a). Size of the Brokk
b). Height of the Brokk (The amount of arm length from the center of the Brokk)
c). Weight of the Brokk
d). Drive system of the Brokk
e). I do not know

9). Which of the following statements about the swivel function is true?
a). Can only move upward and downward
b). Is fully rotational
c). Can only move forwards and backwards
d). It cannot be moved
e). I do not know

10). Which button is used to start the control unit?

77

a). S0 first, then S1
b). S1 first, then S0
c). S2 first, then S1
d). R1 first, then S1
e). I do not know

11). When the Brokk is on an inclined surface as shown in the figure below, the operator should be aware that the risk
zone depends on:

a). The height and slew system of the Brokk
b). The height of the Brokk
c). The outriggers and the drive system of the Brokk
d). The height of the Brokk and the incline of the surface underfoot
e). I do not know

12). Which of the following action(s) must be followed to engage the control circuit?

78

a). Press the right-hand button on the left-hand control level B1.LR (the right LED lights green). Repress
B1.LR to reinitiate.
b). Press the left-hand button on the left-hand control level B1. LL (the right LED lights green). Repress B1.
LL to reinitiate.
c). Press the left-hand button on the right-hand control level B2.RL (the LED2 lights green). Repress B2.RL
to reinitiate.
d). Press the right-hand button on the right-hand control level B2. RR (the right LED lights green). Repress
B2. RR to reinitiate.
e). I do not know

13). The Brokk is positioned incorrectly in the following two scenarios, why?

a). In both the scenarios, the cylinder is being operated in or out to its angle limit positions
b). It is difficult to assess the risk zones in both the scenarios
c). In both the scenarios, the joints of the arms are not connected
d). In both the scenarios, the wrong tool is being used
e). I do not know

14). When using Brokk, operators need to be aware to avoid positioning the arm over the outriggers. Why?
a). Doing so will prevent the operator from driving Brokk
b). Doing so will increase the chance of Brokk tipping over on its side

79

c). Doing so will increase the chance of Brokk tipping forward
d). Doing so will increase the chance of Brokk tipping backwards
e). I do not know

15). Which button is used to start the electric motor?

a). S0
b). S1
c). S2
d). R1
e). I do not know

16). When demolishing a wall with a Brokk, what are the recommended Brokk positioning requirements?

80

a). Brokk should be pointed backwards and Brokk should be moved away from the work object by extending
the arm system
b). Brokk should be pointed forwards (or straight back) and Brokk should be moved away from the work
object by extending the arm system
c). Brokk should be pointed backwards and Brokk should be turned away from the work object using the
slew system
d). Brokk should be pointed forwards (or straight back) and Brokk should be moved closer to the work object
instead of extending the arm system
e). I do not know

17). What is the following tool called?

a). Bucket
b). Breaker
c). Crusher
d). Grapple
e). I do not know

18). At a minimum, what items should be checked at the beginnings of each shift?
a). Visual inspection of the hydraulic oil level – Add oil if low
b). Inspect the first 15 feet of the power cable closest to the Brokk – Change damages cable if needed
c). Visual inspection for oil leaks- Repair as needed
d). Visual inspection of the connections between the arm sections and points where the outriggers connect to
the Brokk- Adjust and tighten connection bolts if necessary.
e). All of the above
f). None of the above
g). I do not know

19). Which button is the stop button and safety stop?

81

a). S0
b). S1
c). S2
d). R1
f). I do not know

20). You want to move the left outriggers down. Which of the following should you do?
a). Using the left control lever (B1), move left lever to the right, S5 must be in top position
b). Using the left control lever (B1), move left lever to the left, S5 must be in bottom position
c). Using the left control lever (B1), move left lever backward, S5 must be in top position
d). Using the left control lever (B1), move left lever forward, S5 must be in bottom position
e). I do not know

21). What is the best position of the arm and tool while driving Brokk?
a). Position the arm over the outriggers
b). The position of the arm does not matter
c). Center the arm in front of the Brokk with the tool position close to the machine
d). Point the arm upward.
e). I do not know

22). When demolishing a wall, which of the following actions should be taken if another personnel enters the operating
zone?
a). Continue working

82

b). Turn off Brokk
c). Warn the personnel to keep out of dust and flying rocks
d). Ask the personnel to wear safety gear
e). I do not know

23). Where is the safest place to stand and operate the Brokk? (If safe space exists)
a). Center forward of the Brokk
b). Center back of the Brokk
c). Corner front of Brokk
d.) Corner back of the Brokk
e). I do not know
24). Which of the following action(s) must be followed when positioning Brokk on an inclined surface?
a). Position Brokk in a way that the arms are not extended and the back of Brokk is down to ensure that Brokk
does not topple
b). Position Brokk in a way that the arms are extended and the back of Brokk is down to ensure that Brokk
does not topple
c). Nothing needs to be done. Brokk is designed to ensure that it does not topple.
d). Position Brokk in a way that the arms are not extended, and the center of gravity is as close to the center
of the machine as possible
e). I do not know

25). You want to move the right track forward. Which of the following should you do?
a). Using the right control lever (B2), move right lever to the right, S5 must be in top position
b). Using the right control lever (B2), move right lever to the left, S5 must be in bottom position
c). Using the right control lever (B2), move right lever backward, S5 must be in top position
d). Using the right control lever (B2), move right lever forward, S5 must be in bottom position
e). I do not know

26). What happens when you do the following with the control unit: (S5 switch is in middle position)

83

a). Operating Cylinder #2 up/down (blue); Cylinder #4 in/out (green)

b). Operating Cylinder #3 (blue); and Slew (green)

c). Operation of Synchronized Track Drive-Forward

d). Operation of Synchronized Track Drive-Reverse

e). I do not know

27) Which of the following statement is true regarding moving of the arms.
a). The operator should move the levers gradually to change the movement speed of the arms
b). The operator should move the levers suddenly to change the movement speed of the arms
c). Moving speed of levers does not have any effect on the Brokk
d). None of above.
e). I do not know

28). What happens when you do the following with the control unit: (Switch S5 is in the middle position)

84

a). Operating Cylinder #2

b). Operating Cylinder #3 and Slew

c). Operating Arm 3 and Slew

d). Operation of Synchronized Track Drive-Reverse

e). I do not know

29). How long will the control levers remain activated without movement?
a). 10 seconds
b). 120 seconds
c). 3 seconds
d). 15 seconds
e). I do not know

30). What happens when you do the following with the control unit: (Switch S5 is in the middle position)

85

a). Operating Arm 2 and Tilt

b). Parallel Function (Run cylinders 1 and 2 at the same time)

c). Operation of Synchronized Track Drive-Forward

d). Operating Arm 3 and Slew

e). I do not know

31). Which button the operator should press to activate the control levers?

a). B2.RL
b). B1.LL
c). B1.LR

86

d). B2.RR
e). I do not know
32). Which choice is correct?
a). Fitting of the Brokk to the place, lighting, and ceiling condition should be checked in a workplace
inspection.
b). Angles between the cylinders should not be less than 30 degrees during the operation.
c). 480 volts can be a suitable power source for the Brokk.
d). All of the above
e). I do not know

Appendix B. Trust in the robot and robot operation self-efficacy survey
When answering the following questions, please consider the Brokk demolition robot that you’ve
been introduced to. Please rate the statements below using a scale from 1 (completely disagree) to
5 (completely agree).
1 2 3 4 5
Completely disagree Somewhat disagree Neither agree nor
disagree
Somewhat agree Completely agree

Trust in the robot survey Answers
1. The Brokk will take my job.

2. Because of the Brokk, people like me might lose their jobs.

3. I am suspicious of the Brokk.

4. I am wary of the Brokk.

5. The Brokk will hurt my career.

6. The Brokk provides security/safety.

7. The Brokk has integrity.

8. The Brokk is dependable.

9. The Brokk is reliable.

10. I can trust the Brokk.

11. I am familiar with the Brokk.

12. I feel comfortable with the Brokk.

13. I feel like I know the Brokk well.

14. I am solid in interacting with the Brokk.

15. I know what to do with the Brokk.

87

16. I know my way around the Brokk.

17. I feel good using the Brokk.

18. I am ok around the Brokk.

19. My skills required for the task are dependable.

20. My skills required for the task are reliable.

21. My skills required for the task are consistent.

Robot operation self-efficacy survey
1. I feel confident around the Brokk.

2. I am confident in the Brokk.

Appendix C. Performance Assessment
Performance Assessment Scenario (open-ended tasks given to the trainee):
First, the trainer asks the trainee the safety checks before starting the Brokk. Then, the trainee will be asked
to start the Brokk. In the next step, the trainee should demolish a concrete slab placed in a sufficient distance
from the robot, so the trainee has to move the robot near the slab, and the trainer can evaluate the trainee's
movement skill performance. Then, the trainee's performance will be evaluated in the demolition of a
concrete slab based on the checks provided.
Question: Please mention the safety checks before starting the Brokk.
The cable connector should not be on the water
The cable should not be on sharp objects
The cable should be behind the Brokk
The cable should not be near the outriggers
The oil level should be checked.
There should be no oil leakage.

88

There should be no objects lying loose on the Brokk
The emergency button on the controller should be checked.

(Scores: 1 = Failed, 2 = Done to an extent, 3 = Perfectly done)

Score
- Participant successfully started the control unit
- Participant successfully started the electronic motor
- The operator stands on the cornerback of the robot to have the best control view during the
operation

- The operator was familiar and confident with using the B2.RL Button when there was a three
seconds gap in the operation to maintain consistency.

- The operator moved the arms in the nearest condition to the swivel function to move it in the safest
way without the danger of tilting the Brokk.

- The breaker did not touch the object before demolition to minimize the deterioration of the
hammer.

- The operator pushed the levers on the controller gradually to get a higher speed of moving arms
instead of pushing levers suddenly.

- The trainee did not extend the arm system in a straight line during the operation.
- The trainee kept the safe distance of the robot from the objects during the operation.
- The trainee kept the angles between cylinders not less than 30 degrees.
- The trainee should shut down the robot immediately when a person entered in the danger zone.

89

Appendix D. Situational Awareness Survey

a. SA Level 1 (Perception)

Pass Fail
Is the cable behind the robot?
Is the cable close to the outriggers?
Is the cable close to sharp edges?

b. SA Level 2 (Comprehension)

Is the distance between the Brokk and the element to be demolished sufficient for a proper
operation?

Are the angles between the arms of the machine in the correct position?

c. SA Level 3 (Projection)

Is the Brokk proceeding to the right position?
Will the arm trajectory hit the operator?
Will the arm trajectory hit any objects?

90

Appendix E. Mental Workload Survey
When answering the following questions, please consider the operation you had with the Brokk
demolition robot.
Mental Demand: How much mental activity was required to perform your job (thinking, deciding,
calculating, remembering, looking, searching, etc.)?

Physical Demand: How much physical activity was required to perform your job (e.g., turning, controlling,
activating, etc.)?

Temporal Demand: How much time pressure did you feel due to the rate or pace at which the tasks or task
elements occurred?

Performance: How satisfied are you with your performance at your job?

Effort: How hard did you have to work (mentally and physically) to accomplish your level of performance?

Frustration Level: How insecure, discouraged, irritated, stressed and annoyed versus secure, gratified,
content, relaxed and complacent do you feel about your job?

91

Appendix F. System Usability Survey (SUS)
When answering the following questions, please consider the VR-based training and the Brokk
demolition robot that you’ve been introduced to. Please rate the statements below using a scale
from 1 (completely disagree) to 5 (completely agree).
1 2 3 4 5
Completely disagree Somewhat disagree Neither agree nor
disagree
Somewhat agree Completely agree

Trust in the robot survey Answers
1. The VR-based training allows users to completely perform tasks.

2. The VR-based training allows users to interact with the programmed interfaces in a
natural way.

3. The VR-based training provides clear and easy to understand interactive feedback during
tasks.

4. The VR-based training provides guidance and help to the user.

5. The VR-based training should be consistent with the user and responds the way the user
expects.

6. The VR-based training is intuitive and easy for learning.

7. The VR-based training use simulated realistic models to drive users in a virtual
environment.

8. The VR-based training allows the user to control it in a flexible manner.

9. The VR-based training allows users to feel completely part of the virtual environment.

10. The VR-based training is intuitive and easy to use.

Please share your feedback by answering the open-ended questions below:
1. Please provide feedback on learning modules' designs. What limitations have you observed in the design of the
learning modules? What is your suggestion for addressing these limitations and further improvement?
2. Please provide your feedback on the modeling of the robot’s controller. What limitations have you observed?
What is your suggestion for addressing these limitations and further improvement?

92

3. Please provide your feedback on the design of virtual environments. What limitations have you observed in virtual
environment design? What is your suggestion for addressing these limitations and further improvement?
4. Please provide feedback on visualization methods. What limitations have you observed in the visualization
method? What is your suggestion for addressing these limitations and further improvement?
5. Please provide feedback on navigation methods in the virtual environment. What limitations have you observed
in the navigation method? What is your suggestion for addressing these limitations and further improvement?
6. Please provide feedback on interaction methods designed. What limitations have you observed in interaction
methods? What is your suggestion for addressing these limitations and further improvement?

Abstract (if available)

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Understanding human-building-emergency interactions in the built environment

PDF

A radio frequency based indoor localization framework for supporting building emergency response operations

PDF

Enabling human-building communication to promote pro-environmental behavior in office buildings

PDF

Towards health-conscious spaces: building for human well-being and performance

PDF

Understanding human-building interactions through perceptual decision-making processes

PDF

Behavioral form finding using multi-agent systems: a computational methodology for combining generative design with environmental and structural analysis in architectural design

PDF

A framework for comprehensive assessment of resilience and other dimensions of asset management in metropolis-scale transport systems

PDF

Semantic modeling of outdoor scenes for the creation of virtual environments and simulations

PDF

Evaluating energy consuming behaviors and the sufficiency of urban systems in the context of extreme heat hazards

PDF

User-centric smart sensing for non-intrusive electricity consumption disaggregation in buildings

PDF

Learning personal thermal comfort and integrating personal comfort requirements into HVAC system control loop

PDF

Automated alert generation to improve decision-making in human robot teams

PDF

Remote exploration with robotic networks: queue-aware autonomy and collaborative localization

PDF

Point cloud data fusion of RGB and thermal information for advanced building envelope modeling in support of energy audits for large districts

PDF

Unreal engine testbed for computer vision of tall lunar tower assembly

PDF

On virtual, augmented, and mixed reality for socially assistive robotics

PDF

Intelligent adaptive automation: activity-driven and user-centered building automation

PDF

Efficiently learning human preferences for proactive robot assistance in assembly tasks

PDF

Memristor device engineering and memristor-based analog computers for mobile robotics

PDF

AI-driven experimental design for learning of process parameter models for robotic processing applications

Asset Metadata

Creator Adami, Pooya (author)

Core Title Impact of virtual reality (VR)-based training on construction robotics remote-operation

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Civil Engineering

Degree Conferral Date 2023-05

Publication Date 05/15/2023

Defense Date 03/28/2023

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag construction robotics,educational technologies,human-robot interaction,OAI-PMH Harvest

Format theses (aat)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Soibelman, Lucio (committee chair), Becerik-Gerber, Burcin (committee member), Copur-Gencturk, Yasemin (committee member), Lucas, Gale (committee member), Meshkati, Najmedin (committee member)

Creator Email adami.pooya@gmail.com,padami@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC113122864

Unique identifier UC113122864

Identifier etd-AdamiPooya-11844.pdf (filename)

Legacy Identifier etd-AdamiPooya-11844

Document Type Dissertation

Format theses (aat)

Rights Adami, Pooya

Internet Media Type application/pdf

Type texts

Source 20230515-usctheses-batch-1044 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu