University of Southern California Dissertations and Theses

Automated alert generation to improve decision-making in human robot teams

(USC Thesis Other)

Automated alert generation to improve decision-making in human robot teams

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content Automated Alert Generation to Improve Decision-Making in Human Robot Teams by Sarah Al-Hussaini A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulllment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (COMPUTER SCIENCE) MAY 2023 Copyright 2023 Sarah Al-Hussaini Dedication This dissertation is dedicated to my wonderful family that has nurtured me since my childhood, and the joyous memories of my late grandfather, Dr. Lutfur Rahman, who has been one of the biggest in uences in my life. ii Acknowledgements I am immensely grateful for the opportunity to have had Dr. Satyandra K. Gupta as my doctoral advisor. I cannot express my appreciation enough for his patience, support, and guidance, which have made my PhD journey meaningful and exciting. His genuine care and dedication towards his students left a lasting impression on me, and always provided a sense of security. The lessons I have learned while working with him will continue to be a strong in uence on me for many years to come. I would like to extend my sincere gratitude to the members of my committee, Dr. Gaurav Sukhatme, Dr. Heather Culbertson, Dr. Stefanos Nikolaidis, and Dr. Quan Nguyen, for their valuable insights, feedback, and suggestions throughout this work. Additionally, I want to express my gratitude to DEVCOM Army Research Laboratory, especially David Baran, for their generous nancial support towards all of my research work. I would also like to acknowledge Cynthia Bedell and Stuart Young from the Army Research Laboratory. Special thanks go to my research collaborators for their valuable guidance, Dr. Peter Khooshabeh and Dr. Kimberly Pollard from the Army Research Laboratory, and Dr. Jeremy Marvel from the National Institute of Standards and Technology. Lastly, I could not have undertaken this journey without the guidance and support of Jason Gregory, who has been an exceptional mentor, friend, and colleague. I am deeply grateful to him for his signicant contribution to my growth as a researcher. I would also like to thank the many great people who I have had the pleasure of working with over the past years. Thank you to all my colleagues and student collaborators, especially Yuxiang Guan, Shantanu Thakar, Brual Shah, Shaurya Shriyam, Neel Dhanaraj, Omey Manyar, Yeo Jung Yoon, Rishi Malhan, Pradeep Rajendran, Alec Kanyuck, Prahar Bhatt, Rex Jomy Joseph and Hyojeong Kim. Their support and feedback was crucial for successfully completing this dissertation. I would like to thank all the wonderful faculties and stas from University of Southern California for making my journey memorable. I am grateful iii to all the wonderful people from Bangladesh University of Engineering and Technology, and Viqarunnisa Noon School and College in Dhaka, Bangladesh, for nurturing me into the person I am today. I would like to specially thank all of my friends, who are located all around the globe, for their endless encouragement and inspiration. Finally, I could not have accomplished this without the support of my family. My love for learning and values for higher education were instilled in me by my parents, Salma Begum and Dr. Tahmeed M Al-Hussaini. My mother took extraordinary care of me and our whole family so that we could focus on learning and building our careers. My father has shown me how to be sincere in both family life and the workplace. My younger brothers, Irfan and Rihan, have always provided me with endless support. I dearly remember my late grandfather, Dr. Lutfur Rahman, who would have been so happy today. His wise yet humble presence in our home left an everlasting eect on my life moving forward. As I have been living away from my parents and brothers for the past seven years while pursuing my graduate studies, my family never ceased to support and inspire me. I would also like to thank my grandmother, aunts, uncles, cousins, and in-laws for always showering me with love and support. Lastly, my husband, Ariyan Kabir, has been my rock, my best friend, my biggest support. He has been the person to keep me sane and grounded throughout the ups and downs in my life. The calm peaceful moments to the crazy and exciting adventures - my years with Ariyan gave me the foundation to grow as a professional, and as a person. iv Table of Contents Dedication ....................................................................................................... ii Acknowledgements .............................................................................................. iii List Of Tables ................................................................................................... viii List Of Figures .................................................................................................. x Abstract.......................................................................................................... xv Chapter 1: Introduction........................................................................................ 1 1.1 Background ............................................................................................ 1 1.2 Motivation ............................................................................................. 3 1.3 Goals ................................................................................................... 7 1.4 Scope ................................................................................................... 8 1.5 Overview ............................................................................................... 9 Chapter 2: An Alert-Generation Framework for Improving Resiliency in Human-Supervised, Multi- Agent Teams in Challenging Environments ............................................................... 11 2.1 Introduction............................................................................................ 11 2.2 Related Works ......................................................................................... 12 2.3 Overview of Approach................................................................................. 13 2.3.1 Mission Description ........................................................................... 13 2.3.2 Language For Commanding Robots ......................................................... 15 2.3.3 System Architecture........................................................................... 17 2.3.4 Specication of Alert Conditions............................................................. 18 2.3.5 Methods to Detect Alert Conditions ......................................................... 21 2.3.5.1 Simulation-based Approach ........................................................ 21 2.3.5.2 Inference-based Approach .......................................................... 22 2.4 Results.................................................................................................. 24 2.4.1 Using Simulation to Issue Alerts ............................................................. 24 2.4.2 Using inferences to issue alerts ............................................................... 25 2.5 Summary ............................................................................................... 26 Chapter 3: Discrete Simulation-based Approach for Estimating Probability Distributions, Predicting Events, and Generating Alerts ............................................................................. 27 3.1 Introduction............................................................................................ 27 3.2 Related Works ......................................................................................... 29 3.3 Generating Alerts Using Simulation ................................................................. 30 3.4 Estimating Spatio-Temporal Probability Distributions ............................................. 33 3.4.1 Discrete Velocity Event-Based Simulation................................................... 33 3.4.2 Adaptive Time Step Size ..................................................................... 39 3.4.3 Robot Groups and Time Interval Selection ................................................. 40 v 3.5 Event Prediction and Alert Generation .............................................................. 42 3.6 Results.................................................................................................. 43 3.7 Summary ............................................................................................... 44 Chapter 4: Preliminary Assessment of Alerts Enabling Humans to Identify Mission Contingencies Using a Human Subject Study ............................................................................. 45 4.1 Introduction............................................................................................ 45 4.2 Related Works ......................................................................................... 45 4.3 System overview ....................................................................................... 46 4.4 Speeding up forward simulations ..................................................................... 49 4.4.1 Discrete event based simulation model ...................................................... 49 4.4.2 Variable time-step ............................................................................. 50 4.5 Extracting alerts from simulation data .............................................................. 52 4.6 User study.............................................................................................. 55 4.6.1 Hypothesis on Performance Improvement with Alerts ..................................... 55 4.6.2 Preliminaries ................................................................................... 56 4.6.3 Study Description ............................................................................. 57 4.6.4 Data Collection ................................................................................ 58 4.7 Results.................................................................................................. 58 4.7.1 Findings from User Study .................................................................... 59 4.7.2 Discussion ...................................................................................... 59 4.8 Summary ............................................................................................... 60 Chapter 5: Generating Task Reallocation Suggestions to Handle Contingencies in Human-Supervised Multi-Robot Missions ....................................................................................... 61 5.1 Introduction............................................................................................ 61 5.2 Related Works ......................................................................................... 63 5.3 Problem Formulation .................................................................................. 66 5.3.1 Mission Model ................................................................................. 66 5.3.2 Contingencies and Updates to Robot Task Assignments ................................... 67 5.3.3 Problem Statement............................................................................ 70 5.3.4 Overview of Approach ........................................................................ 72 5.4 Task Network Generation and Estimation of Task Performance under Contingencies ........... 73 5.5 Heuristics-Based Task Scheduling .................................................................... 74 5.5.1 Preliminaries ................................................................................... 74 5.5.2 Approach....................................................................................... 75 5.5.3 Performance Evaluation ...................................................................... 78 5.6 Incorporating Robot Rescue Decisions............................................................... 81 5.6.1 Preliminaries ................................................................................... 81 5.6.2 Single Rescue Decision Making............................................................... 82 5.6.3 Multiple Rescue Decision Making ............................................................ 85 5.6.4 Performance Evaluation ...................................................................... 86 5.6.5 Complexity Analysis .......................................................................... 88 5.7 Results.................................................................................................. 89 5.8 Summary ............................................................................................... 92 Chapter 6: Assessing Value of Alerts and Suggestions in Improving Human Decision-Making Using a Human Subject Study....................................................................................... 94 6.1 Introduction............................................................................................ 94 6.2 Related Works ......................................................................................... 96 6.3 System Overview ...................................................................................... 97 6.3.1 Alert Generation Framework ................................................................. 98 6.3.2 Robot Tasking Decision....................................................................... 99 6.3.3 Hypotheses..................................................................................... 100 vi 6.4 Human Subjects Study................................................................................ 102 6.4.1 Experimental Design .......................................................................... 102 6.4.2 Mission Description and Decision-Making ................................................... 103 6.4.3 User Interface .................................................................................. 106 6.4.4 Alerts and Suggestions ........................................................................ 107 6.4.5 Participants .................................................................................... 108 6.4.6 Procedure ...................................................................................... 109 6.4.7 Measures ....................................................................................... 110 6.4.7.1 Mission Performance................................................................ 110 6.4.7.2 Decision Time ....................................................................... 111 6.4.7.3 Trust in Alerts ...................................................................... 111 6.4.7.4 Training on Alerts .................................................................. 112 6.5 Results.................................................................................................. 112 6.5.1 Findings ........................................................................................ 112 6.5.2 Hypothesis Testing ............................................................................ 114 6.5.2.1 Hypothesis 1: Alerts improve overall mission progression with time. .......... 114 6.5.2.2 Hypothesis 2: Alerts enable faster completion of higher priority tasks. ....... 116 6.5.2.3 Hypothesis 3: Alerts enable faster decision-making during retasking robots. .. 117 6.5.2.4 Hypothesis 4: An individual's trust in generic automation technology corre- lates to higher acceptance of our suggestions and improved mission performance.117 6.5.2.5 Hypothesis 5: A short and quick training is sucient to use the alert gener- ation framework. .................................................................... 119 6.5.3 Discussions ..................................................................................... 119 6.6 Summary ............................................................................................... 122 Chapter 7: Alerts Seeking Human Help to Manage Plan Failure Risks in Semi-Autonomous Mobile Manipulation................................................................................................. 123 7.1 Introduction............................................................................................ 123 7.2 Related Works ........................................................................................ 126 7.2.1 Risk Assessment ............................................................................... 126 7.2.2 Assessing Risk Awareness .................................................................... 127 7.3 System Architecture ................................................................................... 128 7.4 Modelling Alert Conditions ........................................................................... 134 7.5 Uncertainty Estimation in Environment Model ..................................................... 139 7.6 Estimating Task Failure and Alert Generation...................................................... 145 7.6.1 Collision Probability for a Task Plan ........................................................ 145 7.6.2 Probability of missed grasping ............................................................... 147 7.6.3 Orientation Maintenance ..................................................................... 149 7.6.4 Alert Generation .............................................................................. 150 7.7 Collision Probability Estimation ..................................................................... 150 7.7.1 Collision Probability between Point Clouds with Uncertainty ............................. 151 7.7.2 P2P Collision Probability in the Presence of Uncertainty.................................. 153 7.7.3 Eciency in Computational Time and Estimation Accuracy .............................. 154 7.8 Visualizing Plan Risk ................................................................................. 159 7.9 Results.................................................................................................. 165 7.10 Summary ............................................................................................... 167 Chapter 8: Conclusions ........................................................................................ 168 8.1 Intellectual Contributions ............................................................................. 168 8.2 Anticipated Benets ................................................................................... 170 8.3 Future Directions ...................................................................................... 171 References........................................................................................................ 174 vii List Of Tables 2.1 Supporting functions for robot instructions and alert conditions . . . . . . . . . . . . . . . . . 16 2.2 Example alert condition specications: Probabilistic estimation for enumerated situations . . 19 2.3 Inference-based alert conditions: Prove that robot R i can not reach state S f and location L f by time T f . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.1 Average and standard deviation values for mission parameters, L and T , using N = 250 and N = 20; 000 trials of continuous and discrete simulations. . . . . . . . . . . . . . . . . . . 35 3.2 Probability estimation of four dierent alert triggering situations across the three representa- tive mission scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.1 Performance of participants in Round 2, who were provided with forward simulation-based alerts and mission predictions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 5.1 Nomenclature used in this chapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 5.2 Performance improvement in terms of minimizing the modied makespan, using our heuristics- based approach with relay compared to the heuristics-based allocation without relay incor- poration. Maximum and average % improvement are computed from 6 randomly generated problems for each of the ve (i-v) categories based on problem sizes. . . . . . . . . . . . . . . 79 5.3 Performance benchmarking of our method in terms of computational time and optimality with exhaustive search. We chose to use 30 randomly generated problems of small size (having 710 simple tasks) in order to avoid computational infeasibility for exahaustive search. . . . . . . 80 5.4 Maximum and average performance improvement by using our approach compared to strate- giesX 0 ;X 1 ;X 2 for 30 simulated mission problems . . . . . . . . . . . . . . . . . . . . . . . . 87 5.5 Information on the generated task reallocation suggestions using our method for four mission scenarios (similar to Fig. 5.6, but with 4 dierent settings or modications): percent im- provement in reducing the modied makespan compared to attempting no rescue, maximum number of rescue(s) attempted, number of mission threads k M , and the rescue time(s). . . . 92 6.1 Overall mission progression metrics observed when alerts are provided compared to missions played without alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.2 Average mission time taken to complete all higher-priority tasks { Observed delay in complet- ing high-priority tasks without alerts compared to with alerts, in each of the four missions . . 113 viii 6.3 Number of delayed decision-making with alerts versus without . . . . . . . . . . . . . . . . . . 117 7.1 Maximum error in estimating P2P collision probability using N 2 Monte-Carlo samples with dierent N values, based on randomly generated 1000 cases. . . . . . . . . . . . . . . . . . . . 155 7.2 In case of low-P2P-collision-probability randomly generated 1000 cases, failure in detecting any possible collision for using a limited number of samples (dierent N values when using N 2 Monte-Carlo samples for uncertainty) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 ix List Of Figures 1.1 Example of hazardous mission scenarios related to military operations and disaster rescue where deployment of multi robot teams can be most benecial in terms of reducing human risks. Images from robotic deployment in dangerous operations by multiple research projects show the need to human's involvement in critical decision making. . . . . . . . . . . . . . . . 2 1.2 Human commanders in HA/DR missions needs to process to a lot of information, make de- cisions which are severely consequential such as matters of life or death, which can cause enormous stress and emotional fatigue. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.3 Human Factors Analysis and Classication System (HFACS), proposed in [184], lists several pre-conditions for unsafe acts by humans which can be mitigated by generating intelligent alerts for humans. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2.1 A motivating example scenario with four robots, R1 (yellow), R2 (green), R3 (blue), R4 (brown), visualized in Rviz. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 2.2 An example of the language decomposition oered by our proposed framework. English in- structions provided by the human (a) are converted to pseudocode (b) that is used to generate a task transition model (c). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2.3 Block diagram of the proposed alert-generation framework. The grey slanted rectacles are data blocks and blue rectangles are processing components of the system. . . . . . . . . . . . 18 3.1 A motivating example of mission scenario has a several kilometer-square suburban environ- ment with 12 hypothetical areas-of-interest, where a team of ten mobile robots, I-X, are tasked to collectively explore all regions and nd objects-of-interest (OoI). The gure shows an overlay of the latest mission updates at time 00:57 and corresponding alerts for unwanted situations which have high probability to occur. More details are provided in Section 3.5. . . 28 3.2 The system architecture diagram of our proposed simulation-based framework for generating alerts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 3.3 Assume in a mission, robot A and B are navigating through an environment based on in- dividual task-plans, and the human supervisor would like an alert based on these robots' interactions within a time interval (e.g., when their distance is less than 50m). The probabil- ity histogram of the relevant spatio-temporal parameter d min , generated from simulations, is shown in the gure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 x 3.4 A continuous-time velocity prole (blue) is represented as a probabilistic mixture model of the two discrete proles (red) with t = 10s. The mixture probabilities are such that the average velocity of the mixture is the same as the average velocity of the continuous system over the time period. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.5 A continuous-time velocity prole (blue) is represented as a probabilistic mixture of many discrete proles (red) of which three are shown in this gure. . . . . . . . . . . . . . . . . . . 36 3.6 Plot of percent maximum error in estimating distribution standard deviation (with 95% con- dence) versus computational time (unit: computation time for one continuous run). For a very conservative case, r = 10 and = 5%, we can nd N c = 416. This means, for a total computational time less than the time required for 416 continuous runs, discrete simulations will always perform better with smaller estimation error. . . . . . . . . . . . . . . . . . . . . 37 3.7 Percent maximum error in estimating standard deviation versus computation time forr = 100 and = 2 7%. We can see that discrete simulations are advantageous up to very large numbers (N c ) of continuous runs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.8 Illustrative example of using adaptive time step size, t, in discrete velocity (red) simulation for a single robot's navigation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.9 An illustrative example of identifying a group of robots and selecting a time interval for high delity simulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.10 Illustration of the method for identifying consequential time intervals along with respective robot groups from N 1 exploratory simulation runs. . . . . . . . . . . . . . . . . . . . . . . . . 41 4.1 System architecture proposed in this work that builds on our previous work [10]. . . . . . . . 47 4.2 Generating distributions of navigation parameters for discrete event simulation using data from eld experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 4.3 An example representative mission scenario with some reported information . . . . . . . . . . 53 4.4 Plots generated from forward simulation of mission scenario in Figure 4.3, showing probability distribution of having 0; 1; 2 operating robot(s) in particular regions at specic times. The discrete times in the gures span from present time to 20 30 minutes into the future, as required by the time window in alert conditions. The red markings denote concerning situations where the number of operating robots are not favorable. . . . . . . . . . . . . . . 54 5.1 Conceptual gure showing how a single rescue attempt at time t r might impact the eective team size and hence the mission progression, depending upon each rescue outcome. . . . . . 69 5.2 A solution can include any number of mission progression threads, depending upon the number of possible outcomes from each rescue (in our case, three) and the number of rescues attempted. If the initial thread (s 0 ) hadx number of functional robots, after the rst rescue, threads with s 1 ;s 2 ;s 3 will have x + 1;x;x 1 robots respectively, and therefore these threads will have dierent task plans from the time of rescue outcome. Similarly, the mission threads with s 33 ;s 22 ;s 11 will have x + 2;x;x 2 functional robots, respectively, after the second rescue. . 70 xi 5.3 (a) Updating the task network and task specications based on new information from mission alerts. (b) Original task schedule for robots on the eld, and the estimated task performance based on new information and robot update. R3 robot has been disabled, andR1 andR2 are aected by the change in task precedence graph and task requirements, and they enter idle states not being able to perform tasks as initially planned. . . . . . . . . . . . . . . . . . . . . 73 5.4 One representative curve showing hows i values (scaled with probability of each outcome) and E[S] change with rescue time (x-axis). This curve is for a specic pair of rescuer robot and disabled robot to be rescued. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.5 The process of inserting single rescue multiple times to generate the solution with possible multiple rescues. `Consider inserting a rescue schedule' in the gure refer to calling Algorithm 3. This illustration is a solution case described in Section 7.9 where the two disabled robots eligible to be rescued were R3 and R4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 5.6 Example mission for generating task reallocation suggestions. (a) Estimated current mission status, with generated alerts where robots R1-R4 are on eld robots, R1 and R2 are currently operating based on previous task assignment. R3 and R4 are disabled robots. R5 and R6 are two robots currently available to humans for direct tasking. (b) Estimated task performance based on their current task schedule. R2 will go to risky region-5 after nishing exploring 7, and will get disabled. R3 is unable to complete its task in region-3 due to lack of agents (c) New task network with all tasks which are not nished, updated task requirements, task criticality, and additional task like "relay R2" to address a contingency. The numbers on the graph represent exploration of respective regions and searching for survivors / objects-of-interest. . 90 5.7 Generated task scheduleM = fM 1 ;M 2 ;M 3 g using our approach which incorporated 1 rescue at beginning of the mission for setting II. Here, the probabilities of the mission threads, i =p R3 i ;8i2f1; 2; 3g, which correspond to the task plansM i . . . . . . . . . . . . . . . . . 91 5.8 Five mission threads with corresponding mission schedules are generated for setting IV using our mission scenario, where a second rescue is attempted only when the rst rescue failed (outcome-2), otherwise second rescue is not attempted. . . . . . . . . . . . . . . . . . . . . . 91 6.1 System architecture diagram showing the two main process blocks (gray-colored, highlighted with a green dotted rectangle) of our alert generation framework and their interactions with the multi-robot mission execution and human-robot interaction interface. . . . . . . . . . . . 98 6.2 We conduct a human subjects study to compare a human's ability to task robots with and without our alert generation framework. Here, the emulator is our custom multi-robot simu- lator which is used as the surrogate for an actual mission. . . . . . . . . . . . . . . . . . . . 100 6.3 The four mission scenes, I-IV, each containing 30 Areas-of-Interest. . . . . . . . . . . . . . . 103 6.4 A representative example view of the two monitor setup in the middle of a gameplay of mission II. The screenshot is taken when the alert framework is enabled, and the task assignment recommendations are shown in the text highlighted yellow on the right screen. . . . . . . . . 105 6.5 A representative view of the interactive task assignment section of the second display shows the nominal task performance timeline for a particular task assignment which includes one rescue operation. R1 and R2 are the two available robots. R1 is assigned to attempt a rescue on a disabled robot, R4. The gure shows task assignment issued for three alternative outcomes of the rescue operation having three, two and one functional robots to be tasked after the rescue. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 xii 6.6 Representative display of the alerts and suggestions panel on the task assignment monitor, when the alert framework is (a) disabled and (b) enabled. When alerts are available, the suggestions are highlighted in yellow color to attract the attention of the user. . . . . . . . . 108 6.7 For Hypothesis 1, we measure each participant's (a) increase in the number of missions com- pleted with alerts compared to without, (b) decrease in the total number of unnished mission tasks. The bar plots are showing the frequency of dierent values of the two metrics that we got from the 20 participants' data. They are showing the number of participants (on the y-axis) having dierent values for these two metrics (on the x-axis). Any positive number (on the x-axis) means improvement using alerts, and larger positive numbers indicate bigger improvement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.8 For Hypothesis 2, we measure the time taken to complete higher priority tasks for each mission I-IV, without and with alerts. This comparison is done across the two groups of participants, each group playing with or without alerts for each mission. The plots show the average mission times (in minutes) with corresponding standard deviations of the distributions as error bars. The red and blue data represent cases without and with alerts, respectively. . . . . . . . . . 116 6.9 Scatter plots showing how 20 individual participants' trust in automation (on the x-axis) aects (a) their total number of rejections of the recommendations (on the y-axis) and (b) mission performance in terms of the number of remaining tasks (on the y-axis), both when playing with alerts. Each participant's data is plotted with a specic color and marker shape combination in the plots. Plot (a) also shows a zoomed-in version for the 18 participants, excluding the 2 outliers with the least trust score and largely opposing trends of rejections. In plot (b), a lower number of remaining tasks corresponds to better mission performance. . 118 7.1 The architecture of our mobile manipulator system. White boxes indicate hardware com- ponents, and the remaining things are software modules. Tan boxes refer to process blocks in our software suite and navy rounded-edged boxes are for data or information. Our main contribution lies in the four process blocks (tan-colored) within the Alert Generation System (light blue box). The gray-colored boxes use already-existing state-of-the-art technologies for our human-robot interface and generating robot motion plans. . . . . . . . . . . . . . . . . . 129 7.2 Hardware system of ADAMMS 2.0 [12] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 7.3 Sub-tasks in transporting objects from one place to another by a mobile manipulator. The labelling shows the 3D object/space representations used in expressing alert conditions. . . . 135 7.4 Robot scanning an environment to generate 3D uncertainty model using the RGB-D camera (RealSense D415) mounted at the end-eector. . . . . . . . . . . . . . . . . . . . . . . . . . . 140 7.5 Use of voxel based representation. (a) Occupancy grid of the environment. Green voxels are occupied and white voxels are empty. (b) To calculate the normal vector for a voxel, we nd it's plane parameters using PCA on neighboring points. (c) The eigen vector corresponding to the highest eigen value is used as the principal axis (red colored line) for the cylinder. Let, we are calculating the cylinder geometry for the blue voxel. We hash both the empty and occupied voxels that are lying inside the cylinder. . . . . . . . . . . . . . . . . . . . . . . . . 141 xiii 7.6 Estimation of each nominal point location along with its uncertainty level, using points inside the hypothetical cylinder (with principal axis along the normal direction of the voxel). (a) All points lying inside the cylinder is used to estimate the uncertainty model. (b) The points are projected on the normal vector, and the average location of them is considered as the nominal point, indicated by a green-colored symbol. (c) The standard deviation among the lengths of the blue-colored dashed lines provides of the zero-mean Gaussian distribution for uncertainty of the nominal point along the normal direction (red line). This is the uncertainty metric for the nominal point (green). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 7.7 Representation of the uncertainty model generated for the scene in Fig. 7.4 during our exper- iments. The point cloud shows nominal point locations, and green-to-red color scheme shows the uncertainty level in each point along its normal direction. Greener points indicate lower uncertainty, and redder points higher. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 7.8 Missed grasping is possible under uncertainty. The red colored two-way arrows indicate un- certainty of point locations on the front surface of the work-piece (green). . . . . . . . . . . . 147 7.9 Computing probability of missed grasping under certainty . . . . . . . . . . . . . . . . . . . . 148 7.10 P2P collision checking without uncertainty. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 7.11 P2P collision checking with location uncertainty along normal directions. . . . . . . . . . . . 152 7.12 P2P collision probability computation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 7.13 An instance in visualizing manipulator plan for grasping an object, where the green colored arm indicates the goal pose. [12] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 7.14 Yellow bubble and red points are indicating collision between the gripper and the wall of 3D printer. An alert message pops up to indicate high risk (probability) of failure for a given task plan. An alert sound also gets played along with the pop-up message to attract human attention. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 7.15 Brighter red colored are the points with higher probability of collision, and darker reds have lower risk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 7.16 Two dierent colors, green and blue, to visualize collision region in the gripper and obstacles, respectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 7.17 Choosing the right color scheme to visualize collision region is important. (a) If one color is used which is already dominant in one of the objects, it loses clarity. (b) If one non-dominant color is used, but the colliding objects are too close, its becomes hard to distinguish between the two objects. (c) Using two dierent colors on the objects, and using non-dominant color can provide better clarity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163 7.18 3 arrows, along surface normal direction, extending from the collision region helps to show the extent of location uncertainty of the points. . . . . . . . . . . . . . . . . . . . . . . . . . 163 7.19 In ation of point clouds: (a) with no in ation, (b) in ation of all points by . . . . . . . . . 164 7.20 The scenes we used in our experiments for grasping and retrieving an object. The objects to be grasped are marked with the green arrows. . . . . . . . . . . . . . . . . . . . . . . . . . . 165 7.21 Robot executing a safe plan for retrieving an object from inside the 3D printer in scene-1. . 166 xiv Abstract Human supervision is essential for deploying multi-robot teams in challenging missions, such as military applications and disaster rescue operations. Humans need to have high-level decision-making authority because of the safety-critical decisions and life-or-death situations in these missions. The major challenges are robotic failures, uncertainty in navigation and other task performances, and delayed and intermittent data ow due to restrictive communications. Whenever some mission update gets available to humans, they need to predict the upcoming and unknown mission states, adjust their plan rapidly, and retask robots if necessary. Since humans in supervisory roles in such missions are already under tremendous stress and emotional fatigue, they might not make the best decisions. My dissertation aims to provide computational foundations for risk assessment, alert generation, and robot tasking suggestion generation to assist human supervisors. Firstly, an alert generation framework is presented where the alert conditions are mathematically modeled to oer human users exibility and extensibility. Secondly, a forward simulation-based approach is developed to estimate the probability of mission events under uncertainty in a computationally ecient manner. Next, a heuristics-based approach is presented for automated suggestion generation for robot retasking decisions to handle potential contingencies. Fourth, a human subject study is performed to evaluate the usefulness of the alerts and suggestions as cognitive aids in enabling faster and better decision-making by humans. Finally, human supervision is required in task-level operations with mobile manipulators in the presence of uncertainty, and this dissertation presents a hardware or software-agnostic framework for risk assessment, alert generation, and risk visualization for humans. I anticipate that this work will help improve human decision-making in supervising challenging operations involving robots. xv Chapter 1 Introduction 1.1 Background Multi-robot systems are becoming more adept to serve alongside humans in real-world missions as researchers make advancements in fundamental capabilities and resilience [71,92,110,111,138{140,164,181,210]. These types of systems will be especially useful for dangerous missions in complex environments, because of their ability to increase stando distances and reduce the risk to humans. Military operations and humanitarian- assistance and disaster-relief (HA/DR) eorts can pose signicant risks to human health and lives [157], therefore deployment of multi-robot systems in these applications is being widely researched worldwide [67, 70, 106, 108, 109, 120, 141, 165, 170, 193]. Fig 1.1 highlights some of such dangerous missions which are still typically done by humans, and various research groups are actively working for more and more robotic deployment as shown in the images. Despite the widespread use and ever-increasing capabilities of robotic systems, researchers anticipate that human team members will continue to be necessary - and not be replaced by technology - because of various advantages, including diverse expertise, adaptive decision-making, and the potential for synergy. More importantly, HA/DR missions involve tasks with literal life-or-death consequences and so human-in-the-loop operations are mandatory to ensure proper management of resources and critical decision-making authority (Fig. 1.1). We anticipate that human teammates will serve in a supervisory role to manage resources and make critical decisions because of the gravity of events, including life-or-death situations [46, 60]. At a 1 Figure 1.1: Example of hazardous mission scenarios related to military operations and disaster rescue where deploy- ment of multi robot teams can be most benecial in terms of reducing human risks. Images from robotic deployment in dangerous operations by multiple research projects show the need to human's involvement in critical decision making. high-level, human supervisors will ensure that robots are continuously working on tasks that align with the mission objectives and issue commands that account for the dynamically-changing context. As multi-agent systems will require extensive collaboration, it is imperative that systems be developed with both the strengths and weaknesses of humans in mind, just as researchers and engineers design for the robotic agent. The disaster response system needs to integrate humans, robots, and software agents for eective response to emergency situations [170]. Connecting human language [29], adapting to human intents [117], and co-development of joint strategies [169] can assist successful human-robot teaming. 2 1.2 Motivation The role of a human supervisor in a humanitarian-assistance or disaster-relief mission is incredibly challeng- ing. We envision humans to have complete decision-making authority at the mission level, they will assign tasks to the robots, and robots will simply carry out the actual tasks as scheduled by humans. The events, circumstances, and outcomes of decisions in such missions are severely consequential and complex, which directly impacts a human's capability. Large unstructured environments, restrictive communications, and the possibility of the system- and task-level failures introduce uncertainty in the availability and eciency of agents and signicant delays in the receipt of mission-critical information. Moreover, there is often cognitive overload due to the need to process a large amount of information, and emotional fatigue is unavoidable (Fig. 1.2). In such missions, it is of the utmost importance that humans make the best use of the available information and constantly adapt their strategy, and update task assignments to improve mission perfor- mance. However, perceiving the mission situation correctly from some incomplete or possibly outdated information is a challenging task. Furthermore, these issues are exacerbated if humans are cognitively- and emotionally-fatigued, which could lead to the issuing of slow-paced or ill-conceived commands to the robotic teammates. Figure 1.2: Human commanders in HA/DR missions needs to process to a lot of information, make decisions which are severely consequential such as matters of life or death, which can cause enormous stress and emotional fatigue. There is already an advanced distributed tool for commercially available smartphones and tablets, An- droid Team Awareness Kit (ATAK) [97, 113], to be used for multi-agent teaming, real-time coordination, monitoring, and commander roles. It allows for precision targeting, surrounding land formation intelligence, 3 navigation, and data sharing to improve situational awareness. However, it solely focuses on known and reported information rather than estimating unknown or future mission states. Since many military or dis- aster missions can often have uncertainties and robotic agent failures, and signicant latency in information propagation, we envision mission predictions and proactive alerts are necessary. Alerts provide an ecient and eective way for enabling and improving resiliency because they oer a means to prevent human-introduced mistakes, expedite decision-making, and reduce stress and emotional fatigue. If alerts can forewarn humans of potential concerns, they can signicantly improve human perception and resource allocation. Alerts are already being used in numerous technologies and application domains because they oer measurable value. For example, lane departure warnings and blind-spot detection systems alert drivers of potential collisions [99], which leads to prudent decision-making for safe driving. Similarly, an intelligent alert system can signicantly benet human-robot teams operating in time-sensitive, safety- critical scenarios because alerts can assist the team with the required decision-making that is inherently challenging, taxing, and severely consequential. Figure 1.3: Human Factors Analysis and Classication System (HFACS), proposed in [184], lists several pre-conditions for unsafe acts by humans which can be mitigated by generating intelligent alerts for humans. 4 Human factors analysis is an essential research topic in any application where humans are taking the role of an operator or supervisor. It is more crucial in applications where humans are required to make rapid decisions and take appropriate action where the consequence of human decisions is catastrophic. For example, 70 80% of all accidents in aviation are caused by human errors. Shappell et al. [184] proposed a framework called Human Factors Analysis and Classication System (HFACS) framework to examine underlying human causal factors systematically, as shown in Fig. 1.3. The framework is applied for human operators in dierent applications [183], [50]. It highlights some preconditions for unsafe acts such as environmental, and personnel factors, and the mental and physical condition of operators. It also identies decision errors, perceptual errors, and violations as unsafe acts. Clearly, for our application scenario, human supervisors in disaster missions may encounter these factors, and our alert system is aimed to address all these factors and improve the performance of human supervisors. Building a useful alert system suitable for multi-agent systems requires careful consideration. First, a proper language-based scheme is necessary to facilitate intuitive interaction with human teammates. Second, unlike the aforementioned alert systems for everyday tasks, an alert system for multi-agent systems cannot simply rely on sensors and comprehensive observations. Instead, a eldable alert system requires inference and probabilistic model estimation to account for the inherent uncertainty in mission operations, and estimate future mission states. Alerts need to be generated every time humans receive new information regarding the mission. Finally, an alert system must also be exible to the types of alerts oered, be tailored to human preferences and mission needs, and generate meaningful alerts in a timely fashion so that agents can take the necessary corrective actions. This way, generated alerts can improve human decision-making, by improving how accurately humans can predict mission situations, providing them the opportunity to adjust mission plans accordingly. There remains an open question of how to generate alerts in a computationally ecient manner. A variety of events could occur during the mission, these events are probabilistic, and there is a considerable amount of uncertainty in the mission progression that alerts directly aect. Since we can't guarantee the occurrence or sequence of events as the human-robot team interacts with the real world, we must perform forward simulations to build probabilistic models about the state of the mission. We can perform a large 5 number of Monte-Carlo (MC) simulations to build distributions over the possible outcomes and represent the current belief of mission states to do inference and alert generation. However, we cannot hastily increase the number of simulation runs as this would result in impractically slow model building for alert generation. Our models and simulations of mission outcomes must be accurate enough to predict the possibility of unwanted events. At the same time, the determination to issue alerts to human supervisors must be suciently fast in order to be actionable [143]. This challenge necessitates careful consideration for a simulation model. Simulation-based approaches typically increase accuracy by using higher delity data and more simulation runs, but this increases the necessary processing time and resources for generating alerts and can become prohibitively large depending on the environment, tasks, and team size [44]. To generate alerts in a timely fashion, we need to estimate probabilities within seconds. It means that the forward simulations need to be fast enough. Thus, there must be a balance between the competing objectives of simulation accuracy and alert generation timeliness for critical missions. Even though we consider a human-in-the-loop system, ecient multi-robot deployment would still require mission planning to be at least partly automated to decrease the workload on humans, especially in the case of an extensive mission. Whenever potential contingencies are reported or generated by our alert system, humans need to adjust their strategy and re-assign robots as and when required. It can be particularly challenging if the mission is large-scale, and there is signicant stress, fatigue, and information overload for the human supervisor. Retasking robots in DR missions poses some unique complexities, such as task criticality, disabled robots, handling limited communication with functional robots, and stochastic outcomes of potential robot tasks. These make the problem slightly dierent from traditional multi-robot task allocation (MRTA) problems [63,146] studied in the literature. Many missions might require to have robotic arms mounted on the mobile robots to perform certain manipulation tasks during the mission, such as transporting an object. Human supervision might sometimes be needed in task-level operations by mobile manipulator robots. Despite having a human supervisor, We anticipate most of the task plans to be auto-generated by the robot planner. However, it might be worthwhile to assess the level of risk associated with a candidate plan before executing tasks that are delicate or involve physical interactions by the robotic manipulator. Risk assessment and alert generation 6 for humans can be benecial in ensuring safer operations. Research is also needed regarding how to improve human understanding of the risk situation as detected by automated alerts. 1.3 Goals The research objective of this dissertation is to develop an intelligent alert generation system that can identify risks, generate alerts, and provide valuable suggestions to improve the decision-making of humans supervis- ing robots operating in challenging scenarios. Firstly, this thesis investigates multi-robot teams deployed in large-scale HA/DR missions under the supervision of humans. A novel alert-generation framework is presented that overcomes the challenges regarding the language-based scheme, exibility and extensibility requirements, and timeliness in the computational process to improve the resiliency of multi-agent team- ing. A discrete event-based forward simulation-based approach is presented that balances the competing objectives of mission prediction accuracy and computational time. The main goal is to unburden humans and improve human decision-making and mission performance in challenging missions. This dissertation studies whether alerts can actually be helpful in representative mission scenarios. It details an approach for providing auto-generated suggestions for humans regarding retasking robots to handle the contingencies. This task scheduling needs to include incorporating task criticality, disabled robots, and potential robot rescue attempts with stochastic outcomes, ensuring sucient information propagation in the presence of communication constraints, etc. Experimental evaluation by conducting a human subject study is presented regarding how alerts and suggestions can aect human decision-making in multi-robot missions. This thesis also investigates potential task-level failure modes for human-supervisor in mobile-manipulator operations and how to rapidly and accurately generate alerts to maintain the operational tempo set by the human operator and ensure decision-making relevancy. The human-robot interface must be interactive to enable bi-directional information ow and be comprehensible to facilitate human decision-making. Finally, a human supervisor needs to view and assess risk better with a mobile manipulator performing tasks. To summarize, this dissertation focuses on developing a proactive and prediction-based alert generation framework for humans supervising mobile robots in challenging operations and seeks solutions to the following research questions. 7 • Can an alert framework be exible to support the addition or customization of alerts in accordance with dynamically changing mission needs and strategies during a mission? • Can mission-level predictions and alerts be generated in a computationally ecient way with accuracy consideration? • Can alerts enable humans to identify potential contingencies better? • Can the alert system provide task assignment suggestions to address potential contingencies in complex missions? • Can alerts and robot tasking suggestions result in faster and better decision-making? • Can alerts be generated for task-level operations by mobile manipulator robots? Can we make the user interface comprehensible in terms of presenting risks and improving human decision-making? 1.4 Scope The scope for alert generation for multi-robot missions is limited to a homogeneous team of multiple mobile robots, navigating around a large (a few kilometers square) outdoor environments and performing specic tasks. In evaluation and consideration, a team of 4 12 robots is assumed which can be easily extended. However, considering robots in the order of hundreds might not be computationally feasible to handle using our approach. It is also assumed that the estimated model is available for navigation, exploration, or other tasks to perform our simulation-based approach. It is anticipated that prior experience or small-scale real- world experiments can provide information for building those models. Furthermore, this thesis considers the simulation of robots performing in the mission for several hours, which could be extended to days depending on the simulation model's delity. The method is contingent upon the mission models that are available before or during the mission. It supports a particular set of robot tasks in the mission and some unique constraints relevant to military or disaster response scenarios. The participants in the human subject studies detailed in this work are young adults, all graduate students 8 in engineering subjects. The user interface design problem to present all information in an optimal way to humans is out of the scope of this work. In the problem of task reallocation suggestion generation, only a specic list of contingencies is considered to be addressed, as mentioned in the relevant chapter. The approach has been tested for problems with sizes falling within a particular range, in terms of the number of tasks up to 50 with 4 10 precedence stages, the number of robots between 510 (including disabled robots, available robots, and functional on-eld robots), the maximum number of rescues being 2. However, the method can easily be applied to larger problems that have not been tested yet. In terms of generating alerts for mobile manipulator systems, some specic failure modes for trans- portation tasks are considered to illustrate task-level alerts for a single robot. Some of the addressed alert conditions, e.g., estimated collision risk, might be applicable in various applications using mobile manipu- lators. In contrast, other alert conditions might change in a dierent type of operation. Even though this dissertation aims to have the risk visualizer easily perceivable and actionable by providing a variety of helpful visualization settings, the user interface design research is considered out of scope. 1.5 Overview This section brie y summarizes the chapters in this dissertation. Chapter 2 rst introduces the alert genera- tion framework for human-supervised multi-robot missions in challenging environments. It describes mission tasks, other mission parameters, and language for commanding robots and proposes a way to express alert conditions in a mathematical framework. In Chapter 3, a discrete event-based simulation model is described to generate alerts computationally eciently while considering probability estimation accuracy. Chapter 4 presents some results from human subjects study to highlight the merits of our alert system in improving the human perception of mission situations at static mission states. Chapter 5 is about generating task re-assignment suggestions to assist humans to reduce or prevent potential contingencies from aecting the mission performance negatively. Next, we combine our technology for alert generation and suggestion gener- ation and conduct a comprehensive human subject study to evaluate the benets of using our framework in HA/DR applications. Chapter 6 details this user study and its ndings regarding how alerts and suggestions 9 can improve human decision-making in multi-robot missions. Chapter 7 describes task-level alert genera- tion, and visualization for semi-autonomous mobile manipulator operations. Lastly, chapter 8 presents the intellectual contributions of this dissertation, concluding remarks, and future research directions. 10 Chapter 2 An Alert-Generation Framework for Improving Resiliency in Human-Supervised, Multi-Agent Teams in Challenging Environments This chapter is based on the published paper [8]. 2.1 Introduction A motivating mission scenario is illustrated in Figure 2.1. Our focus is on the generation of alerts for potential dangers or unwanted situations, as well as, erroneous and inecient task assignments issued by humans. To account for the inherent uncertainty in mission operations, a eldable alert system involves inference and probabilistic model estimate. An alert system must also be adaptable in terms of the sorts of warnings it provides, adapted to human preferences and mission requirements, and capable of producing relevant alerts in a timely manner so that agents can take the appropriate remedial measures. We oer a unique alert- generation approach in this paper to address these issues and improve the resiliency of multi-agent teaming. To simulate the likelihood of salient system states throughout the execution of a human-commanded mission uncertainty, we rst dene a formal language and a state machine-based simulation architecture. We enable the humans to describe alert circumstances based on their requirements and preferences using a probabilistic temporal logic framework to support dening ad-hoc alert conditions by humans, and oer the required exibility and extensibility. To demonstrate the usefulness of our proposed framework, we provide some 11 Figure 2.1: A motivating example scenario with four robots, R1 (yellow), R2 (green), R3 (blue), R4 (brown), visualized in Rviz. example scenarios to be detected as results. They show the detection of unwanted situations based on new information which can be complex for humans to infer by themselves in time-critical missions. We also demonstrate the usefulness of smart simulations which can be useful in detecting low probability events in computationally ecient manner compared to Monte-Carlo simulations. 2.2 Related Works Researchers have investigated the human factor concerns associated with supervisory control of multi-robot systems [38,186,221]. To facilitate performance improvement, alert systems are being developed for numerous applications with humans acting as operators or high-level decision-makers [99, 229]. There exist several dierent alert-generating architectures and interfaces for controlling robots, such as an augmented reality- based solution for collaborative assembly [123] and alert systems used by NASA in aviation [136]. There have also been several alert systems designed specically for human-robot teams in the disaster response 12 context [24, 86, 145]. However, all these alerts are purely reactive, where a warning appears to the human once an undesirable event has occurred, e.g., navigation change, obstacle alert, etc. In [175], some predicted interface designs are depicted, where a trained neutral net predicts the operator's evaluation on risk and relevance of a robot performing some task. In this work, we also seek a proactive approach to providing alerts; however, our focus is on the generation of alerts for potential dangers or unwanted situations in the mission, as well as, erroneous and inecient task assignments issued by humans. There are several works done for dierent risk assessments in search and rescue or similar missions [14,15,142,188]. [188] focuses on low-level risk identication in multi-robot mission with human commanders. They use the commander's description of a particular scene along with the scene image from the robots to estimate the danger level at the scene. [142] provides simulation-based risk assessment of a eet of autonomous machines performing a rescue mission under ooding situation. [14, 15] describe a probabilistic framework for reasoning about the safety of robot trajectories in dynamic and uncertain environments. All these technologies related to specic low-level risk assessment can be benecial towards building environment or simulation models used in our alert framework for estimating mission-level risks or contingencies. 2.3 Overview of Approach 2.3.1 Mission Description We consider a disaster-stricken environment, e.g., a city after an earthquake, ood, or wildre, as a represen- tative environment for a generic HA/DR mission. The outdoor environment is assumed to be on the order of several square kilometers and comprised of complex, unstructured terrain. A human-supervised team of robots is deployed for exploring aected regions eciently, collecting important information, and performing certain tasks, according to human preferences. As the team of robots navigate through the environment, there is some nonzero probability of operational failure, which could be a result of spatial factors, such as complex terrain, or stochastic events, such as hardware or software failures. There also are communication challenges because of the large scale operational environment. Limited communications cause substantial delays in humans receiving information from the robots in the eld. We encode these typical challenges and 13 characteristics of a generic HA/DR operation in our specic mission in order to present and test our proposed alert generation framework. Thus, any other multi-robot mission, related to HA/DR-relevant application might be reduced to a variation of our dened mission, and our framework can be tailored for any such operation. In a large-scale environment, usually there are some regions of higher importance which should be given priority in the exploration process. We assume that the humans, typically the rst responders, can use their expertise and protocols to identify some areas-of-interest (AoIs), chosen at the beginning of the mission, or dynamically through out based on latest information. We also assume, there are a few, very sparsely located beacons, and robots need to be physically close to them in order to communicate with humans. Each robot carries out several tasks based on its observations and the situation-specic instructions, and then navigates back to a beacon after some time to reconnect with its supervisors. In our mission, human supervisors at the base station provide instruction-set to each robot in the team, and dispatch them for exploration and data collection. In addition to navigation and exploration, humans can instruct a robot to rendezvous with another robot, or relay new instructions to another robot. Humans can also command a robot to provide assistance to a temporarily-disabled robot in an attempt to make it functional again. Every time humans receive new information from a returning robot, they need to assess the current situation, and make intelligent decisions based on the latest update. Humans might want to re-task the elded robots, which may be currently out of communications range, in order to avoid unwanted situations and prevent undesirable contingencies. Thus, the human supervisors are under constant pressure to make decisions quickly in order to utilize the robots most eectively. Due to the large size of the environment, humans may only receive updates from each robot after some extended time. Therefore, the robots need to be suciently tasked for prolonged periods, and the stochasticity and danger involved in the mission may prompt the humans to give instruction-sets which are incredibly complex. This necessitates a systematic way for humans to command robots, and deployed systems should be equipped to describe a complete instruction-set with adequate complexity. 14 2.3.2 Language For Commanding Robots We present a formal language in this section that can decompose the complex, human-provided instructions in a systematic way. First, we dene a set of tasks that a robot can execute; these are: explore, navigate, rescue, rendezvous, relay, return, wait. The uncertainties in the mission, and dierent stochastic phenomena demand several other actions from the robots, in addition to exploration and navigation. In order to improve resiliency in a mission with a high probability of failure, robots need to provide assistance to temporarily- disabled teammates by attempting rescues, as dened in our previous works [7, 8]. This rescue operation has stochastic outcomes, such as successful rescue, failed rescue, and, in the worst case, the rescuer robot also becoming inoperable during the attempt. In order to tackle the challenge of scarce communication in a HA/DR mission, we have included rendezvous and relay tasks. In a collaborative exploration-based mission, it is useful for multiple robots from dierent regions to meet at pre-scheduled times and locations to exchange information, referred to as rendezvous. The relay task requires a robot to go to a specic region, search for another robot, and relay a specic piece of information to that robot. Here, we have limited the scope of relay tasks to initially clear the other robot's old instructions and then issue a new command-set. Since one of the focuses of this work is to provide alerts for future contingencies, this relay task is particularly helpful. It can be used in sending an available robot to prevent an adverse situation happening to a robot already in the eld. Finally, the return and wait tasks correspond to the robot navigating back to within communications range of the human supervisor, or remaining stationary in one location, respectively. Let R =fR 1 ;R 2 ;:::;R N g be the set of N robots, deployed in the mission. At any time t, each robot R i has state S Ri , location L Ri , event list E Ri , list of other robot's most recent status updates (location, state) I Ri , and its own state history list StateHist Ri . A robot goes through a series of states in order to perform a particular task. For example, TravelToRend, WaitToRend, Rendezvouzing are the states corresponding to task rendezvous; all having the same argument-list, i.e., robot ID to meet, rendezvous location, and time window. The element StateHist Ri gives its state from a time that was instances before the current time. A complete list of events should include relevant environmental events as well as some task related events with a robot itself or other teammates. An example event type is rescue attempt, with arguments robots IDs, location, time, and outcome. Every unique event a robot R i learns about is stored in E Ri . In order 15 Table 2.1: Supporting functions for robot instructions and alert conditions Functions or Literals Description of Return Variables or Values TravelDurationi(X1;X2) Estimated travel duration for robot Ri to navigate from regionX1 toX2 isRiskyRegion(X) True when the regionX is risky, False otherwise NearbyRobotIDi(st;d) ID of robot with state type st, and within distance d from robot Ri CountNearbyRobotsi(st;d) Number of robots with state type st , within distance d from robot Ri CountExploringRobots(X) Number of robots exploring regionX CountEvents(e;j1;a1;j2;a2;:::;im;am) Number of events in E R i whose type is e, and arg j 1 ; arg j 2 ;:::;arg jm are respectively a1;a2;:::;am ToRendi = ( 1; if S R i :type =TravelToRend 0; otherwise True if robot Ri is travelling to rendezvous location EndRendi = 8 > < > : 1; if (StateHist 1 R i :type =WaitToRend; S R i :type6=WaitToRend ) or S R i :type =Rend 0; otherwise True if robot Ri moves to a new task after a rendezvous attempt IsNavi = 8 > < > : 1; if S R i :type2fNavigating;TravelToExpl; TravelToResc;TravelToRel;TravelToRendg 0; otherwise True if robot Ri is travelling to a task location NeverRescue(i;J) = ( 1; if8s2 S i ;:(s:type =rescue; s:arg 1 2J) 0; otherwise True if robot Ri will never attempt rescue on a robot with ID 2J NeverRelay(i;J) = ( 1; if8s2 S i ;:(s:type =relay; s:arg 1 2J) 0; otherwise True if robot Ri will never attempt relay on a robot with ID 2J MinTravelT(i;L) = EuclideanDistance(Li;L) MaxNavigationSpeed R i Minimum navigation time for Ri from its location Li to location L Dened in Inference-based Approach Section to create time-based and situation-dependent instructions, humans need to construct dierent conditional statements which make use of time, the robot's information state at the instance, and model estimation of dierent parts of the system. For crafting these conditions, we present a compiled list of functions in Table 2.1. We use items from this list for the robot's instruction-sets and alert conditions presented in Figure 2.2, and Tables 2.2, 2.3, respectively. Humans issue complex instruction-sets to the robotics teammates. Each instruction-set is dened as a set of tasks, arguments, event- and temporal-based conditionals. Based on the dierent outcomes of the stochastic parameters related to the environment, events, and the states of other teammates, a robot may perform various sequences of tasks for the human-provided instructions. The rst step is to convert the instructions to a pseudocode format in order to properly identify theif andwhile statements along with the conditions. All the conditional statements of interest in this work can be expressed mathematically with the functions and variables described in this section. Additionally, we identify proper arguments for each task in the instruction-set. These aid with generating a task transition model for each robot where each transition 16 Figure 2.2: An example of the language decomposition oered by our proposed framework. English instructions provided by the human (a) are converted to pseudocode (b) that is used to generate a task transition model (c). is dictated by a condition. We provide an illustrative example of language decomposition for a snippet of an instruction-set in Figure 2.2. 2.3.3 System Architecture Our proposed framework is targeted to be used by humans supervising a team of robots in a challenging large-scale mission. The task transition model derived from complex instruction-set is used to model each robot's behavior as a state machine. Whenever any robot returns to a human with new information, they provide state information of other robot teammates encountered in the eld. We assume there are some estimated models of dierent stochastic parameters within the entire system. Using these models, the latest state information, and the state machine models of the robots, forward simulations of the entire system are performed. These simulations are done at appropriately high- level, in order to perform a large number of simulations quickly and generate immediate alerts. Each simulation run is a collection of parallel, but inter-dependent, state machine simulations for all robots in the eld. The results of simulations give probabilistic estimates on feasible outcomes in the mission. Simultaneously, humans dene their preferred list of unwanted situations that they feel are important to detect. These contingency conditions are then expressed as mathematical propositions. Our framework provides an inference engine that utilizes the results 17 from the simulations, and nds Truth values for the user-specied alert conditions. If any of these become True, the framework shows the corresponding alerts to the humans. This alert can be based on probabilistic estimates from the simulations, or it can provide guarantees on particular situations happening with 100% certainty. The proposed framework is represented in the block diagram in Figure 2.3. Figure 2.3: Block diagram of the proposed alert-generation framework. The grey slanted rectacles are data blocks and blue rectangles are processing components of the system. 2.3.4 Specication of Alert Conditions We have identied some exemplary alert-triggering scenarios that humans may nd useful and relevant to an HA/DR mission. We also outline mathematical expressions of the alert conditions for detection of these situations. We formulate the conditions in a probabilistic temporal logic framework [102], using dierent parameters and functions. Probabilistic temporal logic is a powerful language to mathematically express many kinds of complex conditions. Humans are free to choose dierent alert conditions from a potential list relevant to each mission, or craft their own conditions based on their preferences. The description of several alert situations are enumerated in this section, and the mathematical expressions relevant to these contingencies are provided. Note, this is not a comprehensive list for a mission, rather we provide some worthwhile examples depicting dierent types of fundamental expressions and conditions. In our proposed framework we use Metric Temporal Logic (MTL) [151] specications to detect a particular unwanted situation in a single mission outcome. Mission progression ends at some time T end once all of the robots are in returned or disabled states; after that time we can assume the entire system remains constant. Let be a set of atomic propositions, crafted from the aforementioned items relevant to the 18 mission threads, and the MTL formulae are built from using Boolean connectives (and^, or_, not :), propositions >(True) and? (False), and time-constrained or -unconstrained versions of temporal operators (eventually; always; next ; untilU). A time-constrained temporal operator is I , where 2f;;Ug, and time interval I (0;1), while the unconstrained version is (0;1) . If we generate a large number of probable strings of mission threads, representing the dierent ways the mission can progress, we can compute the fraction of strings that satises an MTL formula. This fraction can be taken as the estimated probability which is compared with a threshold valuep th in order to detect high or low probability conditions. P p th indicates that the probability of being True is p th , where,2f<;;>;; =g, 0 p th 1, and is an MTL formula. For clarity, we provide some examples of MTL formula similar to the ones used in Table 2.2, along with their implications in words. • ( (t1;t2) ) is True i eventually at some time between t 1 and t 2 , is True • (( 1 ! 2 )) is True i eventually at some time 1 is True and right after that (at the next time instance), 2 is True . • (( 1 ! ( 2 U ft 0 g >) ) is True i eventually at some time is True, and right after that 2 remains True for next t 0 1 time instances. Table 2.2: Example alert condition specications: Probabilistic estimation for enumerated situations Alert # Condition descriptions Expressions of conditions Non-zero probability of robot Ri having states S1;S2; P>0[((S R i =S1)! (S R i =S2)! (S R i =S1) 1 S1;S2; respectively in four consecutive time steps ! (S R i =S2)] (oscillation between states S1;S2) Where, (fS1;S2g S) ^ (S16=S2) (i) High robability of robot Ri and Rj never being (i) P p th [ (t 1 ;t 2 ) : (L R i =X^ L R j =X))] 2 together at scheduled location X within time (t1;t2) (ii) P p 0 th [Bj _Bi ] (ii) High probability of one of the robots Ri;Rj being Where, Bi := ( (t 1 ;t 2 ) L R j =X)^ ( (t 1 ;t 2 ) :(L R i =X)) at time & location while other one does not Bj := ( (t 1 ;t 2 ) L R i =X)^ ( (t 1 ;t 2 ) :(L R j =X)) (i) Low probability of robot Ri attempting rescue (i) P p th [ (CountEventsi(rescue; 2;X; 3;i; 4;j)> 0 )] on Rj at location X 3 (ii) Non-zero probability of robot Ri attempting rescue (ii) P>0 [ (CountEventsi(rescue; 2;X; 3;i; 4;j)>n )] on Rj at location X more than n times 4 High probability of robot Ri navigating a risky region P p th [isRiskyRegion(L R i )] High probability of more than N Y robots exploring P p th [(' Y ! (' Y U ft 0 g > ) )] 5 region Y at once (more than duration t'), where N Y Where, ' Y := (CountExploringRobots(Y ) N Y ) is maximum number of exploring robots for that region (i) High probability of Ri travelling to rendezvous for (i) P p th [(ToRend R i ! (ToRend R i U ft 0 g > ) )] more than a time duration t 0 6 (ii) High probability of Ri navigating more than t 0 (ii)P p th [(EndRendi! (IsNaviU ft 0 +1g > ) )] duration towards the next task, after any rendezvous 19 The following items narrate some contingency situations that human supervisors may want to receive alerts for. The detection condition corresponding to each situation and its mathematical expression are provided in Table 2.2. How to choose appropriate probability threshold values, in accordance with human preferences on certain situations in a mission, is to be considered for our future work. 1. Wastage of time due to oscillatory states Poorly-dened conditionals in an instruction-set can cause a robot to oscillate between two states which might not be intended by the human commander. This may waste a signicant amount of time without any progress. 2. Improbable rendezvous A pre-scheduled rendezvous between two robots might not happen if humans make mistakes in issuing the commands. It can also be missed if a robot becomes inoperable, or skips the rendezvous to execute a separate branch of tasks due to situations and specications. In crafting the condition expression, we assume that both robots arriving at the designated location within the scheduled time period is sucient for a successful rendezvous. 3. Improbable or redundant rescue attempts A specic rescue task instructed to a robot may never actually occur due to certain, obstructing events. Another unwanted situation can be where a robot continuously attempts multiple rescues of the same robot, when it is not intended behavior. Incomplete or improper conditional statements tied to the rescue task can be a possible reason for this issue. In a broader sense, redundant rescue might also refer to multiple robots attempting the same rescue. 4. Navigation through a risky region If humans receive information about a newly-assessed, risky region they might want to check whether any robot in the eld is likely to navigate through that region based on its instructions. Moreover, humans can erroneously assign a robot to a risky region. If humans are notied of this condition in a timely fashion, dangers can be prevented. 20 5. Redundant exploration Every region, based on its characteristics, has an optimal number of robots for achieving fast, collab- orative exploration. If there are too many robots exploring a relatively small region together, it may make the exploration process inecient. We have provided an expression for such conditions in Table 2.2. Likewise, redundant exploration may occur when a robot is exploring a region that has already been explored by another robot. 6. Excessive travel time for rendezvous or rescue Humans might want to avoid having rendezvous or rescue at a location where a robot needs to navigate for a long time from its previous task or to its next task. We have provided expressions for rendezvous in Table 2.2. There can be many other alert conditions that humans may want to detect, such as, risky rescue, long travel-time for an unsuccessful task, etc. Any condition of a person's choice can be mathematically formu- lated, used for detection of adverse situations, and produce an alert for the humans. 2.3.5 Methods to Detect Alert Conditions Given the specication of alert conditions, the alert system must identify when an alert condition is expected to holdTrue, and issue an alert to the humans. The alert condition detection can be simulation- or inference- based. Our simulation-based approach issues alerts based on probability estimates from simulations, but it lacks any guarantee. On the other hand, the inference-based approach may not be possible in every situation, but when applicable, it is able to provide a guarantee on certain situation. 2.3.5.1 Simulation-based Approach We use high-level, discrete-time simulations of the mission, where each simulation uses the system model and the latest information. It generates data on a single instantiation that the system could progress throughout the mission, out of innitely many possibilities. We perform a large number of simulations, and observe what percentage of the simulations have a specic, unwanted situation of occurring. This is considered as 21 the estimated probability of the alert condition to hold true. If this probability meets the thresholding specication set by humans, the interface shows an alert about the possible contingency. If we consider detection of an extremely low-probability situation, repeated random sampling or Monte- Carlo simulations may become computationally infeasible due to the excessively large number of simulations required. Our smart simulation feature provides a computationally feasible way to detect even these low- probability circumstances. There are often some critical events that prompt the unwanted situation to be detected. Such critical events are identied from the state or task transition conditions, and are articially triggered during the simulation runs. The probability estimate from these simulations give the conditional probability, and using the probability of the critical event, we estimate the actual probability of the unwanted situation. 2.3.5.2 Inference-based Approach The estimated probability of alert-triggering situations, calculated from simulations, might produce a poor representation of the real scenario, as the uncertainty and size of the system increases. Also, it is dicult to build an adequate model of the stochastic parameters in the mission. Therefore, we attempt to perform quick inferences using only the non-stochastic parts of the system, (i.e., instruction-sets of the robots, maxi- mum navigation speed) instead of using simulations to detect certain contingencies with signicantly-higher condence. For a complicated mission, it might not be always possible; nevertheless, it can produce useful results in some cases. Many of the alert conditions that we have presented refer to the probability of certain tasks being completed, i.e., certain states and/or locations that are reached by one or more robots within a time range. If an alert condition can be modeled in this way, and the simulation-based testing shows absolutely zero probability, this inference-based approach can be attempted. This method rstly converts the state transition model of each robot into a directed graph without the transition conditions. Using a graph search algorithm, like Depth First Search (DFS), to conduct reachability tests, and considering some simple Boolean literals, this approach can provide the humans with a much stronger assessment of an alert condition. In fact, in these cases it can guarantee absolute certainty. 22 Table 2.3: Inference-based alert conditions: Prove that robot Ri can not reach state S f and location L f by time T f Condition descriptions Expressions of conditions 1) Ri is functioning, S f is not reachable from its current (i2M alive ) ^ (S f = 2 Si) ^ (8j2M alive ; state, and no functioning robot will attempt a rescue on (NeverRelay(j;fig) ^ NeverRescue(j;M dis ) ) ) any disabled robot, and not relay new instruction to Ri 2) Ri is disabled and no functioning robot will attempt any (i2M dis ) ^ (8j2M alive ; NeverRescue(j;M dis ) ) rescue on any robot who are disabled 3) Ri is disabled, S f is not a reachable state even if Ri is (i2M dis ) ^ (S f = 2 Si) ^ (8j2Mfig; NeverRelay(j;fig) ) rescued, and no robot (functioning, or disabled robot after being rescued) will attempt relaying new instruction to Ri 4) Ri is functioning, but its earliest possible arrival time to (i2M alive ) ^ (ti +MinTravelT(i;L f )>T f ) destination location L f is later than time T f 5) Ri is disabled; even if it is successfully rescued at the (i2M dis ) ^ (EarliestRescueTime(i) +MinTravelT(Li;L f )>T f ) earliest possible time based on other robots' times and where, if is an array with elements j =tj +MinTravelT(j;Li); locations, it still can not reach L f in time T f 8j2Mfig; EarliestRescueTime(i) =max(min();ti) It may appear as though conrming reachability for an individual agent may be sucient in this inference. However, HA/DR missions are actually much more challenging and tasks, such as rescue and relay, can complicate the inference process. Let, the latest information available to humans on each robot R i 2 R is from a time in the past, t i . Its state, previous state before arriving to the latest state, and location are S i ;S prev;i ;L i , respectively. We can perform a graph search for each robot R i , and obtain the set of all possible states that R i can reach from any state s. Note, an inoperable robot does not have any reachable state unless it is successfully rescued by another robot, and then it returns back to its previous state. Let, S i represent the set of all reachable states for robot R i , from state S i if it is functional, and from state S prev;i if it is not functional. Let M alive ; M dis , M represent the sets of IDs of functional, non-functional, and all robots, respectively. Some instruction-set for a robot may require dynamic assignment of an argument of a certain task, for which the state transition model of the robot's corresponding states might have unknown argument values. For example, the instruction can be \if you see a robot within 20m, rescue it". If thisrescue state is s in the state-graph, s:arg 1 will be `unknown', hence s:arg 1 = 2 M. It is important to dierentiate this unknown argument from known arguments in order to properly assess the probability of rescue or relay operation for a robot. Given all the specications, we would like to prove whether R i will never reach state S f at location L f by time T f . The conditions to be checked for this purpose are given in Table 2.3, using some supporting functions from Table 2.1. The rst three conditions of Table 2.3 test for reachability to the nal stateS f , while the last two check whether it is physically possible for R i to navigate to the destination L f in time T f . 23 2.4 Results We have tested several alert conditions in our Python-based custom simulator for some mission scenarios that capture the inherent challenge for a human to quickly process a complex inference or dependency while observing only raw data. We then visualize the resulting outcomes using Rviz from the Robot Operating System (ROS). We provide a few examples in this section to illustrate the value of our alert system and to demonstrate the applicability of alert condition detection. An example case shows how the inoperability of one robot decreases a second robot's probability of successful rendezvous by between 0 41%, depending upon the time and location of the disabled robot. Without an alert system, it might not be possible for the humans to infer such contingencies under pressure, and would not be able to make intelligent decisions that support resilient operations. 2.4.1 Using Simulation to Issue Alerts The individual robots as state machines in the simulation are dependent on each other and the environment. While staying within a state, a robot keeps performing some low-level actions, which aect itself and its teammates. At the start of the simulation, each state machine, i.e., robot model, needs to be properly initiated according to latest update. We start simulation from the earliest update time instance among the robots, and forward simulate all of the robots whose statuses are not known for respective time instances. Example alert triggering situation: is there a non-zero probability of robot R i oscillating between two tasks T 1 and T 2 ? Let robotR i be currently in the eld, with the instruction-set described in Figure 2.2, except that the commander, forgetfully or otherwise, excluded the conditional phrase, \which has not been helped yet", about other inoperable robots to attempt rescues. In this case, if a rescue attempt fails (disabled robot remains disabled after attempted to be rescued),R i will go back to previous task explore (T 1 ), and immediately come back to the same rescue task (T 2 ) again. Repeated failed rescue attempts will keep this cycle going on. Eectively,R i will keep attempting consecutive rescue attempts untilR j is revived, and waste a lot of time. A more serious problem may occur if it gets disabled after making multiple failed rescue attempts, in case of a high-risk rescue. So, it might be worthwhile to detect any possibility of such situation so that the instruction can be corrected before robot R i 's deployment. 24 As we described, other robots getting disabled nearby as well as failed rescue attempts are the critical events that can cause this oscillation. These critical conditions are identied from the transition conditions on task transition model of robot R i , between the two corresponding tasks. We perform detection of this situation with Monte-Carlo (MC) simulations and smart simulations, and compare the performances. The probability of critical events are estimated from the mission model. We use this example of low-probability situation to illustrate the value of smart simulations. 20,000 MC simulations give probability of oscillation to be 0:00015. Therefore, it requires at least 14,000 simulations to detect this situation condently. However, with only 500 simulations using our proposed framework { a reduction in the required number of simulations by a factor of at least 100 { we compute the probability of oscillation to be 0:00013. Thus, smart simulation can provide an ecient way to approximate the estimated probability, and issue alerts. 2.4.2 Using inferences to issue alerts We use arithmetic operations and graph search algorithms on state transition models to make quick infer- ences, and issue an alert instantly, when applicable. Example alert triggering situation: is there a non-zero probability of Robot R i rescuing Robot R j ? Let robot R i be in the eld with after given the instruction \Explore AoI-1 and AoI-2 sequentially, and rescue any disabled robot within the exploring AoIs. If event-A happens, move to AoI-3, and explore AoI-3 and AoI-4 sequentially." Humans receive information on robot R j being inoperable in AoI-2, and R i is last known to be exploring AoI-3. We need to analyze whether R j will be attempted to be rescued byR i . From a reachability test, the system may conclude in a moment that rescuing R j in AoI-2 is not a reachable state for R i , given its latest known state. Any other robots, given their latest known states, do not have any relay task among their reachable states, therefore no on-eld robot's instruction-set will change. Therefore, it is guaranteed that R i will not rescue R j . 25 2.5 Summary We presented an alert generation framework for human-supervised multi-robot teams deployed in challenging applications. We identied several useful warning situations, and have expressed their conditions using Metric Temporal Logic (MTL) specication. Our state machine simulation with an abstract model of the mission is computationally-ecient, which is essential in HA/DR applications for rapid alert generation. More specically, we probabilistically assessed complex conditions with forward simulations of the system and leveraged smart simulations to ensure a computationally-ecient way of detecting situations that have low probability. We also proposed an inference-based approach to detect some alert conditions with absolute certainty. 26 Chapter 3 Discrete Simulation-based Approach for Estimating Probability Distributions, Predicting Events, and Generating Alerts This chapter is built upon a published work, [5]. 3.1 Introduction There is an open question of how to generate alerts in a computationally-ecient manner with accuracy considerations in estimating event probabilities. Monte-Carlo simulations would require enormous number of forward simulations to build probability distributions. This can be computationally prohibitive since alerts must be suciently fast in order to be actionable by human supervisors. Moreover, our models and simulations of mission outcomes must be accurate enough to predict the possibility of unwanted events. Higher delity data and more simulation runs are used in simulation-based approaches to improve accuracy, but this increases the required computational time. We need to predict probability within seconds in order to generate alerts in a timely manner. This necessitates forward simulations that are much faster than actual time. Traditional robot simulators with physics engines, which provide real-time or slightly quicker simulations, are clearly not suitable for this use. An approximate simulation model needs to be used which properly balances the trade-os between simulation delity and accuracy level. Continuous systems can be approximated with discrete-event-based simulations to reduce the computational cost [156], but these models could misrepresent mission progression or overlook important events that can trigger to alerts. 27 Figure 3.1: A motivating example of mission scenario has a several kilometer-square suburban environment with 12 hypothetical areas-of-interest, where a team of ten mobile robots, I-X, are tasked to collectively explore all regions and nd objects-of-interest (OoI). The gure shows an overlay of the latest mission updates at time 00:57 and corresponding alerts for unwanted situations which have high probability to occur. More details are provided in Section 3.5. Our objective is to develop a method to solve this problem, where the multi-robot mission consists of several robots navigating based on some specied paths in a 3D environment with some velocity variation. Additionally, we assume missions such that the alert triggering and overall mission progress rely mostly on robot locations with respect to time. This paper describes a simulation-based method which enables rapid yet suciently accurate alert generation. We utilize ideas from discrete event simulation methodology, and introduce adaptive techniques for reducing computational burden. Given a path for 2D or 3D navigation of an agent, and the uncertainty model regarding velocity variation, our method can generate dierent realizations of how the navigation may progress with time. Similarly, when there is uncertainty in robots performing other tasks, our method uses prior data collected on the performance of the tasks to estimate task progress with time. We aim to balance the competing objectives of simulation accuracy and alert generation timeliness for missions similar to the motivating example shown in Fig. 3.1. 28 We take a novel approach using a discrete-event simulation framework which can represent the uncertainty in future mission states with respect to time, with sucient accuracy. Using our method, hundreds of simulation runs can be performed within seconds to generate alerts real-time. We introduce a few adaptive techniques, adjusting simulation delity for computational speed-up, while maintaining accuracy in issuing alerts. First, we present a technique that adaptively selects time steps for managing simulation delity and providing suciently-accurate approximations of continuous systems using discrete-based simulations. Second, we propose a time interval selection method that focuses the eorts of simulating mission events to only the durations containing the most salient operations, which provides a signicant speed up of the forward simulation process. These contributions are noteworthy because we show that a modest compromise in the accuracy of individual runs does not sacrice the accuracy of alert generated using Monte Carlo based trials, and also provides a signicant speed-up that is necessary for real-time alert generation. We provide theoretical and empirical insights to this trade-o, which indicate certain worst-case performance and limitations, and may oer guidance in choosing suitable delity level specic to each application. 3.2 Related Works There exists several dierent alert-generating architectures and interfaces in dierent robotics applica- tions [123, 136]. There have also been several systems designed specically for human-robot teams in the humanitarian assistance or disaster rescue context, both with [24, 86, 145] and without alert systems [106]. Since all these works are either reactive or based on small-scale operation where humans are essentially con- trolling the robots directly, real-time alerts are generated solely using current information like sensor data. In large-scale missions, with intermittent, incomplete and delayed information ow, a eldable system must make future mission predictions based on past or incomplete data. Several research papers discuss evaluation of multi-robot mission performance in specic angles. Some works focus on performance assessment of AI algorithms and architecture used for multi-robot systems [23, 76], while other works emphasis on risk assessment, especially in disaster relief type operations [14, 15, 142,188]. [76] introduces a framework for design-space exploration, trade-o analysis of multi-robot system architectures and collective intelligence algorithms. [23] proposes an evaluation model for comparing dierent 29 information distribution schemes, and can be used to nd solution for optimizing information distribution. The cited works on risk evaluation draw their attentions on particular risk modes considering low-level risks. These could potentially appear as building blocks in our alert generation framework for relevant applications. We assume models for the low-level risks (e.g. robotic failures, task failures) involved, environment and robot behavior, including uncertainty, are available to be used in our simulation-based mission prediction. Our goal is to estimate probability of mission contingencies under uncertainty, and generate alerts. In order for a user interface to convey future outcomes and the associated uncertainty for a multi-robot mission, forward simulation is necessary [114]. Common simulators such as Gazebo, VREP, and ARGOS are available for multi-robot simulations. All three oer varying capabilities in physics simulation within large environments where a single run can occur in real time or slightly faster [160,161], but they are not designed for our application as discussed before. Forward simulations that require a large number of computationally fast runs will require discrete event-based simulation (DES). This method simulates systems in discrete time steps and reduces the complexity because the requirement for synchronization is reduced. Multiple works have demonstrated that DES can be used for ecient large-scale multi-agent simulation and results show reduced computational time [43,49,49,61,62,83,198]. However, these works do not attempt to use DES for estimating probability of future events in complex missions under uncertainty. While many of these methods are computationally fast, methods with too low delity do not ensure probability estimate accuracy. Each of the mentioned papers choose some arbitrary level of abstraction to represent task performance by robots in dierent applications. Therefore, directly using one of these approaches might not be suitable for our alert generation framework. We need a computationally feasible discrete-time simulation approach while ensuring accuracy in building probability distributions, with theoretical and empirical analysis, which the cited works do not perform or include. 3.3 Generating Alerts Using Simulation Our alert generation framework is intended for human-supervised, multi-robot teams operating in large, unstructured environments where high-delity, physics-based simulations are infeasible due to an intractably large state space. The key to our approach lies in combining signicantly-faster-than-real-time simulations 30 Figure 3.2: The system architecture diagram of our proposed simulation-based framework for generating alerts. with representative, data-driven models. Prior to the start of the mission, we envision operators building models of the robot's capabilities for individual tasks by recording empirical samples in the operational environment. For example, the robot could execute several short distance autonomous navigation trials near the team's deployment area to build lower-order statistical models that represent its ability to move autonomously throughout the environment, e.g., average and standard deviation for time to drive some unit distance. Constructing models in this fashion ensures mission relevancy and requires much less time and technical expertise than attempting to build sophisticated a-priori models for traditional simulators. It also oers exibility in that a working model could be created quickly, if time is limited, and then improved at any point by collecting more trial data, including during the mission. To execute the mission, humans deploy the team of robots and provide instructions for the robots to perform tasks that work toward a common goal. Information regarding the mission may be delayed or intermittent due to communication constraints. Nevertheless, humans must craft and continuously update strategies to accomplish the mission using currently-available information while intelligently prioritizing assets and tasks based on what appears to be most critical. We seek to assist the humans in this decision-making process by providing alerts based on proactive, forward simulation of the mission that leverages the characteristic models built in the eld. 31 Our system architecture is shown in Fig. 3.2 and builds on our previous works [10]. A series of simulations is performed for the mission and robot states to predict certain critical aspects related to human-specied alert conditions. Accurate probability estimation of any mission parameter requires a certain minimum number of simulations (N). Alerts need to be generated rapidly, and so physics-based robot simulators are not expected to work for this application. We use discrete event approximation, along with additional simulation speed up techniques, in order to make the alert generation process faster. The details on how and why it works are provided in the subsequent sections. Firstly, a number (N 1 ) of exploratory runs are performed using xed time-step discrete event simulations (see Section 3.4.1). From the ndings of these simulations, groups of robots along with corresponding time intervals are identied, based on which robots share interactions and when, or in which time interval consequential events occur (see Section 3.4.3). Using these groups and time intervals, we perform the remaining number of simulations (NN 1 ) with adaptive time steps for further reducing computational costs (see Section 3.4.2). Finally, the data from all N simulations are used to generate the spatio-temporal probability distributions of the mission parameters that are relevant to estimating event occurrences and generating alerts. Our earlier work [10] highlights how alert triggering conditions are mathematically modelled as Probabilistic Metric Temporal Logic formula in our system. Each condition corresponds to one or more mission parameters to hold certain values during some time durations. Finally, our system generates alerts by comparing the computed event probabilities with the corresponding alert condition (see Section 3.5). In this work, we focus on developing an informative approach for conducting simulation based assessment and alert generation by taking accuracy considerations into account. To achieve this, we seek to choose appropriate level of discretization for simulating robots in the mission. We use a discrete velocity event simulation model which is capable of encoding varying slowdown throughout each navigation task as opposed to averaged out slowdown used in our previous approach. Our approach use adaptive time-step with robot groups and time interval selection to ensure ecient computation, which extends the variable time-step idea from our previous work. In this work, we also provide insights on choosing the appropriate number of simulation runs within DES paradigm, as we analyze the accuracy of our estimation. This improved method is capable to support more accurate simulations within limited computational time, and the estimated 32 probability distributions can be made suciently accurate which are used for generating alerts. Therefore, these alerts can help humans perceive the mission situations more precisely, and improve human decision making. 3.4 Estimating Spatio-Temporal Probability Distributions Figure 3.3: Assume in a mission, robot A and B are navigating through an environment based on individual task- plans, and the human supervisor would like an alert based on these robots' interactions within a time interval (e.g., when their distance is less than 50m). The probability histogram of the relevant spatio-temporal parameter dmin, generated from simulations, is shown in the gure. Our goal is to generate probability distributions of spatio-temporal mission parameters in order to predict how the mission may progress and generate alerts. To build these distributions, we require a large number of trials that can be obtained from Monte-Carlo simulations. The latest mission updates are used to seed the simulations, and task-plans and uncertainty models (e.g., environmental terrain and navigation performance) guide how the simulated missions progress. Human-set alert triggering conditions specify the relevant spatio- temporal mission parameters. Each simulation run provides one instance of each mission parameter value which is considered as a random sample towards building the probability distributed relevant to an alert (See Fig. 3.3). 3.4.1 Discrete Velocity Event-Based Simulation Our multi-robot mission is comprised of ground robots moving around in an environment for extended periods of time to perform tasks assigned by human supervisors. We assume that there is a model available for each robot's autonomous navigation capabilities. Path generation for robot navigation is determined by 33 a planner while navigation goals are obtained from robot instructions, and there is a probabilistic model of slowdown available for any set of way points. While it is theoretically possible to use robot simulators like Gazebo to simulate the robots navigating within the environment, performing a large number of MC runs might not be computationally feasible. Available physics engines typically operate with constant, very small (millisecond) time steps, dt, on the order of milliseconds to simulate continuous systems with high delity; however, this incurs a substantial computational cost. For example, the velocity-integrator in the Gazebo simulator uses a default maximum time step size of 1ms [1] and, as a result, 30 minutes of simulated operation for a single robot would require nearly 2 million computational steps. In our experiments, 30 simulated minutes of navigation for a single Husky 1 driving in a straight line requires 8:12 seconds using the PyBullet 2 physics-based simulator on an Intel Xeon CPU E3-1245 v5, 3:50GHz processor. This example assumes the simplest scenario of one robot on a at surface using a default friction model and the largest suggested time step (10 ms) indicated in the software documentation. Extrapolating this example to a team of 10 robots in a long-duration mission, a single run for a navigation task will require more than 4 minutes. A realistic mission will require several kinds of complex tasks, update many variables, and use hundreds, possibly thousands, of simulations for a single instance of probabilistic state estimation and mission outcome prediction. Therefore, conventional simulation techniques will be prohibitively slow. Ideally, we would like to perform hundreds or thousands of simulations to ensure accuracy in probability estimates. Additionally, it is important that we complete all simulations rapidly, preferably within seconds, to generate useful alerts. In order to do that, we approximate the continuous system with the occurrence of specic events in order to achieve a signicant speed-up in simulation time. For the purposes of this work, these events are major changes in the velocity prole and we refer to this simulation scheme as discrete velocity event-based simulation or as discrete simulation in short. In our scheme for robot navigation, we can use time step sizes on the order of seconds or minutes, depending on the mission and robots. Although velocity is naturally continuous, we instead use suitable time steps, t, to approximate a continuous velocity prole by accounting for when the nominal velocity is constant. If any t time window includes a change in the velocity value in the continuous system, the 1 https://clearpathrobotics.com/husky-unmanned-ground-vehicle-robot/ 2 https://pybullet.org/wordpress/ 34 discrete simulation technique randomly chooses a single value from the continuous velocity values within that time window. Fig. 3.4 illustrates a simple example of probabilistic mixture model for discrete simulation, which ensures that the average velocity from the proles in the mixture converges to the average velocity of the continuous system. Discrete representation of a more realistic continuous-time velocity prole, with gradual velocity change, drift, and error. is depicted in Fig. 3.5. Even though there exists pairwise mismatch between a continuous system and a single discrete representation sampled from its corresponding mixture, we achieve the same average values when we aggregate from a large number of simulations. Figure 3.4: A continuous-time velocity prole (blue) is represented as a probabilistic mixture model of the two discrete proles (red) with t = 10s. The mixture probabilities are such that the average velocity of the mixture is the same as the average velocity of the continuous system over the time period. Table 3.1: Average and standard deviation values for mission parameters,L andT , usingN = 250 andN = 20; 000 trials of continuous and discrete simulations. Spatio-temporal Number of runs, N = 250 Number of runs, N = 20; 000 Mission Mean Std Dev Mean Std Dev Parameter Cont. Discr. Cont. Discr. Cont. Discr. Cont. Discr. L (T = 500s) 496 14 515 15 116 11 122 12 509 2 509 2 120 1 120 1 L (T = 1000s) 1017 21 1006 25 174 17 202 19 1006 3 1005 3 181 2 182 2 T (L = 600m) 624 16 599 17 127 12 138 13 612 2 611 2 136 1 136 1 T (L = 1200m) 1199 26 1211 25 212 20 200 19 1207 3 1209 3 205 2 204 2 Let's consider we want to build probability distributions for two spatio-temporal parameters related to a single robot's navigation: L , the navigation distance covered after a specic time T , and, T , the time 35 Figure 3.5: A continuous-time velocity prole (blue) is represented as a probabilistic mixture of many discrete proles (red) of which three are shown in this gure. duration for navigating a specic distance L. Note that L and T are two representative examples of mission parameters that are useful for triggering alerts. We perform N continuous and discrete simulations separately, and compute the mean and average for the built distributions. When a distribution mean, , and standard deviation, , are computed from N number of samples, we can theoretically compute ; such that we can say with 95% condence that the actual mean and standard deviation will lie within and ranges, respectively [59]. These ranges from our experiments are presented in Table 3.1 which shows overlaps in the ranges from continuous and discrete runs. We can observe that the aggregated average and std-dev values of discrete simulations converge asymptotically to continuous' as N becomes larger. In fact, the calculated mean values of discrete and continuous simulations will always match asymptotically, due to our random sampling for the discrete velocity prole, given the drift/noise having Gaussian properties. However, it is possible for the discrete system to have some asymptotic deviation in from the continuous system if the time step used in the discrete simulations is too large. We have experimentally observed that if the navigation duration T and t in discrete simulations are such that T=t is at least four times the number of nominal velocity changes, the standard deviation of the discrete system converges to below 1:5% deviation (%) from that of the continuous system. This experimental setup was very conservative, 36 and realistically much more favorable values can be used, bringing down the asymptotic deviation to zero like results in Table 3.1 indicate. In our experiments, the nominal velocity changes about 8 times in 1000s, and v min = 0:25v max . Figure 3.6: Plot of percent maximum error in estimating distribution standard deviation (with 95% condence) versus computational time (unit: computation time for one continuous run). For a very conservative case, r = 10 and = 5%, we can nd Nc = 416. This means, for a total computational time less than the time required for 416 continuous runs, discrete simulations will always perform better with smaller estimation error. Figure 3.7: Percent maximum error in estimating standard deviation versus computation time for r = 100 and = 27%. We can see that discrete simulations are advantageous up to very large numbers (Nc) of continuous runs. We would like to study accuracy in building probability distributions using discrete velocity event-based simulations compared to continuous, since the quality of generated alerts depends on accurate event estima- tion. Let's assume the estimated probability distribution can be characterized by the estimated mean and standard deviation. Inaccuracies in estimation from using a limited number of simulation runs cause the deviation from the asymptotic value using the same model, which applies to both continuous and discrete individually. Approximate discrete runs may introduce additional error due to asymptotic deviation from 37 the actual (continuous) system, as described before. Considering both of these sources of error, we compare estimation performance of continuous and discrete simulation using the number of trials that can be done within a certain amount of computational time. Since discrete velocity event-based simulation is faster, more simulation trials can be performed within the same time period, but each individual trial may be less accurate than one performed using the continuous simulation. To assist with our performance analysis of continuous and discrete velocity event-based simulation, we introduce the following assumptions and mathematical notation. (1) Discrete simulation generates probabil- ity distributions of system parameters, where any standard deviation value asymptotically converge within % deviation of the corresponding standard deviation computed from the continuous system, as introduced before. (2) Each run of the discrete velocity event-based simulation is computationally r-times faster than a run using the continuous system. (3) N c is the critical number of continuous runs, for which the maximum error in estimated standard deviation, of a mission parameter, with 95% condence, is the same as the error with discrete system runs that can be performed within the same computational time (see Fig. 3.6). If we need to perform simulations and generate alerts in a time duration less than the amount of time required for doing N c continuous runs, discrete simulation will always produce more accurate estimations of . Discrete simulation provides convergence to the estimated mean of the continuous system and deviation between the two systems only occurs in the estimation of standard deviation. We observe that within a feasible range of r and values, N c is large enough such that discrete runs always ensure greater accuracy within the maximum computational time that is feasible for alert generation. Fig. 3.7 depicts how the estimation error of changes with respect to for discrete systems withr = 100, as compared to the continuous runs in the same computational time. We have used theoretical error values from 95% condence intervals for estimating standard deviations based on the number of simulation trials [59]. For discrete runs, we have also added the convergence error (%). We observe the magnitudes ofN c for relatively smallr and large values. More practical values would always result in even higherN c . For example,8r> 1, < 3 give N c > 2000, where 2000 is already too high a number for continuous runs (computation time can be in the order of several hundreds of minutes [6]). Therefore, using discrete simulation eectively reduces the error in probability distribution estimation while ensuring feasibly low computational time. 38 3.4.2 Adaptive Time Step Size A traditional way of any discrete-time simulation is to use a xed time step size, t, that is suciently small to capture some desired level of granularity. In many missions, there are certain time periods when nothing majorly impactful occurs and other time periods that precipitate interesting phenomena. We present a tech- nique to dynamically adjust the time step size such that larger t values are used during non-consequential time periods, and smaller t values are used to characterize events of interest. Figure 3.8: Illustrative example of using adaptive time step size, t, in discrete velocity (red) simulation for a single robot's navigation. As an illustrative example of adapting time step sizes, let us consider the simulation of a single robot's navigation, as depicted in Fig. 3.8. Using velocity proles similar to Fig. 3.5, a xed t = 10s would result in 20 computational steps for the total 200s navigation duration, whereas adaptive t could lower it signicantly. By dynamically adapting t with respect to occurrence of velocity change events, we can simulate only t = 0s, 50s, 60s, 150s, 160s, and 200s. Furthermore, we can extend this concept to more complex multi-robot missions by intelligently selecting the time step used for simulation for a variety of events, tasks, and robot interactions and eectively reduce the computational cost of simulation. 39 3.4.3 Robot Groups and Time Interval Selection Forming robot groups and time interval selection make adaptive time step size technique feasible for multi robot missions with complexity and uncertainty. Consequential time periods along with corresponding robot groups can guide the adaptivity of t values. Humans can beforehand specify the consequential events (e.g., mission updates, robot interactions etc.) that need to be captured, let it be for one robot or between a group of robots. If we simulate all robots together using the same time steps, the adaptive process becomes inecient since dierent robot encounter changes or updates at dierent time stamps. Robot groups, along with corresponding time periods allow to simulate robots separately whenever interactions or dependencies don't occur. This means each robot or each robot group's simulation can use a dierent set of time steps that is favorably adjusted based on that robot or that group's state and relevant environment. Figure 3.9: An illustrative example of identifying a group of robots and selecting a time interval for high delity simulation. Fig. 3.9 provides an example of robot grouping and interval selection for two robots autonomously navigating in an environment. These two robots will interact at the cross-section of the two paths provided both robots are physically there within some time period. Thus, a two-robot grouping can be labeled as consequential in the adaptive-t process so that two robots are simulated together with higher delity within that interval and simulated separately at other times with lower delity. 40 Figure 3.10: Illustration of the method for identifying consequential time intervals along with respective robot groups from N1 exploratory simulation runs. We depict the process of time interval selection with robot groups using data from N 1 exploratory high- delity mission simulations in Fig. 3.10. We use two examples of consequential events that can be identied in simulation runs: (1) rendezvous between two robots (4), and (2) a particular robot identifying an object- of-interest (). The occurrences and durations of the events vary from run to run, and the event may not even happen in some simulation run (i.g., in simulation #2). We take the minimum-length time interval such that particular event occurrences in allN 1 runs fall within that interval, and form groups dened by the robots involved in the event for that time interval. For the case in Fig. 3.10 when we perform the remaining NN 1 simulations, Robot 1 and 2 are simulated with high delity during mission time interval 4 , but with adaptive time step size during rest of the time to reduce computational time. Note that in a mission where most of the time horizon is considered consequential for every robot, this technique will not add much value. However, we anticipate that to be rare in most long duration missions. 41 3.5 Event Prediction and Alert Generation Alert generation happens based on event predictions using the probability distributions of relevant spatio- temporal mission parameters. An alert triggering condition species that an alert should be generated when a certain situation may occur with a probability greater (or less) than a human-specied value. Any situation can be expressed with one or more mission parameters taking specic values. An example alert condition based on the event depicted in Fig. 3.3 would be whether the probability of d min < 50m is less than 0:7. Case Study: To validate our approach, we generated alerts for three representative mission scenarios using discrete velocity event-based systems with our proposed enhancements. We compare the results with discrete simulations with very small time step size which are expected to produce the exact same results as continuous system. Scenario 1 is from a mission where ten mobile robots explore some areas of interest in a several kilometer-square environment, and search for objects of interest (OoI), and it is depicted in Fig. 3.1. In these missions, robots can perform rendezvous with other robots, relay information or new instructions to other robots, and become disabled due to the environmental complexity, in addition to navigation and exploration for OoIs. Since there can be signicant latency in mission updates, the statuses shown are from dierent time stamps from robots as they become available. Humans can access this information as well as the current task-plans of all robots. The mission is incredibly dicult to predict because of the complexity in robot instructions, inter-dependency of dierent events and tasks, and potentially multiple con icts occurring simultaneously. Even with simpler mission scenarios than the ones used in this paper, we found that humans are likely to fail when predicting the unwanted mission situations occurring [6]. In Scenario 1, two risky regions have been discovered, which should be avoided by the robots. However, based on the task plans and information states of the robots, some of the robots may navigate into the risky region and become disabled as identied by Alert #1. This would cause robots to miss rendezvous within certain groups, which will trigger some other robot to move from its initial region to another region. Additionally, based on how OoI-search has progressed, certain regions may require more robots than others, and humans need to know which regions are likely to lack in the number of robots to accomplish the mission in a timely fashion; this is indicated by Alert #2. Each of the other three mission scenarios used in this section have similar environment scale, mission complexity, and one generated alert in each mission. 42 3.6 Results During alert generation for our case studies, We use t = 0:1s for the high delity simulations, representative of continuous system, and t = 10s along with additional adaptivity for discrete simulations using our method. We assume that the data from 20; 000 continuous-equivalent simulations provides highly accurate estimates, which we consider as the baseline probabilities to compare our results with. In our method, N 1 = 100 exploratory runs with xed t and remaining 150 runs with maximum adaptivity are performed, and thus the probability estimates are computed from a total of N = 250 simulation runs. Table 3.2: Probability estimation of four dierent alert triggering situations across the three representative mission scenarios Computed Range of Baseline Estimated Probability Probability at 95% Condence Probability (N = 250) (N = 250 samples) 0.73 0.69 0.63-0.75 0.82 0.84 0.79-0.89 0.93 0.96 0.93-0.99 0.65 0.64 0.58-0.70 We found that a total of four alerts were triggered across the three mission scenarios, two of which occurred for scenario-1 as mentioned before. These four alert triggering situations corresponded to not having an adequate number of robots in a region, robotic failures, missed rendezvous, and a robot navigating through a risky region. Table 3.2 shows the baseline probabilities and the estimated probability using our method for these four situations. If we assume these events in each simulation can be represented as a Bernoulli variable, where 1 and 0 indicate the event occurring and not occurring respectively, these N simulation runs can be considered as samples used in the estimation of a Bernoulli parameterp, i.e., the alert triggering event probability. The third column in Table 3.2 shows the computed values of the Wald-type 95% condence interval [59] for the actual value of p, when 250 samples are used in the estimation. We observe that all baseline probability values do fall within the condence intervals, which indicates that the small deviations between the baseline and estimated values merely represent the error due to using limited number (N) of simulations. Thus, the accuracy of our discrete simulation model representing the continuous system deems 43 sucient for our application. Secondly, the computational time taken in these alert generation process is found between 5 21s. Thus, we demonstrate our accuracy in estimating relevant probabilities, while being able to generate alerts real-time. We also observed that adaptive simulation runs were 1:44 2:28 times faster compared to having xed t = 10s. 3.7 Summary We presented an alert generation framework that is applicable for any multi-robot mission where mobile robots experience velocity variations while following some prescribed paths over 3D terrains, and one wishes to forward simulate spatio-temporal parameters of the robot's future states to trigger alerts conditioned on the robot's location with respect to time intervals. In this work we demonstrated how to perform discrete velocity event simulation for a team of mobile robots in complex missions, by approximating the continuous-time system. We showed that this discrete simulation method can provide accuracy similar to continuous systems. We provided experimental observations with theoretical insights on how discrete simulation outperforms the corresponding continuous system in terms of estimation accuracy with respect to the required computational time. Lastly, we presented the results on accurately estimating alert triggering event probabilities, simulation speed up using our adaptive techniques, applied to three complex mission scenarios, allowing for alert generation in complex missions within seconds. 44 Chapter 4 Preliminary Assessment of Alerts Enabling Humans to Identify Mission Contingencies Using a Human Subject Study This chapter is based on the published paper [6]. 4.1 Introduction In this chapter, we describe a method for forward simulation model enhanced with smart features to signi- cantly speed up computation, which is slightly dierent from the method used in [5]. The traces from mission simulations are compared with user-customizable, mathematically-encoded alert conditions to automatically generate alerts in real-time. The main contribution lies in primary assessment of usefulness of our alert system. We conducted a human subjects study and veried, with statistical signicance, that alerts enable humans to identify mission contingencies better and thus improvement human decision-making. 4.2 Related Works Researchers have focused on interface design to control the robots [144], the human factor concerns associ- ated with supervisory control of multi-robot systems [38,186,221], workload and task performance of human commanders [74], ecient task execution by multi-robot teams [120], and system development [106] speci- cally for search-and-rescue missions. [56] demonstrates that dynamically adaptive autonomy level based on 45 situational needs can ensure ecient use of human attention and cognition during remotely operating mul- tiple robots. [54, 89] focus on optimal level of task autonomy for robotic systems versus what key decisions are necessary for humans to be aware of. User-friendly, man-machine interfaces are important for communi- cating commands and information in critical missions, and ubiquitous computing in real-world applications is a research goal at present [78]. [176] highlights the potential of adaptive and immersive interfaces in the improvement of workload, situational awareness and performance of operators in multi-robot missions. [177] demonstrates immersive monitoring and commanding interfaces development for multi robot systems, which can improve the operator's situational aware-ness without increasing the workload. Several characterizations and approaches to this research space have been proposed that address factors such as trust, awareness, cognitive workload and fatigue [38,73,202,221]. In particular, the authors of [35,35] developed the Situation Awareness-based Agent Transparency (SAT) model and assert that SAT promotes eective human-autonomous agent teaming. This model denes three levels of transparency between the agent and human, which include current goals and actions, agent reasoning process and projection of system's future outcomes. This research has been continued in [34, 182, 222] which have experimented with varying the autonomy and the situational awareness transparency level through a user interface. Results show that greater agent transparency leads to better situational awareness [175], trust [36,194], and cognitive process- ing [75]. [179] describes a situational awareness-based framework for design and evaluation of explainable AI, where their human-centric approach focuses on informational needs of humans. A way to categorize dierent types of errors in intelligent robotic systems towards those needs in [185]. Research in user-friendly, man-machine interfaces as well as alert generation frameworks are therefore important for communicating commands and information in critical missions [24,78,106,144,175]. 4.3 System overview We have developed an alert generation framework for human-supervised, multi-robot teaming applications in large unstructured environments, e.g., disaster response missions and military operations. In these scenarios, humans dispatch a team of robots into the operational environment to explore the aected regions eciently, collect mission-critical information, and perform certain tasks. The humans provide the robots complete 46 Figure 4.1: System architecture proposed in this work that builds on our previous work [10]. task plans which may include a nominal task sequence, along with some interrupt tasks, and contingency task plans. Thus the high-level mission strategy is crafted by the human supervisors to meet some mission objectives. Meanwhile, the robots are designed to be capable of making the low-level decisions for performing the tasks given by the humans, such as navigation and exploration, identication and manipulation of objects of interest, and execution of other mission-specic tasks. The entire system architecture is depicted in Figure 4.1, which includes the alert generation framework from our earlier work [10]. Humans receive and view mission updates from the ongoing mission, via a user interface. This mission update, along with current task plan of robots in the ongoing mission, and human- specied unwanted situations to trigger alerts, are then fed into the alert generation framework. We have outlined in [10] the mathematical framework and representation of dierent alert triggering conditions as Metric Temporal Logic (MTL) formulae, the complex task description structure of robots, and the state machine representation of robot behavior to be used in mission simulation. In the current paper, we focus on mission- and task-models for forward simulation, computational speed up techniques, alert extractions, and nally, the usefulness of humans being notied of the alerts via the interface while assigning tasks to the robots. In a large-scale environment, there are usually some regions of higher importance which we call areas-of- interest (AoIs). We assume that the human supervisors use their expertise and protocols to identify these regions and mission goals are set with priorities given to searching or performing tasks in the AoIs. We also assume limited communication in this mission, which causes intermittent or degraded data ow to humans. 47 The humans can only receive mission updates in some time intervals depending upon the communication constraints, instructions given to the robots, and how the mission progresses. As the robots operate in the complex mission space, there is a non-zero probability of failure, which can be due to environmental factors like complex terrain, or stochastic events like hardware or software failures. In some cases, if a robot is disabled or immobilized, we assume that another robot might be able to revive the disabled robot by providing assistance. We call this task robot rescue [7, 8] and the diculty and risk of these rescue operations depends on the specic situation. The human supervisors issue task allocations to the robots from a command center, which is not necessar- ily co-located with the robots. We assume that the humans have some method by which they can eectively communicate with the robots and monitor mission progress using some display. Display modalities could include computer monitors, tablets, augmented or virtual reality displays, or any other interface by which humans receive mission-relevant updates from the robots. In order to be most eective, the humans need to adapt their strategy as they receive information from the robots. Whenever a robot becomes available to be tasked, the humans must make informed decisions about how to best use the asset to accomplish the mission. For every real-world mission, there is likely a set of critical situations that a human supervisor is concerned about. Some of these events may be unwanted situations that are detrimental to the team or mission performance and, in the best case, are prevented or mitigated through improved decision making. However, intermittent data ow makes this process very challenging because humans will receive information at a delayed time. Whenever there is new mission information available it is crucial that humans assess the mission situation and formulate a mental model for how the mission has been progressing since the past updates and how it will progress in near future. This way humans will be able to allocate available resources appropriately. We propose a framework to assist the human supervisor with informed decision making in an eort to overcome the burden of mission modeling and mitigate the negative impact of delayed information ow due to intermittent communication. We introduce forward simulation-based alert generation, which we believe will be very useful for the human commanders of multi-robot teams because it will reduce the cognitive load 48 required to manage teams. The forward simulation uses updates from the past to predict what is currently happening outside of communications range, or what will happen in near future. Human supervisors provide the mission conditions that they value and want notications for as to ensure that the alerts provided to the human are relevant and not distracting. These conditions are expressed mathematically in a probabilistic temporal logic framework [102]. Forward simulations of the mission generate traces of information, which are compared to the alert conditions set by the humans. Whenever a condition is evaluated to be true, an alert message with additional details is provided to the human commanders. This, in turn, improves the humans' understanding of mission progression and helps them to prioritize important issues and resources. 4.4 Speeding up forward simulations Forward simulations are needed to estimate how the mission will progress, and to generate alerts regarding whether any unwanted situations are likely to occur. Each robot initiates a simulation using the most up- to-date information they have received and computes for some simulated time into the future. Some robots whose latest updates are further into the past, need to be simulated for a longer period of time which may be on the order of hours. These simulations will produce data on the distribution of robot locations over time and other mission parameters. In the previous chapter [5] we illustrate a discrete velocity simulation model which can generate suciently accurate probability estimates for mission events while being computationally feasible. In this paper, we take a higher level simulation model to generate alerts with the main objective to assess the impact of alerts in human decision making 4.4.1 Discrete event based simulation model We use a discrete event-based simulation paradigm with xed-increment time progression for forward sim- ulating the missions. So, the operation of the system is modelled as a discrete sequence of events in time. The states of the robots are updated at every time step according to the instructions given to the robots and mission updates. To help mitigate computational cost, we supplement the discrete event simulation with representative task performance models, which are generated oine before the actual mission begins and then used in forward simulations during mission execution. 49 The enabling technology in our discrete event simulation is the task performance models. Because this is a pre-processing step, these can be constructed from physics-based simulations or real-world experimen- tation with robots performing relevant tasks in similar environments. To demonstrate the feasibility of this approach, we built distributions for the normalized completion time and position-based error of autonomous navigation using data from a real-world eld experiment, described in our earlier work [69]. Each mission in the experiment consists of a ground robot autonomously navigating from one precise location to another, given no a-priori information, through a variety of terrains in a complex, urban setting (Figure 4.2 (a)). This navigation task resembles the conditions of what a robot could encounter in military or disaster relief operations. For each successful navigation, we computed the amount of additional time required to complete the navigation, relative to the Euclidean path, which encapsulates the uncertainty in the observed speed of the robot in an unknown environment. We also constructed a distribution of the position-based error by computing the distance between the robot's observed position and the corresponding positions in the com- manded global plan. The empirical distributions for normalized completion time and position uncertainty for autonomous navigation are shown in Figure 4.2 (b). One could then use these distributions by sampling each distribution at each time step of the discrete event simulation to determine a possible speed and error that produce the next location of the robot performing the navigation task. Since discrete event simulation is being done at a high level, we can use larger time steps without loss of performance; we typically would not intervals smaller than 30 seconds. We tested alert generation for four representative scenarios (more details in Sections 4.5 and 4.6). We found that alert generation using 100 simulations took between 3:95 4:76 seconds with a time step of 30 seconds, which is at least tens of thousands times faster than running physics-based simulation. Even though this computation time may seem small enough for usage, a mission model with higher complexity might have signicantly more computational cost. Therefore, further improvement in computational time would be benecial. 4.4.2 Variable time-step The goal of our discrete event simulation is to explore the possible ways a mission could progress and identify when certain situations occur, if at all. Therefore, we need suciently small time steps in order to capture 50 (a) Navigation Missions (b) Empirical distributions for normalized completion time and position un- certainty Figure 4.2: Generating distributions of navigation parameters for discrete event simulation using data from eld experiments. the salient phenomena. The traditional way of doing discrete event simulation is to run the simulation using a constant time step, and identifying an appropriate time step is crucial. Smaller time steps ensure higher delity at the expense of higher computational cost, while the low-delity with larger time step may cause some events of interest to be overlooked. We propose a variable time step to reduce the number of steps in the discrete event simulation, while gen- erating sucient, representative mission data. In applicable missions, there are usually some time intervals for each robot when it operates in accordance with its nominal plan with no external events or interactions, and the interesting phenomena occur during other time periods. To identify this for each individual robot, we perform 10 initial simulated runs with a small, constant time step to identify the less-consequential time periods where we can use a larger time step for updating its status, and other times where we require smaller time steps. It is important to identify what the interesting or consequential items are to be searched in those initial simulations. In some of our representative mission scenarios, interesting events included robot- to-robot interactions, robotic failures and rescues, detection of objects of interest, and any external event. We produce some preliminary results using a two-tier structure of time steps, dt (30 seconds) and 10dt (5 minutes), in our adaptive time simulation. For the four scenarios, the two-tier adaptive method achieved 51 between (3:15 5:70) speed up in computation process as compared to using constant dt time step. In the future, we plan to perform hierarchical time steps, so that we can support multiple dierent step sizes instead of only two. 4.5 Extracting alerts from simulation data The data generated by forward simulations need to be compared with the alert triggering conditions to issue alerts. This section denotes a few potential unwanted situations and relevant alert conditions, mathematically encoded in a probabilistic logic framework. One representative mission scenario is used as an example, and we demonstrate how simulation data are processed in our scheme. Time-dependent probability traces for certain mission variables are produced from a series of forward simulations. Then alerts are issued by checking whether the associated temporal logic formulas are found to be true from the traces. Based on each mission objectives, human commanders may care about dierent situations. Some alerts can be based on whether there are or will be sucient number of robots in a region of need. For example, new ndings can make some regions of higher priority than others, which was not initially known, and still not known to many robots in the eld, so humans may want to track whether the high priority regions have a sucient number of robots operating there. If there are survivors involved, humans may want to check whether survivors are getting sucient assistance. If alerts are given, humans can prioritize those regions early by allocating new resources. Alerts can be also useful when humans do not want robots in certain regions any more. If humans receive information on some risky regions, they might prefer other robots not to navigate around that region to avoid potential failures. Another example might be, if one robot unknowingly plans to explore a region that another robot has completely explored already, time and energy will be wasted. These future unwanted situations can be avoided if humans send an available robot to those robots of concern, and update their instructions. In some cases, it might be worthwhile to track dierent mission parameters, e.g., some robot's status related information like failure, or rescue operations. The specic situations that humans prioritize will be dependent on the mission details, objectives, and updates. One of the example mission scenarios we constructed, which can be representative of an actual mission, is shown in Figure 4.3. There is a few kilometer square sub-urban environment with 12 Areas-of-Interest, 52 Figure 4.3: An example representative mission scenario with some reported information and a team of eight robots, I-VIII. The goal of the mission is to explore all these AoIs and nd as many survivors as possible. The robots also need to assist each survivor based on the person's needs. There are some assumptions for this operation. Firstly, if there is one survivor found in a region, there is higher probability to nd more survivors. Secondly, one robot may not be sucient to provide the necessary aid to a survivor, and two robots will be needed in such cases. The instructions given to the robots in the eld are in accordance with these assumptions and mission goals. A robot can get another robot from the same or nearby region to receive more assistance if needed. Also, robots need not explore a region if it is already explored and reported by another robot. Under these assumptions, we can specify two alert triggering conditions which are relevant in this mission: (1) high probability for any survivor found not getting enough assistance, (2) redundant exploration by a robot. In this mission, robots are sent out with some initial task plans, and a couple of hours have passed since the start. In Figure 4.3 we can see that at current time, 120 minutes, robot VI has returned to the communication station after nishing exploration of AoIs 8 and 9, and is ready to receive new instructions. All other robots are still on the eld performing their tasks based on their instructions and information states. Each robot has a last update time, and status from that time. We can also see that one survivor has been found in regions 2 and 12 separately. Since one survivor means more survivors will likely be found in a region, and most survivors need two robots, human commanders would want at least two robots in AoI 2 53 and 12. Also, humans would not want any robot in regions 8 and 9 since they are already explored. Thus, both the noted alert conditions can be tested from the number of operating robots in specic regions. The more precise alert conditions for this scenario can be as follows: (1) high probability of AoIs 2; 12 (with identied survivors) having less than 2 operating robots, (2) possibility of any robot operating in AoIs 8; 9; both conditions for present time or recent future (time2 [120; 140] min). In our framework, these conditions are mathematically encoded using the framework provided in our previous work [10]. (a) Regions which should have least 2 operating robots in re- cent times, for the survivors (b) Regions which should not have any robots to avoid redun- dant exploration Figure 4.4: Plots generated from forward simulation of mission scenario in Figure 4.3, showing probability distribution of having 0; 1; 2 operating robot(s) in particular regions at specic times. The discrete times in the gures span from present time to 2030 minutes into the future, as required by the time window in alert conditions. The red markings denote concerning situations where the number of operating robots are not favorable. Our forward simulation uses all the reported information on initial states of the robots, and given in- structions, and simulates the mission. We have more interest in knowing about the number of operating robots in regions 2; 8; 9; 12 specically, due to survivors needs or to prevent redundant exploration. If we perform 100 runs of forward simulation, we generate data for 100 possible mission outcomes throughout the time duration. We aggregate the results for the mission variables of our interest, in this case the number of functioning robots in each region. We create a probability distribution for the number of robots operating in a particular region at a specic time. The number of robots in a region can be between 0 to 2. Thus, we generate the plots given in Figure 4.4, and the unfavorable situations are marked in red. We can see that there is high (about 0:8) probability that region 2 does not have enough robots, which triggers the rst alert. In order to handle this situation, an available robot can be tasked to that area and relay a retasking message. Secondly, there is a probability of 0:20 0:40 of a robot operating in AoI 8, which triggers an alert for redundant exploration. To re ect to this second alert, humans need to check another mission variable, 54 each robot's operating area ID. This is to see which robot goes to AoI 8, so that the human supervisor can update its instructions by sending the available robot to it. 4.6 User study We conducted a preliminary human study to evaluate the merits of forward simulation-based alert generation. We identied mission scenarios that could be of interest in the context of human-supervised robot teams, and built a user interface for human commanders to view information for these missions. The participants in our user study were provided sucient introduction and training, and then were asked to assume the role of commander, use the interface, assess the mission situation, and strategize on new task assignment for one or two available robots. Each participant served for four missions and in two of these we oered alert generation with forward simulation-based prediction information. The purpose of our human study is to quantify if, and how eectively, forward simulations might a) help the humans to understand the mission situations better and b) guide them to make more informed decisions to facilitate performance improvement. 4.6.1 Hypothesis on Performance Improvement with Alerts We hypothesize that we can improve performance of human commanders by providing prediction-based alerts and information. We believe that there will always be a small percentage of people who are intellectually sharp enough to make reasonable decisions using only reported data, without alerts or forward simulation. But the remaining larger pool of people will likely be overwhelmed with the interrelated mission information and stress that they will fail to infer the data and make the best decisions. We anticipate that mission prediction information can signicantly improve the performance for a large fraction of this population; while there will still be a small fraction of people who will be overloaded, emotionally and cognitively, and hence unable to make good decisions even with the additional help. Based on this assumption, we propose HypothesisH1 with regards to our human study, and measure statistical signicance from our data to be collected. 55 HypothesisH1: Of the population that are unable to make any good decision without alert messages, more than half of the participants will start making all correct decisions when provided assistance in the form of alerts and forward simulation-based mission predictions. 4.6.2 Preliminaries We constructed four mission scenarios (an example in Figure 4.3), where each scenario starts at a particular time instance of a unique mission comprised of a team of eight to ten simulated robots. Each mission has progressed considerably, i.e., several hours have passed since the mission started with some nominal task plan being executed. One or more robots have returned with new mission information and are available for new task assignment. At this point, the participant is asked to assume the role of the commander. The participant can now access certain information using our interface, and decide on issuing commands to the available robots to facilitate mission progress. The information available to the human commander includes the instructions previously given to the currently out-of-communications-range robots in the eld, the latest robot state updates, and other mission event updates with their corresponding timestamps. The usefulness of the updates varies because it may have been reported for an event that took place a few minutes ago to more than an hour in the past. Each of the scenarios are constructed such that it has one or two unwanted situations that may occur in the immediate future. These are not reported events because they have not been observed by the robots; rather, they are based on predictions from forward simulations. If a commander prioritizes certain tasks and assigns new instructions to the available robots, these undesirable situations can be prevented or alleviated. We refer to this new instruction set as the ideal tasking component, and this intelligent process as a good decision, for each scenario. This ideal tasking is ultimately compared with the tasking done by the participants in our human study. We used two congurations of our interface in this study: (A) baseline in Round 1 and (B) with smart features in Round 2. Both versions provided the same reported information from past to present. However, version B oered one additional tab in the interface that showed alert messages generated by forward simulations based on a set of initial triggering conditions. With this version, the participants are also 56 provided with all of the robots' probabilistically-computed locations and navigation information through out the forward simulation time period during an expected run where the alert triggering situation occurs. 4.6.3 Study Description We conducted our study with engineering graduate students who are over 18 years old, and do not have color vision deciencies. As each participant came in, and was given introduction to the mission and the system, we randomly assigned the person two scenarios with version A of the interface for Round 1. If the person makes any good decision in this round, they were considered reasonably capable and excused from the study. This was done in accordance with our hypothesis. Alternatively, the participants making zero good decisions moved to Round 2, where they used version B of the interface for two of the remaining scenarios. In this second round, we identied the participants who could make all good decisions, measured the performance improvement using forward simulation-based information, and veried our hypothesis with statistical signicance. Each experiment for a single participant in the study took up to 120 minutes. During the rst 20 30 minutes, the participants were given a presentation about the class of missions in this study, the mission- specic goals and assumptions, along with an introduction to the interface. They were also given generic guidelines on specic mission aspects to assess, and ways to handle relevant situations. The remaining 6090 minutes of the study consisted of the two rounds that controlled for the amount of available information by using a specic conguration of our custom interface shown on large computer monitors. At the beginning of each mission, each participant was briefed on the initial strategy and task plans that were initially executed, and then given ten minutes time to process the information using the interface, and decide on a strategy to task the available robots. After ten minutes, the participants explained their desired task allocation verbally, we recorded their responses and compared their instructions to the ideal tasking for that scenario. If their plan included the key components from the ideal tasking, it is labeled as good decision, otherwise it is labeled a failure to make the high quality decision. 57 4.6.4 Data Collection In each round, there were a total of two or three good decisions to make, depending upon the two randomly- assigned scenarios that the participant was evaluated in. Only three out of the 13 participants made at least one good decision in the rst round and were removed from the data set to be used in hypothesis testing. The re 10 participants, failed to make a single good decision in the rst round with only the reported information and no alerts, so they continued on to the second round. We observed that 9 people, i.e. all except one, made all correct decisions in the second round. We terminated conducting our user study at this point since we had obtained enough data for claiming our hypothesis with signicant statistical condence. 4.7 Results We use the data collected from the user study to test our HypothesisH1. Table 4.1 presents the perfor- mance of the 10 participants during Round 2, where they had access to forward simulation based alerts and information from predictions. For each user in this round, there were 2 or 3 decisions to be made, and we note the number of decisions the person made correctly. Thus, we calculate the percentage of good decisions each participant made which is also given in the table. Table 4.1: Performance of participants in Round 2, who were provided with forward simulation-based alerts and mission predictions. User # of Decisions # of Good % Correct to be made Decisions Decisions 1 3 3 100% 2 2 2 100% 3 2 2 100% 4 2 2 100% 5 2 1 50% 6 3 3 100% 7 3 3 100% 8 2 2 100% 9 3 3 100% 10 2 2 100% 58 4.7.1 Findings from User Study We use one-sample hypothesis testing to test our HypothesisH1 given in Section 4.6.1. If a participant can make all of the good decisions in the second round, it is considered a success, otherwise it is labeled a failure. We model the outcome for each participant in the second round as a Bernoulli random variable, X, that can take two values, 1 if there is success, and 0 otherwise, with probabilities P and 1P respectively. We construct hypothesis tests for the Bernoulli parameter P where the null hypothesis H 0 is that P has some valueP 0 . The alternate hypothesisH a is that the true value ofP less or greater thanP 0 . We have a sample size of N = 10 corresponding to the number of participants in Round 2, who could not make a single good decision in Round 1. According to our hypothesisH1, we perform one-sided hypothesis testing for P >P 0 , where P 0 = 0:5. We compute the number of successes, Y = P N i=1 X i , where X i denotes success or failure of the ith participant. By denition Y has a Binomial distribution with parameters N and P , dened as P Y (K) = N K P K (1P ) NK . For signicance level 2 (0; 1), let b n;p () denote the quantile of order for Binomial distribution with parameters n;p. Since the Binomial distribution is discrete, only certain (exact) quantiles are possible. Our hypothesis testing is then, reject H 0 :PP 0 versus H a :P >P 0 i Y b N;P0 (1). We consider the signicance level = 0:05 corresponding to 95% condence. We calculateb 10;0:5 (0:95) = 9. Table 4.1 shows that the total number of successes in our ten trials is 9. Therefore, we can reject the null hypothesis and claim our HypothesisH1 to be true. In fact, we can say with 95% condence that the percent of people making all good decisions with alerts, who could do zero without, is actually greater than 60%. 4.7.2 Discussion The three participants who were dismissed from Round 1 could make exactly one correct decision, and thus scored between between 33:33 50:00% correct decisions. The remaining 10 participants, comprising of 76:92% of the total population, failed to make even a single good decision without predictive assistance features. This poor performance in the rst round indicates the inherent diculty for humans in assessing a mission without any aid from forward simulation based predictions. We aggregated participants' answers 59 in the questionnaires which asked about the challenges that they faced in each scenario. According to the participants, the main challenge was that they felt overloaded with information. Therefore, it was not possible for them to deduce how the mission will progress by assessing so many dierent variables. It is to be noted that the scenarios were designed in such a way that a reasonable human would be inclined to do some dierent tasking if they can not foresee the mission progression. This was done in an eort to minimize the possibility of participants arriving at the ideal tasking randomly. In Round 1, after the participants made their decisions, we verbally provided the alert message and the robot navigation information, and asked for their revised answer. We found that 88:97% of the revised decisions given by the participants during this conversation included the ideal tasking. This high percentage can verify further that our chosen good decisions are indeed the superior choices, according to most people, when provided enough information. In the second round, nine out of ten participants could make every good decision as they were given alert messages and predicted location data of the robots for the corresponding situation in version B of the interface. Collectively, the participants successfully made 95:83% of the good decisions, which is a substantial increase compared to Round 1 without alerts. This performance is also better than the revised decisions from the after-study conversation in Round 1. This indicates the usefulness of presenting forward simulation information up front in a structured way. Lastly, the participants were asked to provide free responses which revealed that the participants found the forward simulation-based prediction information very useful. 4.8 Summary We have shown that alerts can be extracted from forward simulations of the mission. We have demonstrated speeding up these simulations by more than three orders of magnitude, as compared to using physics-based simulations. We have veried that forward simulations based prediction information helps humans to make better decisions. 60 Chapter 5 Generating Task Reallocation Suggestions to Handle Contingencies in Human-Supervised Multi-Robot Missions This work is based on the journal paper [9]. 5.1 Introduction In our previous works [5, 10], risk-assessment-based alert generation framework was presented that proac- tively identies the possibility of potential contingencies or negatively-impactful mission situations. Examples of contingencies can be disabled robots, a task requirement changing, identication of risky tasks that need to be avoided, a new task appearing, etc. [6] demonstrates that these alerts improve human decision making by providing a better perception of the mission and creating opportunities for humans to retask agents to handle the contingencies. If an automated system could directly provide suggestions on this task reallocation based on mission objectives, it could facilitate further improvement in human decisions regarding retasking robots. Humans will still have the ultimate authority to accept, reject or modify the suggestions according to individual preferences and risk tolerance levels before commanding the robots. The unique characteristics of our task reallocation suggestion generation problem set it apart from the multi-robot task allocation (MRTA) and contingency handling problems, typically solved in the literature [146, 190, 191]. First, our system aims to auto-generate task reallocation suggestion, and handle some given contingencies. Hence, it might not be best to simply allocate all incomplete tasks to the entire team of robots from scratch. Instead, the task reallocation suggestion generation system needs to consider previous task 61 plans and the eect of contingencies on the task performance of the robots. Some robots would be available to receive task plans directly from humans and become informed, while other robots will be uninformed and outside the communication range. Therefore, we introduce an optional task, called relay, which refers to an informed robot being sent to an uninformed robot to convey updated task plan information. Secondly, the task and robot lists, their properties, and constraints must be updated based on the provided contingencies. For example, suppose an uninformed robot is scheduled to navigate to a danger zone in the future because of its previous task schedule. In that case, a new relay task should be created with a deadline so that it becomes reassigned before falling into danger. Moreover, There might be some critical tasks that need to be prioritized during the task reallocation process, and be scheduled to be performed as soon as possible. Another unique challenge in the retasking suggestion generation system is to consider incorporating another optional task, robot rescue, that has probabilistic outcomes dynamically aecting the total number of functional robots operating within the mission [7, 8]. The complexity of the operational environment, as well as the occurrence of unpredictable events, can cause a robot to fail or get immobilized temporarily, but it may be able to recover with the help of another robot. We assume that expert human supervisors will be able to examine the received data (image, video, etc.) regarding each disabled robot, estimate the risk level of rescuing it, and assign a probability of each rescue outcome, i.e., successful rescue, failed rescue, or rescuer robot getting disabled. For each disabled robot, whether to rescue, when to rescue, which functional robot should attempt the rescue- all three decisions need to be made. Moreover, if there is more than one disabled robot, careful consideration should be made for the potential benets and risks of multiple rescues separately and together. Whenever any rescue operation is considered, as the probabilistic outcome aects the team size of robots, mission performance can get severely impacted. We need to consider combinatorially all dierent ways the mission may progress considering the possible rescue outcomes, and generate a dierent task assignment for each mission thread. Rescue scheduling should be done in such a way that the expected mission performance is maximized. We aim to auto-generate a complete task re-assignment suggestion to the human supervisor, which addresses corrective actions based on the contingency being reported or predicted. We want to complete the mission and minimize the expected mission completion time. Researchers have been using expected mission 62 makespan as the metric to minimize in various multi-robot task allocation [190] and contingency handling problems [191]. But our problem requires the incorporation of task criticality, which the makespan would not re ect. Moreover, the possibility of robotic failures during the mission and task deadlines and pre-requisites can create situations when all mission tasks can not be completed. Makespan also does not re ect which tasks remain incomplete and how many. To incorporate task criticality and penalize cases with incomplete mission tasks, we use a variant of makespan, which we dene in Section 5.3.3. The contribution of this paper lies in providing methodologies for generating task reallocation suggestions and a preliminary evaluation of our approach. The standard MRTA approaches can not be directly applied to solve our task reallocation suggestion generation problem due to its complexities. After careful consideration, we choose a heuristics-based approach along with additional process blocks for incorporating the optional tasks, relay, and rescue. The approach includes the evaluation of combinatorially all mission realization threads to account for rescue outcomes when applicable. 5.2 Related Works The problem being considered in this paper is related to multi-robot task allocation problems [22, 149, 150, 173, 190, 226]. [63] presents a domain-independent taxonomy of MRTA problems, which was expanded by [146] for temporal and ordering constraints (MRTA/TOC). Our problem involves task assignment to available robots, which lies in single-task robots, multi-robot tasks, time-extended assignment category with precedence constraints, in addition to some time window constraints in limited scope. In our problem, we choose a centralized approach since we generate task assignments as suggestions to human supervisors rather than actually assigning them to the robots. There exist a wide variety of methods to solve MRTA problems. Classical approaches rely on mixed- integer linear programming (MILP) [55, 158]. Auction-based approaches are promising, particularly for distributed MRTA problems [101,133], including certain communication-limited environments [150]. Genetic algorithms are being used in various scheduling problems [13, 137] and have the potential to be used in solving MRTA problems that require further research. Some of the recent works include chance-constrained simultaneous task assignment and path planning [226], graph-search approach for optimal task allocation 63 with global specication [22], and learning task scheduling policy for single-robot tasks and single-task robots [218, 228]. Probabilistic approaches for solving MRTA problems include ant-based algorithms [30] and simulation-based learning techniques [31,37,81]. Our problem requires forward-looking and real-time centralized decision-making under uncertainty; there- fore these approaches are not suitable. Furthermore, besides solving an MRTA problem, our system requires to begin by assessing how the newly-found contingencies aect the task constraints and the robot team's anticipated performance considering the previous task plan. In addition, we have complexities arising from optional relay tasks and rescue options with probabilistic outcomes causing dynamically increase/decrease in team size, which prohibits the direct application of the variety of approaches from the MRTA literature. There are several works which consider \task reallocation", some level of uncertainty, and repairing prior plans [31, 37, 81]. However, these approaches, as well as exact approaches in optimization (e.g., MILP), are computationally infeasible due to the inherent complexity arising from communication constraints and relay tasks, probabilistic outcomes of robot rescue tasks, and precedence constraints in our targeted domains. Existing literature on scheduling policy or parameter learning-based approaches focuses on simpler classes of MRTA problems. Another approach to performing multi-robot task allocation is based on heuristics guidance. Our work takes inspiration from [189{191]. This series of works focus on developing a centralized heuristics-based method in ST-MR-TW:SP problems, including contingency management methodologies. As multi-robot scheduling problems are often similar to multi-processor scheduling problems, processor scheduling techniques are frequently used [42, 230]. Dierent heuristic-based algorithms from processor scheduling applications assign tasks to robot coalitions greedily [88, 209]. We choose to incorporate a heuristics-based approach as one of the process blocks in our task reallocation suggestions, along with other processes to handle additional complexities. This paper deals with robot failures and rescue decision-making. This topic is closely related to contin- gency management. Researchers have developed methods for contingency management for failure recovery to promote the deployment of multi-robot systems in real-world applications to handle unexpected and uncer- tain contingencies [51,58,131,192]. [132] employed contextual multi-arm bandit algorithms to train robots how 64 to recover from failures by choosing the best assistants while taking into consideration uncertain/dynamic operating conditions and assistant skills. [227] used advances in prognostics and health management systems to create a proactive and automated contingency management system that takes prognosis information into account for failure recovery as well as mission planning. The majority of previous research has focused on developing specic procedures for various failure scenarios, applying them to robotic systems when needed, and measuring the success rate. [189, 191] worked on centralized ways of dealing with contingency duties while accounting for their uncertainty. However, they did not include communication constraints during task allocation, agent failures, and the possibility of changing team size from contingency tasks. Therefore, we use our own modied heuristics, and additional algorithms to schedule relay and rescue schedules ap- propriately. In this work, we proactively generate separate task assignments for combinatorially all possible mission progressions. Alert generation and task reallocation suggestions generation are essential for human-robot teaming, which is a vast research topic [45, 106, 122]. The problem being studied in this paper is related to human factor concerns associated with supervisory control of multi-robot systems [38,186]. There are several works that propose reactive alert generation for robotic applications with the human in the loop settings [123,136]. Examples of proactive alerts are found in [10,175] which refer to issuing alerts for the task- or mission- level contingencies. All these alerts assist in drawing human focus into critical aspects in the robotic operations and motivate to take corrective actions. Automated adjustment of autonomy level in robotic operations with human supervision is often needed for optimal performance, and [48] studies this sliding autonomy concept for multi-robot systems. [21] highlights generating automated suggestions for optimal autonomy level for optimal performance. We develop a new assistive tool for humans, a system for automated generation of task reallocation suggestions to enhance human decision-making. 65 Table 5.1: Nomenclature used in this chapter Symbol Description T Set of all mission tasks Tc Set of all critical tasks G Task precedence graph m Total number of mission tasks n Total number of robots (functional/disabled) Ra Available robots for direct commanding R f Functional on-eld, but out-of-communication, robots R d Disabled robots Rr Relay-eligible robots or informed robots n min Minimum number of allowable robots for task n max Maximum number of allowable robots for task h All constraints related to task p r 1 Probability of successful rescue attempted on disabled robot r p r 2 Probability of failed rescue attempted on disabled robot r p r 3 Probability of the rescuer robot getting disabled while rescuing robot r t Time duration taken by the informed robots to collect rescue outcome information after completion of a rescue task w Weighting factor for critical tasks M A single task plan with each robot's task plan which includes it's activity throughout the time horizon for a single mission realization T M left Mission tasks (2T) that can not be completed according to planM S M Our objective function to minimize: Modied makespan computed for task execution planM (Eq. 5.1) M Our solution set of task plans, Set of allMj s where j refers to the index of each mission progression thread. k M Total number of dierent mission progression threads based on the task execution plan setM j Probability of mission thread-j happening 5.3 Problem Formulation 5.3.1 Mission Model We assume there arem mission tasks that need to be completed by a team of n homogeneous robots, where they need to navigate around in a large environment. The mission starts with an initial task plan given to the robots. Under the restricted communication assumption, the robots can only exchange information within a certain distance. Similarly, humans receive mission updates as and when a specic robot carries the information and navigates near one of the communication stations situated within the environment. 66 Whenever humans receive new mission updates and have one or more robots within the communication range, an automated alert generation system [10] identies potential contingencies occurring in the mission, either reported or predicted from the mission updates. Based on how a mission has been progressing, the previous strategy or task allocation might cause degradation in mission performance. Therefore, human supervisors might update their plans whenever there are newly-found potential contingencies, e.g., changes in agent availability or failures and changes in the mission task list or task requirements. They might re-assign tasks to robots in order to avoid performance degradation due to the contingencies and the shortcomings of the initial plan. We aim to generate suggestions for this re-tasking schedule to handle the contingencies and optimize the mission objective. The nomenclature list of most symbols used in this paper is provided in Table 5.1. Let, the set of all mission tasks are T, and T c 2 T be provided as the set of critical tasks. We dene each task, taking inspiration from [191]. Each task has a location, the minimum and the maximum number of robots, and task duration (varying with robot group size). A limited number of tasks can have a task deadline (details in Section 5.5.1). The mission tasks have inter-dependency, and a task can have specic tasks as pre-requisites, i.e., the task can start only after the pre-requisite tasks get completed. A directed acyclic graph,G, is provided as the task precedence graph, which includes all tasks 2 T as nodes and directional edges for denoting inter-dependence of mission tasks [191]. In graphG, all predecessor nodes of a node represent the pre-requisite tasks of the task . We represent all the requirements or constraints related to task as h . We also assume that the nominal travel distance between any two locations (task locations or robot locations) is known. 5.3.2 Contingencies and Updates to Robot Task Assignments Suppose the robots were sent to the mission with task planM 0 at the beginning of the mission. We assume that any task planM describes each robot's task schedule, which includes its activity throughout the time horizon, i.e., when a robot is performing a task, navigating from one task to the other, or sitting idle. Now, some time has passed, humans have received some information (current or past) regarding mission progression, and the alert generation system has identied a few potential contingencies that have occurred 67 or are about to happen in the future. At this point, humans have R a set of robots at their disposal for direct communication, while a group of robots, R f , are functional, working on the eld based on previous planM 0 , and are out of communication range. All robots in R a and R f are functionally identical, i.e., homogeneous, with only a dierence in whether they have direct communication with humans at that particular time. In addition, there might be news of some disabled robots, R d , who became immobile during mission performance and are currently unable to perform tasks. Thus, task scheduling is performed with an initialjR a [ R f j number of robots, but it can increase or decrease with mission progression. In this work, we assume the alerts for predicted or known contingencies fall into the following six cate- gories, 1. certain tasks identied as risky (need to be abandoned) 2. certain tasks becoming irrelevant 3. changes in task specication/constraint 4. new tasks and/or new task-dependencies arising 5. certain tasks being identied as critical tasks 6. one or more disabled robots on the eld. Depending upon the newly available mission updates and contingencies, the task performance of the robots on the eld (following planM 0 ) might be negatively aected. Therefore, humans need to assess the situation and make decisions on task assignments considering all robots and the remaining mission tasks. Apart from the mission tasks in T, we assume humans can create and assign relay and rescue tasks as needed, according to the mission objective while generating a new task schedule. Relay tasks are meant to propagate task re-allocation and other information to robots who are uninformed and out of direct communication range from humans. If humans want to change the individual task schedule for a robot in R f , an informed robot needs to be sent to it in time. The location of a relay task might be dynamically changing with time, based on the expected location of the robot to be relayed at a particular time. A rescue task is where a functional robot attempts to revive a disabled robot. Each rescue has three probabilistic outcomes, (1) successful rescue which revives the disable robot to functional state, (2) failed 68 rescue where the disabled robot does not recover, and (3) catastrophic outcome where the rescuer robot also becomes disabled [7, 8]. Figure 5.1 illustrates this concept. For each disabled robot r 2 R d , the probabilities of the three outcomes from rescue attempts are p r 1 ;p r 2 ;p r 3 respectively, where p r i 2 [0; 1]; 8i2 f1; 2; 3g; such that P 3 i=1 p r i = 1. We assume human experts can assign these p r i values by inspecting the received data (image, video etc.) regarding each disabled robot, and estimate risk level of rescuing it. Figure 5.1: Conceptual gure showing how a single rescue attempt at time tr might impact the eective team size and hence the mission progression, depending upon each rescue outcome. For simplicity, we assume that the outcome of a rescue attempt will propagate to all informed robots with a xed time delay t, the informed robots (also the rescuer and rescued robot) would pause executing assigned tasks during that time period. Informed robots at that time refer to the robots available (R a ) at the stage of generating a task schedule with the rescue task, and the robots (2 R f ) which have encountered one of the informed robots by that time. The justication of this assumption is that we are considering information propagation only to the robots aware of the rescue attempt. It is reasonable to assume they will 69 look out for this rescue outcome information by spending that t time. Therefore, these robots can update the task plan based on each rescue outcome. 5.3.3 Problem Statement Let span(M; T 0 ) be the makespan (time) of completing all tasks in T 0 according to mission planM. If all tasks in T can not be completed with scheduleM, T M left T is the set of incomplete tasks. The modied makespanS is dened as follows. S M = (span(M; T) +wspan(M; T c )) (1 +jT M left j) (5.1) Here, the penalty for incomplete mission tasks is added by using the multiplying factor (1 +jT M left j). It incorporates task criticality in makespan with a weighting factor w> 0. Figure 5.2: A solution can include any number of mission progression threads, depending upon the number of possible outcomes from each rescue (in our case, three) and the number of rescues attempted. If the initial thread (s0) hadx number of functional robots, after the rst rescue, threads with s1;s2;s3 will havex + 1;x;x 1 robots respectively, and therefore these threads will have dierent task plans from the time of rescue outcome. Similarly, the mission threads with s33;s22;s11 will have x + 2;x;x 2 functional robots, respectively, after the second rescue. Since the mission might contain up tojR d j number of rescues with probabilistic outcomes, we need to generate dierent task schedules for the possible outcome combinations. Assume, in a mission with initially 70 two disabled robots, one rescue is performed during the mission. Each outcome of the rescue operation will require a dierent task plan, adjusting the number of functional robots. Also, each of these threads might or might not include a second rescue. Fig. 5.2 illustrates the concept with nine mission realization threads, where all threads have two rescue attempts. Each thread has a unique trend of the number of functional robots with respect to time, therefore each thread requires a separate task plan from the time of the diering rescue outcome. Each thread also has an associated probability. For example, if a thread-j (s 13 in Fig. 5.2) includes rescue attempts on two robots r 1 ;r 2 with the rst rescue being successful and the second rescue failing, the probability of that thread would be j = p r1 1 p r2 2 . This thread will have a task execution plan M j provided by humans. Not all mission threads need to have the same number of rescues. After execution of the rst robot rescue, depending upon the outcome, each rescue thread may or may not have a second rescue. Let's assume that a second rescue is not scheduled if the rst rescue fails. Then we would only have a thread with s 3 , threads with s 31 ;s 32 ;s 33 will not exist, and the total number of mission threads would be 7. We aim to generate an overall task plan setM, which contains one plan (if no rescue is scheduled) or a collection of of plans, (M 1 ;M 2 ;:::;M k M). Here, k M refers to the total number of mission progression realizations with dierent task plan schedules, originating from the probabilistic outcomes of scheduled rescues. Each realization represents a unique rescue outcome combination, and an associated estimated probability j computed from p r i values for relevant rescues, such that P k M j=1 j = 1. We would like to generateM which minimizes the expected value of mission span parameterS as shown in Equations 5.2- 5.3, while not violating task-agent-time constraints (Eq. 5.4). GenerateM = arg min M E[S]; (5.2) where,E[S] = k M X j=1 j S Mj (5.3) 71 such that each task is scheduled once its prerequisite tasks arecompleted, the number of scheduled robots for a task is within the task-specic range, and each task deadline ismaintained8M j 2M (5.4) 5.3.4 Overview of Approach The Task reallocation Suggestion Generation System has three main components. As a rst step, the contin- gencies highlighted by the alert system are used to update the task precedence network, along with the task requirements and constraints. Based on the new task network, the system estimates the task schedule of the robots on the eld who are operating based on their previous task assignments. Some previously assigned tasks might not be fullled anymore due to the changes in mission requirements based on the most recent information, which the robots on the eld might not be aware of. Moreover, the disabled robot's assigned task will be completed, and the dependent tasks will also be aected. This process is described in Section 5.4. Next, we take a heuristic-based approach to assign tasks while considering the previous task allocation provided to the robots on the eld and the robots available for direct tasking by the human supervisor. Our approach is depicted in Section 5.5 which considers available robots, R a , and functional robots on the eld, R f . The process generates a single task execution plan as it does not consider probabilistic outcomes of any task, i.e., possible rescue operations on disabled robots, R d . Each rescue operation has its merit in the likelihood of increasing the number of agents in the elds, while there is a risk of losing one. How to make robot rescue decisions and incorporate that in generating a task plan in our heuristics-based approach, is described in Section 5.6. When one or multiple rescue tasks are scheduled, the automated system computes task assignments for all dierent ways the mission might pan out for each possible rescue outcome combination. 72 Figure 5.3: (a) Updating the task network and task specications based on new information from mission alerts. (b) Original task schedule for robots on the eld, and the estimated task performance based on new information and robot update. R3 robot has been disabled, and R1 and R2 are aected by the change in task precedence graph and task requirements, and they enter idle states not being able to perform tasks as initially planned. 5.4 Task Network Generation and Estimation of Task Performance under Contingencies First, a new task network is generated based on the changes that occurred. Some mission might have been completed by the current time, and naturally those task nodes and associated edges must be removed from the task precedence graphG. Next, the alert triggering items are addressed. For alert categories 1-2, we remove that task from T andG. If a task in alert category-1 is in the previous task assignment of any robots in R f , we insert arelay task to that robot and set the deadline of the relay to be the time when the robot was originally expected to enter that task. This way, the robot will not enter the abandoned task. For categories 3-5, T (includingh ;82 T) andG are appropriately updated, e.g., a task to be abandoned will be removed from the task network, a task newly appearing will be created and added to the task network appropriately etc. As for category-6, rescue decisions for the disabled robots are dynamically made during the task re-assignment generation process (see Section 5.6). As a small-sized example problem, let's assume a search and rescue mission where there are 20 remaining tasks, 1 20, which are required to be completed by the robots from now and onwards. There are 3 robots (R1;R2;R3) on the eld, where functional robot-set, R f =fR1;R2g, and disable robot-set, R d =fR3g 73 (alert category-6). The blue-colored nodes and edges in Fig. 5.3 (a) form the previous task precedence network. Apart from disabled robots, there are two more alerts (categories 3-4), (i) task 3 now requires two robots that previously needed one, (ii) task 4 has a new pre-requisite task, 3. Therefore, h 3 is updated, and a new edge (green) from 3 to 4 has been added to the task precedence graph. These alert situations also aect the task performance of the robots on the eld who are operating according to old instructions. The two graphs in Fig. 5.3 (b) indicate previous nominal task execution and the estimated task execution based on mission updates. Depending upon the previous assignment, the alert situations can make the assigned robots for the aected tasks (or their successor tasks) unable to perform, and enter idle state. On the other hand, the disabled robot R3 stays where it failed and is unable to perform any assigned task. It means the robots are not being properly utilized due to the contingencies occurring. Our heuristics-based task scheduling algorithm uses the estimated task execution of those functional robots and intelligently propagates new instructions to them to improve mission performance. On the other hand, we also make robot rescue decisions for disabled robots based on mission needs. 5.5 Heuristics-Based Task Scheduling 5.5.1 Preliminaries We take a heuristics-based task scheduling approach to solve our problem. [191] highlights a performance comparison between ve dierent heuristics-based task scheduling strategies inspired from the processor scheduling literature [209], and two other strategies without task prioritization phase that performed worse. From the performance analysis provided in [191] in terms of minimizing nominal mission span time, for task networks similar to what we are envisioning in our works, we choose the strategy for relatively superior performance. It works based on the HLFET scheduling algorithm (Highest Levels First with Estimated Times), which uses level-based heuristics. The notion of level is the sum of costs of all the nodes along the longest path from the node to an exit node which can be applied to the task precedence graph. [191] considered task performance time duration (using minimum robot) and transition/travel time between one task to another in order to compute the sum of costs from a node (task) to one of the exit nodes (tasks). 74 HLFET heuristics can perform well in minimizing makespan as it prioritizes a task that has the longest length chain (in terms of time duration) towards an exit node/task in the task precedence graph. Graph search algorithms can be used in computing the heuristic values. We take inspiration from this approach and modify the strategy to handle the higher complexity and variability in our task scheduling problem. Our problem has several dierences from the problem in [191] which makes it more challenging. The main challenge arises from communication constraints and disabled robots. However, any task with a probabilistic outcome, like the rescue task on a disabled robot, is not considered in this section; it is addressed in Section 5.6. Limited communication in our problem, some robots being out of the communication range, might require dynamic creation of relay tasks (not in mission task-set T) and assigning them to robots to complete the mission tasks. Even though our work and [191] both aim to minimize mission span time on a high level, our problem has incorporated critical tasks, considers the inability to complete the mission, and uses a weighted makespan metric to optimize. We are also required to handle any acyclic directional graph as the task precedence graph, without any symmetry requirements in terms of the number of tasks or robot requirements between dierent task precedence stages. There might not be enough robots to complete tasks in a precedence stage at once. In fact, some tasks might not be ever completed due to a lack of resources/agents and will be left out of the task schedule. [191] considered some symmetry requirements in the task network while benchmarking against an exhaustive search and always considered cases with more robots than minimum task requirements at every task precedence stage. They only considered complete homogeneity and were unable to address task deadlines, whereas we are considering a limited form of robot eligibility to do specic tasks. We also include task deadlines for a limited number (<jR a j +jR f jjR d j) of one-robot tasks for which all robots are equally eligible. 5.5.2 Approach In order to address task criticality and task deadlines, we re-dene the \level" of HLFET scheduling algorithm as a multiplied version of the original sum-of-cost, the multiplying factor being x level specic to each task. For all critical tasks, initially x level = (1 +w), where w comes from our modied makespan in Equation 5.1. Others tasks use x level = 1. However, for tasks with a deadline, x level is dynamically 75 Algorithm 1 TaskSchedulingWithTaskPriority(G; R a ; R f ) 1: (M;Rr;T left ) (;;Ra;;) 2: Update x level for tasks with deadlines 3: whilejG.nodesj6= 0 orG.nodes6=T left do 4: T 0 set of available tasksfnodes without predecessors inGg 5: T 0 T 0 T left 6: highest priority task in T 0 7: C all possible coalitions from Ra[R f for task 8: (t ;c ;R r ;R ; t i ; t t ) (1;;;;;;;1;1) 9: for all c2Cdo 10: if (c;) does not violate task-resource constraints then 11: ifjcj =n min orjRa[R f jjcj minimum robots required for remaining tasks in T 0 then 12: (R 0 ;R 0 r ;M 0 ) AddRelaySchedule (M;c;;Rr;Ra;R f ) 13: t 0 estimated time of being completed 14: (t i ; tt) total idle time and total travel time of robots forR 0 [ (c;) according toM 0 15: if t 0 <t or (t 0 =t andjR 0 r Rrj>jR r Rrj ) or (t 0 =t andjR 0 r Rrj =jR r Rrj and t i < t i ) or (t 0 =t andjR 0 r Rrj =jR r Rrj and t i = t i and tt < t t )then 16: (t ;c ;R r ;R ) (t 0 ;c;R 0 r ;R 0 ) 17: (t i ; t t ) (t i ; tt) 18: endif 19: endif 20: endif 21: endfor 22: if c =; then 23: T left T left [ 24: Continue 25: else 26: M M[R[ (c ;) 27: G:RemoveNode() 28: Update r:TaskPerformance based onM,8r2R f Rr 29: Rr R 0 r [ implicitly relayed robots from the new assignment 30: Update all robot locations and time pointer for availability 31: Update x level for tasks with deadlines 32: endif 33: endwhile 34: returnM,T left Algorithm 2 AddRelaySchedule (M;c;; R r ; R a ; R f ) 1: R 0 cRr 2: ifjR 0 j<jcj then 3: R ;fimplicit relay occurredg 4: else 5: R relay task schedules re ecting earliest possible relays to R 0 by robots in Rr 6: endif 7: Rr Rr[R 0 8: M 0 M[R[ (c;) 9: for all r2c do 10: t i idle time spent by r before execution according toM 0 11: if t i is sucient to add atleast one relay to a r 0 2R f Rr then 12: Add possible maximum relays toR 13: Add newly relayed robots to Rr 14: ifjRrj =jRa[R f jthen 15: break 16: endif 17: endif 18: endfor 19: returnR,Rr ,M 0 76 updated before every task assignment stage based onM; R r ; R a ; R f . Let, a task with a deadline t d could be completed latest by timet x by any robotr if it was assigned at this stage. The maximum possible travel time between any two tasks and maximum possible single task duration considering all tasks inG are t 1 ;t 2 respectively. If t x + 2t 1 +t 2 >= t d , the task's x level is set to a suciently large number that it will be scheduled in the next round with certainty, i.e., a number greater than other remaining tasks. This updating is done in Lines 2 and 31 in Algorithm 1. If multiple such tasks are found at once, they will be assigned one by one before other tasks, in a random order. We use task priority heuristics to schedule tasks one by one, and the entire process of generating the task schedule (M) is depicted in the pseudocode in Algorithm 1. After initializing, the while loop iterates until all tasks have been attempted to be scheduled (line 3 to the end). Inside the loop, one task is chosen and all possible robots or groups to be assigned are considered. The for-loop between lines 9-21 iterates through all robot or group options and selects the best option. The remaining portion (lines 22-32) updates everything and congures everything for the next round of task allocation. In this scheduling, we assume tasks are not divisible, which means a robot will nish an already assigned task rst and then becomes available for the next assigned task. The main while loop runs until all the assignable tasks inG are assigned and assigns one task at a time. At each step, which ever task 2 T 0 (tasks with prerequisites fullled) scores highest based on our heuristics get selected (line 6). We consider all possible robot coalitions of size betweenn min andn max that could be assigned to task based on task-robot constrains. We avoid assigning more robots than n min if there is a lack of agent abundance, considering T 0 . The function AddRelaySchedule (line 12) works as described in Algorithm 2 which considers implicit relay rst, and then required explicit relays, and then adds additional relays if any robot has sucient idle time from the particular assignment. Upon assessing all possible assignments for , the condition on line 15 dictates how one assignment is selected over the other. The one with the earliest completion time or the one with the highest number of relays performed if task completion time is the same across multiple options gets selected. If time and relays both are the same, the assignment with less idle time or the same idle time but less travel time is considered better. While focusing mainly on minimizing our weighted modied makespan, our method encourages information ow with relay tasks, reducing idle and travel time. Line 28 updates 77 estimated task performance by relay-ineligible robots based on the current and past task schedules. Certain previously assigned tasks can become available (prerequisites fullled) after a new task is performed. A robot might suddenly become capable of performing tasks from previous assignments without being relayed. Also, an implicit relay can occur if a newly assigned robot is sent to a previously assigned task location when a relay-ineligible robot is expected to be there (line 29). The Algorithm generates the task scheduleM, and a list of remaining tasks T left (; if the mission tasks are completed), which can be used to compute the modied makespan in Equation 5.1 to assess performance. 5.5.3 Performance Evaluation We do performance evaluation using some simulated problems to verify the usefulness of our approach and assess the quality of our solutions. In all the mission cases used for generating the results presented in Sections 5.5 and 5.6, we use randomly generated problems which meet certain criteria, such as the number of mission tasks, number of stages in task precedence graph etc. to illustrate the usefulness of certain features in our approach. Any such criteria is mentioned in the discussion of each table presenting results. Task specications such as individual task duration, travel time, task criticality, number of robots required for each task, are always randomly generated. In all cases tested in this paper, we use distributions such that the task duration times are larger than task transition times in expected sense. First, in order to observe the value of incorporating optional relay tasks to robots, we compare the performance using our approach with task reallocation without incorporating relay tasks. This baseline allocation is based on the same heuristics, inspired from the HLFET algorithm, but without the process block for incorporating optional relay tasks to robots. It considers allocating one task at a time to one of the informed robots. A robot that was initially out-of-communication (2 R f ) can become informed about new task allocation only after attempting or completing its assigned tasks and returning to an area within communications range of the human, or after coming across an already informed robot, i.e., getting relayed implicitly. We assume that information sharing between robots occurs only when two robots appear in the same task location at the same time, let it be for a scheduled relay task or due to prior independent task scheduling. Implicit relaying occurs when an uninformed robot communicates with an informed robot 78 regarding new task schedules. If an uninformed robot attempts an assigned task which it is unable to complete due to a new mission update it is unaware of, we assume that it stays in idle state for some time duration, and then it goes to the next task or returns to humans once all tasks have been attempted. Table 5.2: Performance improvement in terms of minimizing the modied makespan, using our heuristics-based approach with relay compared to the heuristics-based allocation without relay incorporation. Maximum and average % improvement are computed from 6 randomly generated problems for each of the ve (i-v) categories based on problem sizes. Category ID # Tasks (m) # Robots (n) Max % Improv Avg % Improv i 10 3 29.3 19.4 ii 15 4 30.2 20.4 iii 20 5 26.3 16.1 iv 25 6 21.9 14.3 v 30 5, 2 18.1 14.9 We have tested the performance on a total of 30 simulated problems, six problems for each of the ve categories in terms of dierent problem sizes. The maximum and average performance improvements for incorporating optional relays for each category are provided in Table 5.2. We consider R a 2f1; 2g in all cases. All of the cases show signicant improvement by incorporation of optional relay tasks as it utilizes the functional robots more eciently. However, the improvement is dicult to predict as the number of variables in the problem is very high. Previous task allocation of robots, task duration, task transition times, the task network topology - all eect the performance. In our tested problems, we enforce 80% of the robots in R f to have atleast one task in the previous task allocation which it is unable to do, in order to observe the value of intelligent incorporation of relay tasks. If previous task plans are less aected due to the lack of contingencies, the value of incorporating relay may decline accordingly. Next, we try to evaluate the performance of our heuristics-based approach with relay against the optimal performance. The optimal solution can be found from an exhaustive search over all mission tasks in additional to relays as needed, which is computationally infeasible for most practical problems. There can be an extremely large number of ways task assignments could be done, even for a small-sized problem. We chose 30 small problems with task networks with 3 task precedence stages, a total of 7 10 simple tasks (all 79 havingn min =n max 2f1; 2g, no eligibility or deadline criteria) including 0 2 critical tasks,jR a [ R f j = 4, jR a j2f1; 2g. Each stage has 2 4 mission tasks, and we consider all kinds of assignments for each stage, where each robot can be assigned 0 2 tasks. Then we consider dierent possible relay assignments for the required information propagation. In the exhaustive search for these 30 problems, we found that the number of task assignment options evaluated for nding the minimum modied makespan value was in the range of 1 20 million. Table 5.3: Performance benchmarking of our method in terms of computational time and optimality with exhaustive search. We chose to use 30 randomly generated problems of small size (having 7 10 simple tasks) in order to avoid computational infeasibility for exahaustive search. Computational Time Optimality Exhaustive Our method % Deviation Avg. Deviation Achieve from optimality from optimality Optimality 32 51 9 12 [0; 11:6]% 4:1% 8=30 minutes seconds problems We applied our heuristics-based method on the same problems. We observed the computational time, as well as the percentage deviation in modied makespan values from the optimal values. Our ndings are presented in Table 5.3. We used a standard desktop (Intel Xeon CPU Quad-Core E5-1620 3.6 GHz processor with NVIDIA Quadro 600 and 16GB RAM) for generating the data related to computational time. Exhaustive search took half an hour to an hour whereas our method only took several seconds. Due to the intractable computational expense for obtaining the optimal solution, we have not tested for larger problem sizes. Our method achieved optimality in 26:7% of our test cases and deviated from the optimal makespan by only 4:1%, on average, which suggests our approach oers competitive performance. Since many parameters are randomly generated from dierent distributions in the tested cases, we expect similar performance in problems where task duration is typically longer than task transition times. 80 5.6 Incorporating Robot Rescue Decisions We consider attempting rescue to the initial set of disabled robots, and each of the robots can be attempted to be rescued only once. Therefore, if a rescuer robot becomes disabled upon rescue attempt, it won't be considered to be rescued. There can be up tojR d j rescue tasks attempted, each having 3 possible outcomes. If we attempt n 0 jR d j number of rescues, there are 3 n 0 dierent ways the mission can pan out, and each mission progression has a unique task schedule computed by our method. Theoretically, our approach can consider any number of rescues; however, the size of the problem might limit the number for ensuring a feasible computation time. The number of all possible ways to incorporate 0jR d j number of rescues in the task schedule is extremely large, even for a relatively small problem. Each rescue decision includes which disabled robot to rescue rst, which functional robot will attempt the rescue when to schedule the rescue operation, and insert it between two tasks within the nominal task schedule without rescue. An exhaustive search would assess all options and choose the optimal one. If we have 20 tasks, 2 disabled robots, and 4 functional robots, an average of 1:5 robots deployed per task, there are more than 900 ways to accommodate 2 rescues. Therefore more than 8100(= 3 2 900) mission outcomes need to be evaluated. Moreover, about 60 ways of accommodating 1 rescue need to be considered along with the case with no rescue. We have devised an approach to assess single rescue insertions one by one in the time horizon to keep our search tractable. For each mission realization based on the combination of rescue outcomes, we compute task execution planM j , with a view to solve Equations 5.2-5.4. 5.6.1 Preliminaries Let the value of modied makespan (S) achieved by generated task planM 0 (Algorithm 1) without any rescue bes 0 . At any point of time, if a rescue on disabled robotr d is by a particular robot is inserted into the planM 0 , the three probabilistic outcomes would be (1) successful rescue, (2) failed rescue, (3) worst outcome with rescuer robot being disabled. Fig. 5.1 shows how it might aect the mission progression, as the number of functional robot might change. Typically, increased team size would increase mission progression rate while the decrease in team size would reduce the rate. Thus, it can decrease or increase mission completion 81 times (achievet 1 ;t 2 ort 3 , compared tot 0 without rescue), which is closely related to the modied makespan (S). Since rescue outcome-2 does not change the functional robot list, the task plan remains the same as M 0 , only with the added time delay for rescue attempt. However, outcome-1 and -3 require generating new task schedule for increased/decreased number of the functional robots, by using algorithm 1 from time t end r =t r +t rescue d . It is needed to use appropriate information propagation delay (t) associated with other informed robots. Whenever the expected value of modied makespan with rescue,E[S] = P 3 i=1 p r d i s i =s 0 then 8: return r d ;r r ;t ;s ;M 1 ;M 3 9: endif 10: Rrescuers R f [Ra 11: for all rr2Rrescuers do 12: (r d ;t;s;M 1 ;M 3 ) BestRescueScheduleGivenRescuer (rr;s 0 ;s 1 ;G;w;M;T;Tc;R d ) 13: if s 10. In such cases a hierarchical team formation-based approach is applicable where the large team will be split into smaller teams and one can apply our method for the small sub-groups of robots. Under this assumption, the time complexity of our algorithm with respect to number of tasks become O(mlog(m)). Algorithm 3 represents our method which considers inserting a single rescue schedule into a task plan. Every time, Algorithm 1 is called once in the beginning (line 4). Line 10 onward is called only when a rescue insertion decision has already been made. This means this portion can only be called up to 4 times as that is the maximum number of rescue scheduling possible for two disabled robots in all mission threads, as Fig. 5.2 shows. This portion of the algorithm tries to make decisions on rescue time, rescuer robot and the disabled robot to be rescued. It iterates through all rescuers (line 11 in Algorithm 3) in constant time and iterates considering rescue insertion between any two tasks in each rescuer's task plan (line 3 in Algorithm 4) in O(m) time. Inside the inner loop, the method calls Algorithm 1 twice (lines 11, 15 in Algorithm 4) with decreasing number of tasks as inputs to the algorithm. Therefore, considering rescue incorporation, the worst-case time complexity of our approach becomes O(mmlog(m)) or O(m 2 log(m)). 5.7 Results To illustrate some practical examples, we have generated task re-assignment suggestions for some mission scenarios similar to those we illustrated in our earlier works for alert generation. The scenario depicted in Fig. 5.6(a) is the estimated current state of a mission where six mobile robots explore some areas-of-interest (AoI) in a several kilometer-square environment, and search for objects-of-interest (OoI). Two robots have been disabled, two robots are operating in the eld based on previous task assignments, and two robots are 89 Figure 5.6: Example mission for generating task reallocation suggestions. (a) Estimated current mission status, with generated alerts where robots R1-R4 are on eld robots, R1 and R2 are currently operating based on previous task assignment. R3 and R4 are disabled robots. R5 and R6 are two robots currently available to humans for direct tasking. (b) Estimated task performance based on their current task schedule. R2 will go to risky region-5 after nishing exploring 7, and will get disabled. R3 is unable to complete its task in region-3 due to lack of agents (c) New task network with all tasks which are not nished, updated task requirements, task criticality, and additional task like "relay R2" to address a contingency. The numbers on the graph represent exploration of respective regions and searching for survivors / objects-of-interest. available to humans to communicate directly. The generated alerts show dierent critical mission updates, which will be negatively impacting the mission. Fig. 5.6(b) shows the estimated task performance of the on- eld robots between the current time and the future, operating based on previous task assignments. Due to the change in the mission, none of the robots would be able to perform as previously planned, and potential danger is awaiting. Based on the mission updates, a new task network is created with all tasks that are not completed by the current time as provided in Fig. 5.6(c). It has some critical tasks. A relay task with a deadline has been created to relay information to R2 in time to prevent it from going into a risky zone and becoming disabled. Human supervisor estimated the risk of rescuing each of the disabled robots, and assign values to p r i ; 8i2f1; 2; 3g;8r2fR3;R4g. We use 4 dierent mission settings based on the mission scenario in Fig. 5.6 to observe how the task reallocation suggestions change with changing mission situations and gain some insights regarding optimal rescue scheduling. We also compute the percent reduction of modied makespan with respect to strategy X 0 90 Figure 5.7: Generated task scheduleM = fM1;M2;M3g using our approach which incorporated 1 rescue at beginning of the mission for setting II. Here, the probabilities of the mission threads, i =p R3 i ;8i2f1; 2; 3g, which correspond to the task plansMi. Figure 5.8: Five mission threads with corresponding mission schedules are generated for setting IV using our mission scenario, where a second rescue is attempted only when the rst rescue failed (outcome-2), otherwise second rescue is not attempted. that does not incorporate any rescue attempt. Let, two alternative sets of critical tasks are T a =f3; 12g (Fig. 5.6) and T b =f1; 2; 3; 12; 13g. We assume that both the disabled robots have the same rescue risk; therefore the probability of the same outcome is the same regardless of the robot being rescued. Two alternative sets of rescue outcome probabilities are P a = (0:9; 0:08; 0:02) and P b = (0:65; 0:2; 0:15), where the three numbers in the tuple refers to the probability of three outcomes (p r 1 ;p r 2 ;p r 3 ) from attempting rescue on disabled robot 91 Table 5.5: Information on the generated task reallocation suggestions using our method for four mission scenarios (similar to Fig. 5.6, but with 4 dierent settings or modications): percent improvement in reducing the modied makespan compared to attempting no rescue, maximum number of rescue(s) attempted, number of mission threads k M , and the rescue time(s). Setting-I Setting-II Setting-III Setting-IV (T b ;P b ) (Ta;P b ) (T b ;Pa), task 4 removed (Ta;Pa) % reduction in E[S] using our method w.r.t. Strategy X0 0.0 2.6 7.1 9.7 Maximum # of rescues attempted 0 1 1 2 # of mission progression threads (k M ) 1 3 3 5 Rescue time(s) - Beginning Middle Beginning, back-to-back r2fR3;R4g. Clearly, P a refers to low-risk rescue situations, while P b is indicates higher risks. Settings I, II, III and IV's specications are given in Table 5.5, along with the information or features of generated task reallocation suggestions for each of them. Setting I requires no rescue attempts, which has a risky rescue option. Setting II also has the risky rescue setting, but it has less number of critical tasks, for which we found that a single rescue was performed in the beginning. The generated task plan is provided in Fig. 5.7. In setting IV, when we only change the rescue probability parameters compared to setting II, we observe a maximum of two rescues attempted in certain mission threads. The format of the generated task plan is shown in Fig. 5.8 with ve possible mission progression threads depending upon the rescue outcomes. Setting III presents a dierent mission setting as mentioned, with task 4 removed from the task network. In this case, the generated task plan scheduled the single rescue attempt in the middle of the mission. We have observed in this case that scheduling the rescue after completing all the critical tasks is more advantageous than scheduling it at the beginning, which could cause a delay in the execution of the critical tasks. 5.8 Summary We have presented our heuristics-based approach for the generation of task reallocation suggestions to handle contingencies and promote faster mission completion by multi-robot teams operating in challenging environments. Our method has minimized the expected value of the modied makespan that incorporates task criticality and incomplete tasks. We have been able to intelligently schedule optional tasks, such as relay and rescue, to promote retasking information propagation and taking a measured risk with a possibility 92 of increasing team size to improve expected mission performance. We have found competitive performance using our heuristics-based method compared to the optimal expected value of the metric found from an exhaustive search applied to simulated problems without rescue consideration. We have also demonstrated performance enhancement compared to three simplistic approaches regarding incorporating the decision of rescue tasks. Lastly, we have used our task reallocation suggestion generation system for some specic search and rescue type mission scenarios to observe the applicability and value of our work and gain insights on the choice of optimal rescue scheduling depending upon mission conditions. 93 Chapter 6 Assessing Value of Alerts and Suggestions in Improving Human Decision-Making Using a Human Subject Study This chapter is based upon a manuscript under review [11]. 6.1 Introduction An alert generation framework, described in the earlier chapters, was developed to help human supervisors make informed decisions, reduce the load of mission modeling, and mitigate the detrimental eects of delayed information ow caused by intermittent communication. These alerts are generated for human-specied, negatively-impactful situations that can occur during the mission [5, 6, 10]. These alerts only forewarn the supervisors about important or unwanted mission situations but do not explicitly give any assistance on how to address them in the decision-making process. Towards this, an automated suggestion generation for robot retasking decisions to handle potential contingencies was developed [9], as described in the previous chapter. This chapter details on a human subjects study, conducted to evaluate the usefulness of our alert framework with robot tasking suggestions. Here, we are specically interested in investigating whether alerts make a human supervisor's decision-making better and faster in challenging multi-robot missions. We also investigate how an individual's belief or trait can aect performance and whether users can be easily trained to use our system. In one of our earlier works [6] we performed a preliminary user study to nd how alerts can improve the human capacity to identify potential contingencies in a single decision-making instance with a static mission state. However, it did not include robot tasking suggestions which we later presented in [9]. 94 Our earlier study only had one-o decision-making, i.e., it did not involve sequential decision-making, which is an important aspect of supervising a long-duration multi-robot mission. The sequence of making decisions, observing the consequences of decisions, and adapting one's strategies with mission progression are many dimensions found in an actual mission. In this chapter, we combine our approaches for generating alerts and suggestions and perform a com- prehensive human subjects study that represents an actual mission more accurately by including sequential decision-making and other aspects. This work follows the guidelines presented by Homan et al. for plan- ning, executing, analyzing, and reporting hypothesis-driven experiments in the human-robot domain [79]. We design and conduct our user study experiments with the aim to answer the following research questions. (i) Can auto-generated alerts and suggestions improve mission performance? (ii) Can auto-generated alerts promote faster decision-making? (iii) How does an individual's trust in automation technologies aect per- formance? (iv) Can our framework be used with minimal training overhead? We present our user study ndings and provide insights on the eects of using our alert framework in this chapter. This chapter is structured as follows. A brief literature review is presented in section 6.2. Section 6.3 details the overview of our system architecture. It describes how dierent modules are integrated and how they interact with each other. Section 6.3.1 includes a brief summary of our technology for the automated generation of alerts and suggestions. Section 6.3.2 introduces what kinds of robot tasks we consider in our work, and how humans are to make decisions on task reallocation to robots. Section 6.3.3 lays the ground for our user study design and lists down our hypotheses. Next, section 6.4 describes everything related to our human subject study. The design of experiments is explained in section 6.4.1. The description of the mission, the decision-making of the participants, the user interface, and the alerts and suggestions provided in the study are detailed in sections 6.4.2-6.4.4. The remaining subsections 6.4.5-6.4.7 contain information about our user study participants, a detailed description of the study protocol, and the key measures towards hypothesis testing in our experiments. Section 6.5 has a detailed description of our results from the user study. The generic ndings are presented in section 6.5.1, and the results of testing each of our hypotheses are illustrated in section 6.5.2. Finally, section 6.5.3 highlights the key takeaways and limitations of our user study. 95 6.2 Related Works Many research papers in cognitive sciences have illustrated human limitations, human errors, cognitive overload and related topics [128, 147, 159, 171]. Human decisions get negatively aected by psychological stress [98], time stress [116], information overload [126] and resource limitation [148]. Decision support systems (DSS) have the potential to overcome these human limitations and enhance the decision-making of humans. Research has shown that computerized cognitive aids and AI-generated decision aids can have a remarkable positive eect on both decision-making eciency and eectiveness [25,26,65,195]. In the literature, there are several forms of assistance that DSSs typically provide to the decision-maker. [68] outlines multiple stages of decision support (the simplest form to the most advanced kind) in the context of the human experimenter with mobile robots. The most basic type of decision support can be some information aiding decision-making such as alerts, \what-if" analysis, and anomaly monitoring and detection [4, 5, 17, 96, 124, 187, 219]. Even though this kind of information assistance is helpful, most of the burden for decision-making still lies with the humans. More sophisticated support is suggestion aiding decision making, which typically takes the form of recommendations for decisions or decision parameters [77,119,163,178,211]. This can signicantly reduce the decision-making burden. There are also works that provide both information and suggestions for assisting the decision-making process [66, 208, 214]. In this paper, we take such an approach by providing a combination of alerts and robot tasking suggestions to human supervisors in multi-robot missions. There are many relevant works in the context of human-robot interactions across dierent application domains. Research towards identifying risks and anomalies, and preventing unsafe robot actions are found in [4, 84, 100, 134, 135, 174]. A DSS platform for situational awareness, planning, observation, archiving and data analysis in ocean exploration using robots is presented in [66]. [82] describes a decision-making behavior model for human-robot interaction in multi-robot systems. [125] presents an adaptive decision- making system supported by user preference predictions for human-robot interactive communication. In the context of search and rescue operations with robots, [20] focuses mainly on the user interface aspect, whereas [27] talks about situational awareness, team communication, and the interaction, and [220] presents a human-robot interface which covers a wide range of autonomy, planning and behavior modeling. There are 96 some works that are even more aligned with our alert generation framework in terms of providing assistance in human decision-making. [28] proposes a framework that does an automated selection of robots so that humans can simply issue commands to the robot team, instead of assigning tasks to individual robots. [152] illustrates a human-centric approach for supervisory control of multi-robot teams in dynamic domains, and investigates the complexity of the task-assignment situation and its correlation to the mental workload of the supervisors. Another work presents a multi-robot cooperative learning technique to allow eective task allocation of robots by themselves, where human intervention is requested only at the time of facing contingencies [120]. 6.3 System Overview Our alert generation framework is applicable to multi-robot teaming scenarios that require human supervision in large, unstructured environments, such as those used in military campaigns and HA/DR eorts. In these cases, people send a group of robots into the operating environment to eciently investigate the impacted areas, gather vital information, and carry out specic tasks. The robots receive task plans from the humans, including a nominal task sequence and contingency task plans as necessary. Human supervisors develop a high-level mission strategy to accomplish some specic objectives. Robots are designed to be able to make low-level decisions in order to carry out activities that humans assign to them, such as navigation and exploration, the identication and manipulation of objects of interest, and the completion of other mission-specic tasks. The system architecture is depicted in Figure 6.1 which contains three major components{ multi-robot mission, human-robot interaction interface, and alert generation framework. The multi-robot team is avail- able for mission execution and humans serve in a supervisory role. The human supervisors use the human- robot interaction interface to communicate with the robots, either to view newly-received mission updates or to assign tasks to robots. In the back end of the user interface our alert generation framework is integrated to assist with the supervisor's decision-making process. It processes the incoming data from the mission and provides alerts and suggestions to humans. As the gure shows, there are two process blocks in the 97 Figure 6.1: System architecture diagram showing the two main process blocks (gray-colored, highlighted with a green dotted rectangle) of our alert generation framework and their interactions with the multi-robot mission execution and human-robot interaction interface. alert framework. The rst provides mission predictions to generate alerts for potential consequential mission situations and the second generates robot tasking suggestions to handle the contingencies. 6.3.1 Alert Generation Framework We focus on missions with communication constraints where humans will not remain in constant commu- nication with the robots executing mission tasks at distant locations. So, humans receive intermittent and delayed information from the robots returning to any base station or communication station. The user in- terface is where humans can view mission updates. These mission updates, along with existing task plans of robots on the eld in the ongoing mission, are fed into the alert generation framework. The alert generation module uses human-specied unwanted situations to trigger alerts. The alert conditions are mathemati- cally expressed as probabilistic metric temporal logic formulae [10]. This oers human users exibility and extensibility. We developed computational speed-up techniques for performing a large number of forward simulations with the mission- and task-models in a short amount of time [6]. The forward simulation, performed in a discrete event simulation paradigm, uses mission updates from the past to predict what is currently happening outside of the communications range, or what will happen in the near future. The goal is to improve the humans' understanding of mission progression and prioritization of important issues and resources. These mission predictions are made with sucient accuracy and alerts are generated within seconds to ensure timely decision-making [5]. We also proposed a method for robot retasking suggestion 98 generation at any mission state [9]. It takes into account how probable mission contingencies may aect the tasks, the robots, and the performance of the robots when functioning following earlier task plans. These rec- ommendations promote mission progression while handling contingencies, incorporate task criticality where humans need to prioritize assigning certain tasks over others, and ensure faster completion of higher priority tasks. 6.3.2 Robot Tasking Decision The decision to be made by the human supervisor involves assigning tasks to the robot(s) available at the communication station. A decision-making instance can be at the beginning of the mission to dispatch the robots or in the middle of the mission when some robot has returned. This decision can be made based on available mission information, with or without the additional assistance from the alert framework. In a multi- robot mission, there are some robot tasks that directly contribute to the completion of a mission, which we call the mission tasks. For example, \search and explore" is the main mission task when a multi-robot team is sent to explore some region and identify certain objects-of-interest. If there are several discrete regions, then the search and exploration of dierent regions can be considered as the tasks that can be assigned to the robots. Naturally in a mission, the list of available tasks and critical tasks dynamically change throughout a mission. Humans need to strategically choose which tasks to assign to the available robot based on task priority, spatial locations, risk levels, robot availability, and mission objectives. We introduce two more optional tasks which can improve resiliency and mission performance given that robots can become immobilized due to terrain conditions and other challenges [9]. The rst optional task is relay, which can be used to share new task assignment information from one robot to another robot that is outside of communications range. The robot that is assigned the relay task must rst navigate to the target robot before sharing the desired information. Relay can be used to spread pertinent information such as preventing an uninformed robot from entering dangerous scenarios or prioritizing critical tasks based on the latest mission updates. Thus incorporating a relay can extend the reachability of humans to the robot team and, in some cases, remove the need for robots to return within communications range. However, this task introduces more complexity in the task assignment process. 99 There is a non-zero likelihood of robotic failure as the robots operate in the complex mission space, which can be due to stochastic events like hardware or software faults or environmental factors like challenging terrain. When a robot becomes immobile we assume that other suciently-equipped robots may be able to help resuscitate the disabled robot. We call this task robot rescue [7, 8], which is the second optional task. The diculty and risk of these rescue operations depend on the specic situation. There are three probable outcomes of a rescue operation { (i) successful rescue where the disabled robot is successfully revived, (ii) unsuccessful rescue where the disabled robot remains disabled, and (iii) catastrophic failure where the rescuer robot also becomes disabled. We assume that expert human supervisors will be able to examine the received data (e.g., image, video, etc.) regarding each disabled robot, estimate the risk level of rescuing it, and assign a probability of each rescue outcome. For each disabled robot, whether to rescue, when to rescue and which functional robot should attempt the rescue are some decision parameters. Moreover, if there is more than one disabled robot, there are more complexities in the robot-tasking decision-making process. As a robot rescue can dynamically aect the total number of functional robots, mission performance can be impacted signicantly. 6.3.3 Hypotheses Figure 6.2: We conduct a human subjects study to compare a human's ability to task robots with and without our alert generation framework. Here, the emulator is our custom multi-robot simulator which is used as the surrogate for an actual mission. 100 We hypothesize that the supervisor's decision-making and the team's mission performance will improve when additional assistance is provided by our alert framework. In this regard, we design an experiment to compare results when the alert framework is turned on versus disabled as shown in Figure 6.2. Based on our preliminary human subject study presented in [6], we believe that alerts help humans perceive the mission contingencies better at static mission states, and thus help with making decisions regarding robot tasking. In this work, we conduct an extensive study that investigates the eects of incorporating our alert system. In order to achieve a holistic picture, the participants play the entire mission, which requires sequential decision- making of robot tasking, observing the consequences of their decisions, and adjusting their strategies as the mission progresses. Our hypotheses are based on important metrics and aspects of multi-robot search and exploration mis- sions. First, better decision-making as a supervisor should be correlated to improved mission performance. In the kinds of missions we are studying, one important aspect is mission progression over time. Another aspect is task prioritization which involves discovering objects-of-interest sooner during the mission. Second, making prompt decisions in time-critical missions is essential. Third, we would like to investigate how a person's individual traits can aect performance while using our system. Specically, whether an individ- ual's trust in automation can in uence how accepting a person is towards the recommendations provided by our automated system, and is this aects performance. Lastly, we want to design a system that is easily understandable and easy to use. Our hypotheses are as follows: Hypothesis 1 Alerts improve overall mission progression with time. Hypothesis 2 Alerts enable faster completion of higher-priority tasks. Hypothesis 3 Alerts enable faster decision-making during retasking robots. Hypothesis 4 An individual's trust in generic automation technology correlates to higher acceptance of our suggestions and improved mission performance. Hypothesis 5 A short and quick training is sucient to use the alert generation framework. 101 6.4 Human Subjects Study 6.4.1 Experimental Design Our human subject study is designed to evaluate the eects of incorporating our alert generation framework in human decision-making. Participants receive in-person training on a multi-robot mission and our user interface and then play a series of missions as a supervisor with our alert framework enabled or disabled. We take a mixed approach using within-subject design and between-subject design for testing our hypotheses. In this paper, we compare cases when alerts are not available and when alerts are available. We use a limited number of participants (N = 20) and rely on within-subject comparison for mission performance with alerts versus without. However, this requires each participant to play more than one game to measure the performance under both settings. If a participant plays just one game per setting, the study could show some biases due to learning eects and fatigue. To mitigate these issues, we design four missions with similar size and characteristics (related to mission progression and number of decision-making instances) which each participant will play in a random order and a random setting, for a total of two games per setting. This way the learning eects and fatigue, if applicable, are expected to aect the performance in both settings similarly. We perform within-subject analysis for Hypotheses 1, 3, and 4. Hypotheses 1 and 3 include a comparison between two settings within each participant. Hypothesis 4 is implicitly within-subject in nature, where we compare individual traits of a participant and correlate that to that person's performance. Hypothesis 2 also deals with a comparison between \with alerts" and \without alerts". However, the four missions used in the experiment had a varying number of high-priority tasks, and therefore the assumption made on \missions having similar characteristics" used for Hypotheses 1, 3 does not hold for the metric for Hypothesis 2. For this, we separate each of the four missions and use comparison between groups of participants that receive and don't receive alerts. For Hypothesis 5 we consider the binary outcome for each participant with respect to training suciency and use the twenty samples to predict on the trend to be expected in a large generic user pool. 102 Figure 6.3: The four mission scenes, I-IV, each containing 30 Areas-of-Interest. 6.4.2 Mission Description and Decision-Making As mentioned in the previous subsection, we design four missions (I-IV ) with similar size and characteristics to use in our human subjects study. For each mission, a user plays a simulated eight-hour period of a unique mission which includes approximately 10 15 decision instances. The game can end sooner if all tasks are completed or all robots become disabled before the time runs out. Referred to as an emulator, we use a custom multi-robot simulator designed using the Robot Operating System (ROS) framework as the surrogate of an actual real-world mission. The emulator performs discrete-time simulations for robot navigation and other tasks using probability distributions. It runs up to 100x faster than real-time so that humans receive new information from robots that return to the base station every one or two minutes and need to assign new tasks to robots. This way, a large-scale mission is completed in twenty minutes of wall-clock time by scaling down the long wait times between robots returning to the communication center. Most trials in this human subject study proceed in the following way: A participant (i) observes mission updates (and alert/suggestion information, if available), including asynchronous information as some robot(s) return with 103 new mission updates every 1-2 minutes and (ii) makes a (re)tasking decision that dispatches the robot(s). Steps (i) and (ii) are repeated as needed by the participant. We dene \search and explore" missions in larger than one hundred square kilometers of a simulated geographical region. There are some discrete areas in the large environment, identied prior to the start of the mission, which we call areas-of-interest (AoIs). Humans need to utilize a multi-robot team to explore all the AoIs and search for objects-of-interest (OoIs). The locations of the OoIs are unknown at the beginning, but as the mission progresses humans can make better guesses as we assume neighboring areas from any OoI have a higher likelihood to contain more OoIs. Search and identication of OoIs is critical and so areas, where OoIs are found, are considered higher priority areas. AoIs also have a risk pattern that is unknown a-priori and relates to the likelihood of robots becoming disabled while operating in that area, including the probability of successful robot rescue. As with higher priority areas, the supervisors can gain a better understanding of the regional risk factors as they start receiving more mission updates. Each mission has 30 mission tasks corresponding to \search and exploration" of the 30 AoIs. Tasks become available as the participant progresses through the mission. To reduce the number of decision variables we limit the number of mission tasks that can be assigned to each robot at one decision-making instance. In addition to the available mission tasks, there are two kinds of optional tasks, \relay" and \rescue". The number of relay options depends on the number of functional robots on the eld outside the communication zone. Similarly, the number of robot rescue options depends on disabled robots. In our study, we assume that the supervisor can assign at most one mission task and one optional task at any decision-making instance. Once an assignment is issued to a robot, it will attempt to complete its task and then return to the communication station with the latest information if it remains functional. Some robots can become disabled during their operations and other returning robots might bring that news to the supervisor so that robot rescue can be considered. A returning robot also brings progress updates of all other functional robots that it encountered during its operation. 104 (a) Left Monitor: Map View (b) Right Monitor: Task Assignment Tool Figure 6.4: A representative example view of the two monitor setup in the middle of a gameplay of mission II. The screenshot is taken when the alert framework is enabled, and the task assignment recommendations are shown in the text highlighted yellow on the right screen. 105 6.4.3 User Interface We use RViz, the 3D graphical interface of ROS, for developing our user interface. A dual computer display setup is used, as shown in Figure 6.4. The left monitor contains the overlay of all major mission update information viewable on the satellite image. The last seen location, state, and time information of all the out-of-comm robots on the eld appear on the map. The location of OoIs, if discovered or known by any means, also appears on the map. There are three status conditions of robots { disabled robots which will remain immobilized unless rescued, functional robots on the eld which are outside the communication range, and lastly the robots at the communication station which are ready to accept task assignments. The spatial locations of the disabled robots and OoIs provide hints of the internal risk map in the environment and higher priority areas, respectively. Additional mission updates, for example, information related to mission task progresses and rescue tasks (scheduled or completed), appear in the text panel on the left of the map image. The right monitor is reserved for everything related to task assignments. The supervisor can view the nominal task performance timeline of the robots on the eld based on their previously assigned tasks. The main portion of this second display is the interactive tool for assigning tasks to the robots available at the communication center. This interaction portion primarily shows the available-to-be-tasked robots and some interactive blocks as task bars. These task bars include the available mission tasks as well as the two types of optional tasks at a particular instance. A user can drag the task bars to each of the robots and click on a button to see the nominal task performance timeline for that particular assignment. This timeline internally considers the nominal travel times for the robots and other dependencies, if any. Assigning any optional task can allow the user to assign tasks to other robots apart from the initially available robots at the communication center. The task assignment tool has special buttons to incorporate this dynamism in task assignments. When a \relay" task is scheduled for a particular out-of-comm robot, that robot also becomes eligible to accept task assignments from the time it gets relayed to. The task assignment tool brings this additional robot and the nominal task performance timeline can be viewed for the relayed robot as well. \Rescue" tasks can have three outcomes and a user needs to issue alternative task assignments for all possible realities ahead of time. If there are two available robots and one is assigned a 106 Figure 6.5: A representative view of the interactive task assignment section of the second display shows the nominal task performance timeline for a particular task assignment which includes one rescue operation. R1 and R2 are the two available robots. R1 is assigned to attempt a rescue on a disabled robot, R4. The gure shows task assignment issued for three alternative outcomes of the rescue operation having three, two and one functional robots to be tasked after the rescue. rescue task, there will appear three alternative task assignment sections. The rst one for a successful rescue includes the rescued robot accepting new task from the rescue time. The second one for unsuccessful rescue has no change in robots to be tasked. Finally, the third one will have one less robot, i.e., the rescuer robot will become disabled and not available to perform any tasks after the rescue attempt. Figure 6.5 illustrates such a representative situation on the task assignment tool. 6.4.4 Alerts and Suggestions The left text panel on the right display of the user interface is dedicated to showing the alerts and suggestions (see Figure 6.4 (b)) from our framework. In our user study, our alert framework provided one or more discrete task assignment suggestions based on mission updates, risk assessment, and priority information at every decision-making instance. The framework is an independent module that uses the latest mission update to do a probabilistic assessment of the benets and risks/costs associated with incorporating any optional task (relay and rescue). It then makes a suggestion to maximize the expected mission performance and highlights high-priority areas to explore based on the most recent information on OoIs. In order to make a reasonable 107 Figure 6.6: Representative display of the alerts and suggestions panel on the task assignment monitor, when the alert framework is (a) disabled and (b) enabled. When alerts are available, the suggestions are highlighted in yellow color to attract the attention of the user. evaluation using a limited number of human participants in the study, we assume that the mission execution follows a highly probable route. This is done so that a very low probability event does not skew the ndings in our user study by unfairly aecting some mission gameplay. When a mission is being played without the alert framework, this panel shows \Alert/Recommendation - Not Available". Otherwise, this panel shows some suggestions for robot tasking as presented in Figure 6.6. 6.4.5 Participants We recruited N = 20 participants who were engineering graduate students at the University of Southern California (USC). The participants were required to be more than 18 years old and not have cognitive impairment that could make it challenging for them to distinguish between colors and play computer games. A recruitment email was circulated among all graduate students at USC from the graduate study coordinator. The email mentioned the eligibility criteria, study duration (2 hours), location (USC Center for Advanced Manufacturing), and compensation amount ($20 Amazon gift card). A calendar appointment scheduler was used to schedule the study session with each participant. The participants were unknown to the PI and 108 were required to be unaliated with the faculty advisor of the study. The consent form was sent to the participants before the study, which they signed on the day of the study. The consent form mentioned that anonymity would be maintained with the saved data and the faculty advisor would not access any of the data or participant information. 6.4.6 Procedure The study procedure involved four steps totaling between 105 and 120 minutes. Step 1 { Introduction and Training: The study was conducted at the Center for Advanced Manufactur- ing, an o-campus research facility of USC. After the participants arrived at the location, an examiner let them into a private oce room with a two-monitor computer setup. Once seated, they were rst provided the informed consent form and asked to sign the form which includes the questionnaires related to the eli- gibility criteria. Next, they were given training for the user study. For this, the examiner left them in the room alone to watch a video tutorial for 20 30 minutes. They could pause, rewind and rewatch as needed. The computer has a speaker and microphone system, and a zoom audio call connected at all times. They could ask the examiner questions at any time for clarication. The training video contained the mission description, assumptions, and objectives. Since the participants need to make robot tasking decisions in multi-robot missions, they would need to know how to assess dierent mission situations/contingencies and how to handle them. The tutorial contained detailed guidelines. The tutorial also included details on the user interface and how to use it. Once they were comfortable with the training video contents, they let the examiner know over the voice call. Step 2 { Trial Round: This is the stage where the participants used the interface for a trial run with a smaller-scale mission set up with ve robots and twelve AoIs. The examiner uses remote access to the computer to launch the trial mission and maintained access throughout this phase so that they could take control whenever needed. This way they could guide the participants and show them whenever any questions arose or the participants were not using or understanding the interface well. This portion took 15 20 minutes. 109 Step 3 { Mission Gameplay: Once the participants became comfortable using the system, the actual mission play began. Each participant was randomly assigned a sequence of four missions, of which two randomly selected missions would run with the alert system and the remaining two without. They were not told beforehand which features (alerts/suggestions) were available in each mission. Each mission was launched using remote access to their computer and then the participants were to play the mission themselves without outside intervention. The participants could ask the examiner questions if any problems occurred via the voice call. In between two consecutive missions, they were asked if they would like to take a 5-minute break. Each mission took about 15 20 minutes, which included ten or more decision-making instances. Step 4 { Post-Surveys: After the missions were completed, there were some concluding surveys and questionnaires. The rst portion included questions and discussions mostly related to the mission gameplay across the dierent missions, with or without alerts. It included questions on the understandability of the missions and also the usage of the interface. The participants also answered questions about their decision- making and reasoning. They were asked about their remarks on the alerts and how they aected their decisions. They also provided suggestions to improve the system. Lastly, they completed a standard test questionnaire used for assessing an individual's trust in autonomy (as described in Section 6.4.7.3). The examiner stayed at a remote location the entire time and went back to the participant after nishing all surveys. 6.4.7 Measures To evaluate the impact of our alert system, we measured multiple metrics related to mission performance. We also measured the time taken to make decisions and participants' trust in automation. We also evaluate the training of the system qualitatively. 6.4.7.1 Mission Performance Mission performance has two orthogonal metrics to consider, as highlighted in Hypotheses 1 and 2. First, the overall mission progression needs to be measured for comparing performances when alerts are provided and when they are not. We use two metrics to measure mission progression, (a) whether the mission was 110 completed (i.e., no unnished task) within the eight simulated hours of mission time and (b) how many tasks remain after the mission timeout. Here, tasks refer to the search and exploration of AoIs; the unnished tasks refer to the areas that are yet to be explored. This corresponds to the rst objective provided in the training phase, exploring as many areas as possible within the mission time limit. We also observe if any mission gameplay hits \game over" before the mission time-out occurs, i.e., if all robots become disabled at any time before eight hours of mission time. Secondly, we measure the time taken to discover OoIs in selective areas which are not known beforehand. As the mission progresses and new information is revealed, humans are able to identify which areas are more likely to contain OoIs. The participants are instructed in the second mission objective during training to prioritize exploring those areas sooner. 6.4.7.2 Decision Time We measure how much time the participants take to make each decision. This measured time corresponds to the time taken by the participant to observe all reported mission updates in the user interface, assess the mission, make task assignment decisions, and use the interactive tools to issue the task assigned to the robots to dispatch them. There is a count-down timer of ninety seconds, which is the preferred time limit for making the decision as mentioned in the third objective in the mission training. We anticipate that participants would need more time than that in several decision-making instances, especially when there are several contingencies, and con icting situations arise from multiple robotic failures, risky regions, the discovery of OoIs, and high-priority areas. However, we anticipate that people will make more timely decisions when alerts and recommendations are provided as stated in Hypothesis 3. 6.4.7.3 Trust in Alerts We use a randomly ordered version of the original test questionnaire for measuring \trust in automation" by Jian et al. in [87]. This is because Gutzwiller et al. recommend the randomized version in a recent paper [72] as it achieves more unbiased results than the original. All twenty participants in our study completed this questionnaire and we used it to derive a trust score for each of them. We also recorded whether each participant accepted or rejected each recommendation provided to them during the missions when alerts 111 were provided. We use these measures of trust score, acceptance of alerts and mission performance to test Hypothesis 4. 6.4.7.4 Training on Alerts Our detailed protocol for the study mentions that if a participant needs to interrupt a mission out of confusion or to ask questions during or after any mission, we will discard that mission data. We note the number of missions played with the alert framework which need to be discarded due to any confusion or queries from any participant. This number can be indicative of the insuciency of training for our system. In addition, we use the subjective ndings from the concluding surveys of the participants with regard to the understandability and usage of the system. Thus, we attempt to draw conclusions about Hypothesis 5 based on whether the short training was sucient for the participants, which would indicate that the system can be used with minimal training overhead. 6.5 Results 6.5.1 Findings The data collected in our human subject study shows an overall positive impact of providing alerts and suggestions in the tested multi-robot missions. The rst objective was to nish the search and exploration of as many AoIs as possible within eight simulated hours of mission time. If all AoIs are explored the mission is completed. We observed that 19 out of 20 participants completed at least one mission out of the two missions played with alerts. If we compare each participant's mission completion rate with alerts versus without we see that 18 participants completed more missions with alerts than without. To measure mission progression for both completed and unnished missions, we observe each participant's total number of unnished tasks in the two missions with alerts versus the two missions without alerts. We observe that all participants had fewer unnished tasks when alerts were available. Table 6.1 contains the mission progression results for all participants. All missions have similar features with respect to exploration tasks, e.g., the same number of AoIs, same level of risks, similar geographical 112 Table 6.1: Overall mission progression metrics observed when alerts are provided compared to missions played without alerts Missions completed Average number of remaining Game Over (all robots becoming disabled) tasks in incomplete missions before 8 hours of mission time Without Alerts 2/40 5.50 10/40 With Alerts 26/40 2.20 1/40 area, and navigation speed; therefore, it is fair to combine the results. Overall, we observe that there is a tenfold improvement in terms of achieving mission completion with alerts. If we consider only the incomplete missions, we observe that missions played with alerts have a much smaller average number of remaining tasks. We also observe alerts provided tenfold improvement in terms of avoiding \game over" situations. Table 6.2: Average mission time taken to complete all higher-priority tasks { Observed delay in completing high- priority tasks without alerts compared to with alerts, in each of the four missions Mission ID I II III IV Number of high priority areas 7 11 7 8 Average delay (in mission time) in discovering all objects-of-interest without alerts 75 mins 157 mins 81 mins 142 mins The second directive provided to the participants is to prioritize exploring areas that contain or are likely to contain OoIs. Only some AoIs (711 out of 30) contained OoIs, which were considered the higher priority areas. Exploring such areas sooner means that OoIs are discovered sooner. As each mission contained a dierent number of high-priority areas and OoIs (see Table 6.2), we kept the data for each of the missions separately. We have observed that all 80 mission playouts had all high-priority tasks completed before mission time out or game over. We compute the average time taken to complete all high-priority tasks with alerts and without alerts for each mission and calculate the dierence as the delay, and we observe that the games played without alerts took more time on average. The average delay and number of high-priority areas for each mission are presented in Table 6.2. We see a higher delay in missions which has a higher number of high-priority regions as expected. 113 The third directive provided to the participants was to make robot tasking decisions quickly, preferably within the 90 second time limit. We consider all decisions made within that time limit to be \timely" decisions and cases when robot tasking took longer than that are considered \untimely" decisions. We collected this data for 15 participants as the remaining participants' data regarding the time taken to make decisions could not be recovered due to some malfunction with the screen recorder (Kazam). We observe that more than 50% of participants in this population made fewer untimely decisions with alerts compared to those without. There were only three participants that made fewer untimely decisions without alerts; two of these performed particularly poorly without alerts and seemed to be following a random or overly simplistic strategy. We analyzed the questionnaires for trust in automation completed by all 20 participants. The score is computed on the scale of [0; 6], where [0; 3) range represents distrust and (3; 6] represents positive trust. We observe that all participants scored higher than 3, which indicates some level of trust in automation in general. The average score was 4:168 with a standard deviation of 0:579. Overall, we observe that higher- scoring participants showed rapid acceptance of the suggestions when played with the alert framework. Only one participant (with a trust score of 3:08, which was the lowest of the population) rejected several recommendations. No other participants completely rejected any suggestion. However, six participants delayed accepting some recommendations, i.e., they did not accept the rst instance when a suggestion was provided, but later during the mission, they accepted those suggestions. One participant's data was not much aligned with the generic pattern of higher trust scores correlating with lower rejections. The participant had the second-lowest trust score (3:16), but accepted almost all suggestions provided by the alert framework promptly, and mentioned verbally in the post-study discussion that he had signicant trust in automation. 6.5.2 Hypothesis Testing 6.5.2.1 Hypothesis 1: Alerts improve overall mission progression with time. We compare the performance of each of the twenty participants when alerts were provided in two missions and when alerts were unavailable in the other two missions. The rst metric ( 1a ) denotes the increase in the number of missions completed (out of two) with alerts compared to without. The values can range between 114 [2; 2], where negative values indicate a fewer number of missions completed with alerts versus without. There was no participant whose value came to be negative, and only 2 participants had an equal number of missions completed with and without alerts. The ndings are illustrated in Figure 6.7 (a). Figure 6.7: For Hypothesis 1, we measure each participant's (a) increase in the number of missions completed with alerts compared to without, (b) decrease in the total number of unnished mission tasks. The bar plots are showing the frequency of dierent values of the two metrics that we got from the 20 participants' data. They are showing the number of participants (on the y-axis) having dierent values for these two metrics (on the x-axis). Any positive number (on the x-axis) means improvement using alerts, and larger positive numbers indicate bigger improvement. The second metric is in terms of the total unnished tasks in the two missions played with or without alerts. The decrease in total unnished tasks ( 1b ) played with alerts compared to without (# unnished tasks with alerts - # unnished tasks without alerts) shows improvement in mission progression. We nd that all twenty participants had at least one fewer unnished task with alerts and the values lie within [1,16] for all participants as shown in Figure 6.7 (b). We perform Student's t-tests for the hypothesis testing. For Hypothesis 1 we use a one-tailed t-test with paired samples (same participant's measures with alerts vs without), where N = 20. The null hypothesis and alternative hypothesis are as follows: H 0 : 1x 0, H a : 1x > 0 , where x2fa;bg. We found a signicant increase in the number of missions completed with alerts (t(19) = 8:72;p<:0001) and a decrease in the number of unnished tasks with alerts (t(19) = 10:61;p<:0001). 115 6.5.2.2 Hypothesis 2: Alerts enable faster completion of higher priority tasks. Each mission was conducted a total of 20 times by 20 participants, some of them conducted with alerts, some of them without alerts. Because of the random assignment of \with alerts" or \without alerts", each mission did not have an equal split between with alerts and without. The four missions had between 7 11 missions conducted with alerts and the remaining of the twenty were without alerts. Figure 6.8: For Hypothesis 2, we measure the time taken to complete higher priority tasks for each mission I-IV, without and with alerts. This comparison is done across the two groups of participants, each group playing with or without alerts for each mission. The plots show the average mission times (in minutes) with corresponding standard deviations of the distributions as error bars. The red and blue data represent cases without and with alerts, respectively. Figure 6.8 shows the average mission times to complete all higher priority areas (i.e., discovering all OoIs), and the standard deviation in the form of error bars for all four missions, with and without alerts. We attempt to test whether there is a statistically signicant decrease in the expected mission time for completing high-priority tasks with alerts compared to without, for each mission separately. For this, we use a t-test with unequal sample sizes (N 1 6=N 2 ) and similar variances (0:5< 2 1 2 2 < 2). All N 1 +N 2 = 20 samples for each mission are independent samples coming from dierent participants. The null hypothesis and alternative hypothesis are as follows: H 0 : 1 2 , H a : 1 > 2 , where the subscripts 1 ; 2 refer to without and with alerts, respectively. We have found (t(18) = 2:148;p =:0231), (t(18) = 4:831;p =:00006), 116 (t(18) = 2:3778;p = :0144), (t(18) = 4:8173;p = :00007), respectively for missions I-IV. Therefore, we can reject the null hypothesis with 95% condence for all four missions and accept the alternative hypothesis. 6.5.2.3 Hypothesis 3: Alerts enable faster decision-making during retasking robots. In this hypothesis testing, we analyze the total number of untimely decisions each participant made in the two missions with alerts and the two without alerts. We hypothesize that the number with alerts will be signicantly smaller than without alerts. Since all missions have similar characteristics in terms of the number of AoIs, risk factors, contingencies, and need for allocating optional tasks, we can do a pairwise comparison between each participant's performance with alerts versus without regardless of which mission they played for each conguration (i.e., with alerts or without). Table 6.3: Number of delayed decision-making with alerts versus without More delayed decisions with alerts No change Less delayed decisions with alerts # of participants 3=15 4=15 8=15 Table 6.3 presents the summary of our ndings with the data collected for 15 participants. 8 out of 15 people made 1 3 less delayed decisions with alerts compared to without. There were 4 participants who experienced no change, and 3 participants who performed slightly worse with alerts in terms of timely decision-making. We perform students' t-tests for the hypothesis testing. For Hypothesis 1 we use a one- tailed t-test with paired samples (same participant's measures with alerts vs without), where N = 15. The null hypothesis and alternative hypothesis are as follows: H 0 : 0, H a : > 0 . We found a signicant decrease in untimely decisions with alerts (t(13) = 1:785;p = 0:0488). 6.5.2.4 Hypothesis 4: An individual's trust in generic automation technology correlates to higher acceptance of our suggestions and improved mission performance. To analyze the acceptance of alerts, we observe the total number of recommendations that each participant ignored or rejected in the two missions played with alerts. We see this rejection number being between 0 and 5 (out of > 25) for all participants except one. This one participant had 26 rejections and also had the lowest trust score. We also look into our primary metric for mission performance, the number of tasks left 117 in the two missions with alerts, which inversely corresponds to mission progression achieved. We attempt to nd the correlation between each participant's trust score and their number of rejected recommendations and the number of tasks left. We observe that people with higher trust scores exhibited a lower number of rejections and a lower number of unnished tasks as presented in the two plots in Figure 6.9. Figure 6.9: Scatter plots showing how 20 individual participants' trust in automation (on the x-axis) aects (a) their total number of rejections of the recommendations (on the y-axis) and (b) mission performance in terms of the number of remaining tasks (on the y-axis), both when playing with alerts. Each participant's data is plotted with a specic color and marker shape combination in the plots. Plot (a) also shows a zoomed-in version for the 18 participants, excluding the 2 outliers with the least trust score and largely opposing trends of rejections. In plot (b), a lower number of remaining tasks corresponds to better mission performance. We perform a hypothesis test of the signicance of the correlation coecient, assuming a linear relation- ship between (a) trust score and recommendation rejections and (b) trust score and number of unnished tasks. We use the computed Pearson correlation coecient of the sample data r to hypothesize on the pop- ulation correlation coecient . The null hypothesis and alternative hypothesis are as follows: H 0 : x = 0, H a : x 6= 0, where x2fa;bg subscripts corresponds to the number of recommendation rejections and the number of unnished tasks, respectively. We nd (r =0:4073;t(18) = 1:8919;p = 0:0375) for (a), and (r =0:7654;t(18) = 5:0463;p = 0:00004) for (b). We found a signicant correlation between each participant's trust in automation and higher acceptance of alerts and improved mission performance. 118 6.5.2.5 Hypothesis 5: A short and quick training is sucient to use the alert generation framework. The introductory training video had two minutes' worth of material related to the alerts and suggestion panel. In the trial round, each person spent between zero and two minutes asking questions and getting answers related to the alerts. Thus, all participants took less than four additional minutes to understand the assistance provided by the alert framework. We consider a Bernoulli variable (x) corresponding to this which takes a value of 1 when a mission play needs to be discarded and 0 when no mission needs to be discarded. We hypothesize that likelihood of a participant's short (< 4mins) training being insucient (as dened as when a mission was discarded due to interruption by participant questions) is less than 20%. Thus, ifp is thep-parameter of the binomial distribution ofx, our null hypothesis and alternative hypothesis are as follows: H 0 :p 0:2, H a :p< 0:2. We perform a one-sided test for this hypothesis testing binomial distribution. For a 95% condence level, = 0:05, the critical value, BINOM:INV (n = 20;p = 0:2; = 0:05) = 1. In our study, we found that no mission run was discarded for any participant and so the number of participants with insucient training was zero which is less than the critical value of 1. Therefore, we can reject the null hypothesis. When we observe the participants' responses in the post-study survey about the understandability of the system, we again nd that all participants reported believing that the training was sucient. We can formulate this as a Bernoulli variable the same way as we did for the metric of discarded mission plays, and reject the null hypothesis using the subjective self-evaluation of the participants. 6.5.3 Discussions The key ndings in our human subject study demonstrate the possible merits of cognitive aids to human su- pervisors of multi-robot teams, specically cognitive aids in the form of alerts and decision suggestions. This is not surprising as a large number of studies have shown that in unaided conditions humans' performance suers when they need to process considerable amounts of information [147]. In this work, we only explored a particular set of robot tasks in the mission and some unique constraints relevant to military or HA/DR scenarios. However, we posit that such an alert framework can be useful in any human-supervised multi- robot operation that involves distantly located tasks, human-robot teams that are not always co-located, 119 probabilistic failures, limited communication, and uncertainty. Our method is contingent upon models that are available before or during the mission, which might be feasible in many domains by obtaining some eld experiment data or data from similar missions in the past. Our earlier works on alerts and suggestion generation provide some analysis of the quality of alerts and suggestions. This paper helps to gain insights into human performance while using our framework. As we did a two-hour study with young adults, it is dicult to say whether all participants were playing the missions sincerely the entire time. We observed a few participants making decisions too quickly with the alerts which raises the question of whether they were paying close attention. Future experiments could include exit questions about whether the participant paid close attention or could include built-in attention checks during the scenario. A user's trust in automation technologies was found to correlate with strong performance outcomes when using our alert framework. Since all our participants were engineering students pursuing graduate studies, it was expected that most of them would have reasonable trust in automation. In the post-study survey and discussion, there was a short but open-ended conversation with every participant. Several participants pointed out that they would appreciate it if there was more explanation or reasoning provided in addition to the alerts and suggestions. However, it is often dicult to provide additional reasoning as these missions have many things simultaneously happening, and have complex interactions between agents and events. It is possible in some cases, for example, when the system identies the possibility of a robot falling into danger in the near future and a relay task is recommended to help divert that robot from the danger, the reasoning is simple enough to be presented to the decision-maker along with the relay suggestion. On the contrary, when the automated system is internally assessing all probabilistic aspects, e.g., regional risks and robotic failure risks, for recommending a rescue operation, there is no straight-forward reasoning to provide to the user. Multiple participants also mentioned that they would like to see some data on the accuracy of the alerts (mission prediction) and quality of the suggestions. They believe that having access to the reliability and accuracy data of our alert system would help them make better decisions, whether it is to accept the system's recommendations or reject them. Limitations: User interface design is important but was not the focus of the work presented here. In this work, the user interface was developed to suciently provide the required functionalities only. One 120 problem that many users faced is that if someone mistakenly clicked and dragged on the background when they actually intended to click on an interactive task block, it would cause the viewing angle to change on RViz. Any time this mistake happened, the user needed to reset the viewing angle and then resume their task. While recording the time to make each decision for identifying delayed decisions, we deducted that extra time if it happened. Typically only a few seconds got wasted when this problem occurred for someone, but if we did not make this adjustment it could unfairly label some decisions to be delayed. We have veried all our hypotheses in our experiments; however, the ndings could have been dierent if the missions turned out to be very dierent from the mission models used for alert generation process. In an actual mission, there might be new contingencies occurring which were never encountered in the past missions or experiments. Humans might become so dependent on the alerts that they will not predict any new possibility which they could potentially identify if there were no alerts to rely upon. Cognitive overload can happen when humans are provided with too many alerts and suggestions. Observing cognitive load of the participants could provide valuable insights in this regard. In our experiments, only a limited number of alerts and suggestions appeared at a time. The DARPA SubT program is a good example of the current state of human-led multi-robot missions in communications-constrained environments [2]. We expect supervisors in actual missions to be mature adults with specialized training. Since these roles will require people to use the user interface and command robots, we expect them to be tech-savvy and have some basic understanding to work with robots. We expect them to have a certain level of trust in automation technology in general which would correlate to better performance. We used graduate students as surrogates for our intended users. Graduate students were familiar with robotics and tech-savvy. In our experiments, most participants also had a good level of trust in tech. However, there are certain inherent limitations in using a surrogate population in conducting this study. The actual mission commanders will be invested in the operational success whereas the graduate student participants were not because the mission was not real and it was not their actual job. 121 6.6 Summary We have conducted a human subject study to evaluate the eects of providing auto-generated alerts and suggestions on human decision-making while supervising challenging disaster-relief multi-robot missions. We have veried that such assistance can promote better and faster decision-making in terms of assigning tasks to robots, and improving mission performance. We have also demonstrated that an individual's trust in automation technology correlates with higher acceptance of automated suggestions and overall better mission performance. Lastly, we have shown that our alert framework technology can be used with minimal training overhead for the users. 122 Chapter 7 Alerts Seeking Human Help to Manage Plan Failure Risks in Semi-Autonomous Mobile Manipulation This chapter is based upon two published papers, [4,12]. 7.1 Introduction This chapter focuses on human-robot interactions (HRI) in semi-autonomous mobile manipulator systems. This chapter highlights the estimation of task failure risk, alert generation and risk visualization towards safe operation. Mobile manipulation systems have become very popular due to their versatility, precision, and ability to complete tasks safely. By combining one or more robotic manipulators with a mobile platform, mobile manipulation systems extend the working volume compared to xed-based manipulators and oer novel navigation-based capabilities. These systems, as a result, can perform a wide range of operations in a variety of applications { everything from industrial deployments [16,203] to medical, domestic and military or disaster response [80,97] settings. The unique benets of a mobile manipulation system do not come without technical challenges, however, and the suitable level of autonomy required in a system is mainly dependent on the application scenarios. Fully autonomous operations of mobile manipulation systems have been studied in many works [199, 205] , and have produced solutions that require humans to only specify high-level task goals while the robot plans the trajectory of the mobile base and manipulator to grasp objects using dierent articial intelligence (AI) 123 technologies. However, the inherent sensing uncertainty in semi-structured, unstructured, or cluttered envi- ronments complicates workspace modeling and introduces risk that could make fully autonomous operations infeasible or unsafe. For example, errors in the estimation of obstacles in the environment model can lead to collisions, which can damage the robot, its surroundings, and critical objects-of-interest that the robot is expected to interact with. One method to mitigate these risks is for a human to operate or supervise the mobile manipulator [12, 104, 197, 224] . Humans can observe a robot and its workspace, monitor progress, dene conditions and requirements for safe operations, detect and predict potential issues, intervene or cor- rect robot plans, and make context-dependent decisions that aect performance and eciency. It is likely, however, that the human's attention may be divided across multiple systems, necessitating clear, timely, and actionable information to enable ecient and eective contextual awareness. We posit that humans play an important role in some mobile manipulation applications, and systems need to be designed in a exible fashion so that human input can be leveraged when appropriate to improve, correct, or override autonomous operation. Properly designed mobile manipulation systems can then strike the critical balance of autonomous and manual control that maximizes the benets of autonomous agents performing metric and computationally-challenging tasks while humans perform higher-level decision-making and risk-mitigating tasks. In [12, 16], we present an example of semi-autonomous mobile manipulation systems that oer exibility by providing both autonomous and manual control modes to remotely-located human operator. An ecient mobile manipulation system can oer the human dierent tools for autonomous operations that reduce the human's workload and ensure safety and quality in task performance. To this end, mecha- nisms can be designed specically for automatically requesting assistance from the human in an intelligent way. The human should be notied judiciously of potentially unsafe or high risk tasks to avoid consequential failure, but not too often to avoid distracting or fatiguing the human. Automated alert generation is an eective means for soliciting human input, overcoming dicult situations, and managing risk. A suciently- intelligent, autonomous agent can plan trajectories for navigation, manipulation, and grasping as well as quantify uncertainty in its own actions. Given some set of human-dened alert conditions, an autonomous 124 agent should also be able to accurately analyze the current conditions using its state estimation and un- certainty quantication to rapidly and proactively alert the human if there is excessive risk in safety or performance in its planned actions. These alerts provide the human an opportunity to adjust risky plans, discard plans altogether, or suggest alternative actions. Alert generation also provides supplemental safety assurance in addition what is already integrated in many state-of-the-art motion and trajectory planners. Research Challenges and Contributions: In this paper, we investigate eective incorporation of human assistance for risk mitigation in semi-autonomous mobile manipulation systems, specically for the application of object transportation. We have identied three research challenges within this scope, proposed solutions to address each challenge, and developed a system architecture to unify these techniques, which we've implemented on hardware (Section 7.3). For the purposes of this work, we dene the risk metric to be the probability of task failure and task plan to be the manipulator trajectory plan, including opening and closing of the gripper on the end eector to complete manipulation tasks. The rst research challenge we explore is how to design exibility and extensibility in mobile manipulation systems. Humans should be able to naturally dene ad hoc alert conditions using their native verbal or written language and the system should accept these new conditions without any additional programming or refactoring. We propose using probabilistic temporal logic, specically Metric Temporal Logic (MTL), to mathematically encode the human-provided, alert-triggering conditions and failure modes to a machine- interpretable form that enables time- or task-stage-dependent computational analysis (Section 7.4). The second research challenge we address is how to rapidly and accurately generate alerts to maintain the operational tempo set by the human operator and ensure decision-making relevancy. Precise mobile manipulation in real-world environments can require redundant sensing and dense three-dimensional (3D) model building of the workspace in order for the robot to plan and act. We present computationally- ecient methods to estimate uncertainty in our 3D workspace models (Section 7.5), estimate the probability of collision between the robot and objects (Section 7.7), and estimate task failures using these models to generate alerts (Section 7.6), all on the order of seconds. Finally, we investigate how to improve human understanding of the mobile manipulation system and the automated alerts. The human-robot interface must be interactive to enable bi-directional information ow 125 as well as be comprehensible to facilitate the human's decision making. Information regarding the source and location of potential task failure for each task plan under consideration is useful for the human to evaluate risk as well as for the autonomous system to provide a form of transparency; however, the specic method for displaying this information requires careful consideration so that the human is not overwhelmed. We propose a wide range of visualization tools for the human supervisor to view and assess risk better (Sections 7.8 and 7.9). 7.2 Related Works In human-robot collaboration, risk is an ever-present factor that must be addressed. As such, a considerable portion of the existing literature is dedicated toward the identication, awareness, and planning to compen- sate for risks, largely stemming from collisions or other task-related hazards. While the research presented in this paper is focused on the estimation and communication of these risks to remote human operators for mobile manipulation, the literature is full of relevant research that warrants highlighting. Given the focus of this paper is on risk-based planning, related works for the topics of risk prediction, risk awareness, and risk-based planning are discussed in this section. A review of works relevant to human-assisted operation and teleoperation of robots of task execution is discussed in [12], and is not detailed here. 7.2.1 Risk Assessment At the heart of any robotic application is the expectation that there will exist some element of risk. Whether it is risk to the robot or equipment, risk to human operators in the environment, or risk to the underlying task performance, identifying and quantifying risks is a topic of investigation in and of itself. Generalized risk assessment methodologies for robotic applications (e.g., the Association for Advancing Automation (A3): Robotics technical report on risk assessments for industrial robots [174], or more research-oriented approaches for risk identication and mitigation [84,217]), walk through the process of identifying dierent sources and severities of hazards to humans, and can be based on risks posed by the equipment, the environment, or the task itself. Such approaches are rst-pass eorts to addressing risks before operations begin, and are generally leveraged to integrate engineering and administrative controls during the commissioning stage 126 of a robotic application. Approaches for dynamically allocating tasks or responsibilities in collaborative operations (e.g., [53, 130]), take advantage of risk assessments and prior experience to provide estimates of hazards to humans, and determine what actions should be taken over entirely by robots to reduce the risks to humans. Such approaches, however, are overwhelmingly based on a priori knowledge of the task, environment, and equipment. Even so-called \black swan" events{though based on extreme outliers on the probability spectrum{require anticipating hazards before they are encountered [18]. Indeed, even in the context of this paper, the focus on risk awareness is exclusively with regards to the robot colliding with objects in its work volume in tool tending and part manipulation tasks. To this end, the literature is lled with many dierent approaches for predicting, measuring, and avoiding collisions in applications ranging from robotics to graphics, and from naval navigation to missile guidance. See the review by Marvel and Bostelman [129] for an overview of some of the most prevalent means of assessing collisions across the myriad of domains. In terms of risk measurements, most collision-aware robot systems are with regards to collisions with humans (e.g., [118]). However, in the context of this paper, the focus is on assessing risk based on the likelihood of the robot colliding with its environment while planning and executing grasping operations. To this end, the literature is rich with dierent approaches to measuring and estimating risks. For example, in the work by Kozai and Hashimoto, a \picking risk" metric is introduced that denes inter-object relationships to provide estimates for grasping parameters to minimize the likelihood of colliding with objects during picking operations [105]. Other approaches assess reachability in cluttered environments (e.g., [3, 19]), collisions during manipulation of grasped objects (e.g., [154, 201]), and collision-based challenges of grasping moving objects (e.g., [127]). Once the risks have been identied and, presumably, quantied, motion planners can either generate and execute paths that minimize the risks (e.g., [223]), or{as is the case of this report{propose alternatives to the operator such that the human assumes responsibility of the risks. 7.2.2 Assessing Risk Awareness Once the hazards or risks have been identied, they must be brought to the attention of the human operators. Raising awareness of hazards is fairly simple and straight-forward. Knowing that an issue exists is only part 127 of the equation. The operator must also be made aware of the nature of the risk, the risk severity, and the means by which the risks can be resolved. As such, providing information in a way that is clearly understandable, actionable, and timely is a signicant issue [216]. While many robotic systems attempt to raise awareness of risks or systematic issues, many of the ap- proaches in the literature (as was demonstrated in Section 7.2.1) are focused on the identication of the sources and severity of risks. In many cases, researchers focus on directly measuring the situation awareness (SA) of operators in human-robot teams. Support and measures of real-time situation awareness is gener- ally dicult, and, in the literature, many attempts to measure situation awareness rely on physiological or directly-observable measures (e.g., [52,196]). Otherwise, researchers are limited to post-factor questionnaires regarding situation awareness (e.g., the work by Gervits et al in which human-robot team performance is leveraged against subjective surveys [64]) or operator condence in the interface and information quality (e.g., via the International Organization for Standardization (ISO) standard 25010 for software quality [85]). Still, other approaches focus on the evaluation of the system itself in terms of its ability to maintain sit- uation awareness. For example, the Risk SituatiOn Awareness Provision (RiskSOAP, [32]) methodology for assessing the performance of the dissemination of meaningful information regarding situation awareness of risks. In this case, RiskSOAP provides objective measures to evaluate how information is shared among inter- ested parties, and supports communication networks to enable and maintain distributed situation awareness of a robotic system's risks. Similarly, the research presented by Schaefer et al evaluates user displays in their capacity to share autonomous systems' task-relevant information to a variety of dierent stakeholders [180]. Schaefer's work focused on high-level assessments of SA for trust by leveraging the Situation awareness-based Agent Transparency (SAT) model by Chen et al [33], and is meaningful in that it provides an approach for measuring SA from the perspective of multiple operator roles. 7.3 System Architecture A typical mobile manipulator system [47, 207, 215, 225] has one or more robotic manipulators mounted on a mobile base, along with robot controllers, a planner, and a perception system with some sensors. In most applications, the robot is not fully autonomous, and some level of human supervision is present. In 128 that case, the mobile manipulator system also includes a human-robot interaction (HRI) interface which typically supports trivial tasking or teleoperation by humans [162, 212, 231]. Existing system architecture with mobile manipulator robot hardware and controller, perception system, planner, and a basic HRI unit, is not sucient to support risk-aware decision making by remotely located human operators [121]. In real world applications there is uncertainty and possibility of potential hazards during operation. Therefore, more assistance and guidance to human operators are required to enable human intervention towards better and safer task performance. Figure 7.1: The architecture of our mobile manipulator system. White boxes indicate hardware components, and the remaining things are software modules. Tan boxes refer to process blocks in our software suite and navy rounded- edged boxes are for data or information. Our main contribution lies in the four process blocks (tan-colored) within the Alert Generation System (light blue box). The gray-colored boxes use already-existing state-of-the-art technologies for our human-robot interface and generating robot motion plans. In the presence of uncertainty, humans need to know about potential risk associated with any task plan before executing the plan. Being in a remote location the person might not be able to perceive the possibility of certain hazards just from the perception data coming from the robot sensors. Therefore, we require the system to perform risk assessment, and provide a meaningful interaction with the human operator to enable risk-aware decision making. This calls for a novel system architecture with an independent module for risk assessment and communication with humans, which is agnostic of the rest of the system. The new module should be easily integratable with any mobile manipulator system with robot hardware and controller, any 129 perception system, any planner. Therefore, we propose the system architecture in Fig. 7.1. Our main contribution lies in the alert generation module (light blue rectangle) and its interaction with HRI unit to facilitate superior operation. Our goal is to assess the risk associated with any system-generated or operator- provided task plan (e.g., manipulator trajectory plan), issue an alert to the human operator when there is a risk, and eciently convey the risk information to the human. To achieve this, we have four process blocks in our alert module, which serve the following purposes { mathematical modelling of alert triggering conditions, estimating uncertainty in 3D model of the environment, computing failure or risk probability for any given plan and generating alerts when the computed risk is high, and providing visualization for humans to assess the risk themselves and make decisions. Humans provide necessary specications related to failure risks which are used to mathematically encode the alert conditions for plan risk as described in Section 7.4. Probability is required to represent the level of failure risk, and the failure conditions might depend heavily upon time. The framework should allow humans to add or remove alerts as and when needed without the need of programming. We choose probabilistic MTL formula to encode alert conditions for the failure condition with time, and the probability portion indicates indicates high (greater than a human-set value) probability of such task failure. We illustrate alert modelling using dierent sub-tasks for transportation of an object from one place to another. To generate an alert by estimating the probability of failure, we need to quantify the uncertainty in our measurements. Towards this, the point cloud data coming from the perception system is processed to generate an uncertainty model of the 3D workspace. We use the data coming from the red-green-blue plus depth (RGB-D) camera mounted on the manipulator end-eector, as the manipulator moves, and scans the environment. The reason for using the camera on the robot arm, instead of a stationary camera, is to get a better eld of view. We can move the camera around, and view the workspace from many dierent locations and angles, and generate a more complete model of the environment. Cameras at xed locations can only view very limited regions on the work-space. However, the moving camera and the uncertainty in manipulator joint angles might cause additional uncertainty to the 3D model of the environment on top of the generic error in the camera captures. We use multiple measurements to build the uncertainty model of the 3D environment. Uncertainty estimates are generated by performing multiple scans of the environment 130 from many dierent camera poses. This data is fused together and spatial discrepancy in the fused data is used to estimate the uncertainty in the model built by the system. This process in described in Section 7.5. The system should rapidly issue an alert to gain human attention in case of potential high risk. Whenever the planning system generates a plan for performing a task or human handcrafts a task plan, our alert generation module computes the risk associated with that plan before plan execution, and provide alerts when the risk is higher than the human-specied level. For any given plan, the alert generator uses the uncertainty model compute task failure probabilities, and checks for alerts, as depicted in Section 7.6. The estimation accuracy should be suciently high, and the computation time for alert generation should be relatively small in order to provide useful, real-time assistance. Finally, humans need to visualize and assess the plan risk which is auto-generated by the system, and make their own judgements before executing or abandoning a plan. Our plan risk visualizer provides an human-interpratable and interactive way for this visualization in the user interface which is described with details in Section 7.8. It shows potential failure modes, and related them to the source of uncertainty. The interaction between the alert generation module and the HRI unit needs to be intuitive and adjustable based on the operator's preference for complementing the human's operations. The operator needs to be able to receive alerts, change the alert condition settings on-the- y, and visualize plan risk. This visualization should have a variety of options so that the operator can customize the settings based on their preference and the situation for better perception. Based on the risk assessment provided by the system, we believe the operator should be able to provide suggestions to the planner or edit the plan directly to generate safer plans. Therefore, in addition to the usual features like perception data from sensors, task goals and robot commands, we have additional incoming and outgoing data blocks in our HRI module in Fig. 7.1. We implement our framework and conduct experiments using our mobile manipulator system, ADAMMS ( [12], [16]), which is designed to nd a balance between reducing the workload on remotely-located human operators, and ensuring safe task execution by the mobile manipulator in the presence of high uncertainty. AI technologies are used both for automating task planning and execution (when possible), and providing alerts and information to humans to make risk-informed decisions to accept, rene, or abandon system- generated plans. The hardware system (Fig. 7.2) of our robot consists of a dierential drive mobile robot, 131 Figure 7.2: Hardware system of ADAMMS 2.0 [12] with a Universal Robots UR5 1 robotic manipulator physically mounted on the chassis. A Robotiq 2-ngered gripper is attached to the tool ange of the manipulator. A multi-sensor suite is used to monitor the work volume, and consists of multiple depth cameras attached to both the mobile base and the manipulator, multiple color cameras (forward, backward, left, and right), a LiDAR (light detection and ranging) area scanner, a 9 degrees-of-freedom inertial measurement unit (IMU), and encoders on the mobile base's wheels. For localization of the mobile base, RTAB-Map [112] is used in Robot Operating System (ROS) to take advantage of sensor data from the Kinect and two-dimensional (2D) LiDAR for mapping and localization. The Kinect has 3D depth data, but this data is subject to signicant measurement uncertainty. In contrast, the 2D LiDAR provides a more accurate and larger range of depth data. By fusing data from both sensors, we observe enhanced mapping of the environment and robust feature detection. Also, improved odometry 1 Commercial equipment and materials are identied in order to adequately specify certain procedures. In no case does such identication imply recommendation or endorsement by the University of Southern California, the National Institute of Standards and Technology, or DEVCOM Army Research Laboratory, nor does it imply that the materials or equipment identied are necessarily the best available for the purpose. 132 estimates are provided as inputs to RTAB-Map by incorporating data from wheel encoders and an onboard IMU using an extended Kalman lter. There is an Intel RealSense camera on the end-eector which is used to generate 3D point cloud of the scene or workspace. Our planning modules are designed to leverage the existing, state-of-the-art motion planning techniques to execute tasks as instructed by the human operator. The operator tasks the system by providing task goals. These tasks could be either to scan the environment to generate 3D model of the workspace, manipulate an object or navigate to a dierent place. Motion planning for mobile manipulators can be done in two dierent ways. The mobile base and the manipulator can move concurrently ( [40, 90, 91, 93, 94, 204, 206]) or the mobile base can be held stationary at a specic location while the manipulator plans and moves for grasping and accompanying tasks. The mobile base motion for arriving at this location can be planned using several planners, in this work we use Dynamic Window Approach (DWA) [57] in ROS. The algorithm initially samples discretely in the robot's control space, which in our case is the velocity of the left and right wheels. These samples are ltered and rened to generate appropriate motions. Once the mobile base arrives at the appropriate position, the manipulator moves to a target conguration. There are several motion planners developed in literature for manipulators based on random sampling [115,166{168] and optimization [95,172]. In this work we use RRT-Connect [107] motion planner available in the Open Motion Planning Library (OMPL) [200]. The plan determined from growing two rapidly exploring random trees, one from the start and one from the goal conguration and connecting them is further smoothed to make sure the robot motion is smooth. For collision avoidance, the Flexible Collision Library (FCL) [153] is used with mesh-to-mesh collision detection. The goal conguration needs to be determined which can be a grasping pose or a pre- grasping pose for the gripper. The inverse kinematics solution for those gripper poses are determined which are then used as goal congurations. Since the planner randomly samples congurations, there is a risk of failure to complete the motion. Hence, there may be a need to re-plan several times before determining a successful motion plan. The uncertainty in the environment model also adds to the diculty of generating a safe plan. Therefore, a risk assessment tool is useful, regardless of the type of planner used, in order to make sure that plans are achievable. 133 7.4 Modelling Alert Conditions We want to model complex alert conditions mathematically in order to oer exibility and ease of use to the human operators. This way, humans would be able to dene new alert conditions that correspond to their risk preferences. The framework should be extensible, and allow humans to add or remove alerts easily, without the need of any programming. We require an appropriate language framework to provide the exibility to the human operator, and sucient expressibility to model risk based alerts. Dierent operations would require dierent failure modes or events which need to be captured with atomic conditions. Moreover, some assessment of risk level under uncertainty needs to be presented to humans to enable superior decision making. Alert conditions are to be expressed in an appropriately-chosen language framework. Applications with mobile manipulator involves the robot and/or other objects moving with time, and so a language with temporal properties is necessary to capture time-sensitive hazards. In many operations, certain failure mode can be sensitive to a specic time window. Therefore, MTL is a good option as an MTL formula can support particular time intervals. It oers the expressibility and exibility that our system requires for modelling dierent kinds of task failure modes. In the presence of uncertainty, probability is important for decision making as it provides a measure of assessed risk. Therefore, we choose probabilistic MTL framework to model alert conditions. [103] Human operators can set up alerts so that alerts are issued if there is a high probability of task failure for the existing level of uncertainty and a specic task execution plan. Humans can provide necessary parameter values or settings required for the alerts. For example, humans are to provide the probability threshold value for dening high probability for task failure (e.g., probability greater than 0.7 or 0.8) which triggers an alert. Moreover, humans need to use system parameters to devise MTL formula which dene task failures caused by dierent situations, e.g., colliding with obstacles while moving or missing an object during grasping. In the presence of uncertainty, the automatically-generated task plans might have some risk of failure. Alerts are geared to assist humans to assess the risk associated with any plan before authorizing execution. Consider the complex operation of transporting an object from one place to another using a mobile manipulator as an illustrative example for how modelling alert conditions can be done using our probabilistic 134 Figure 7.3: Sub-tasks in transporting objects from one place to another by a mobile manipulator. The labelling shows the 3D object/space representations used in expressing alert conditions. MTL framework. This entire task of transportation is broken down into seven sub-tasks as depicted in Fig. 7.3. After the robot has reached the object to pick, the rst sub-task is achiving a pre-grasp pose where the manipulator extends into the workspace and positions the end-eector in preparation for object grasping. The second sub-task is grasping by closing the gripper. The next sub-task is retrieval of the object where the arm retracts while holding the object, and the object gets released from the surface it was resting before it was picked up. Then the robot transports the object near its goal location for placement. The fth sub-task, placement of the object, is to move the manipulator such that the grasped object nally touches the surface where it is meant to be placed. The sixth sub-task is to open the gripper and the release the object, and nally the arms needs to be retracted as the seventh and nal sub-task. Each of these sub-tasks has dierent requirements to be successfully completed. It has dierent reasons or conditions to fail, therefore we encode failure condition for each sub-task with a unique MTL formula, i , wherei2 [1; 7][N is the sub-task index. Each of these formulae maps the operation state to a Boolean, such that it obtainsTrue value when the corresponding sub-task fails, andFalse otherwise. Alert triggering condition is expressed as a probabilistic temporal logic formula, P >p th , where 0 p th 1, and is an MTL formula referring to one sub-task failure or a combination of several failure modes. Thus, P >p th 135 indicates that the probability of being True, i.e., probability of failure, is greater than p th . Humans operators set this p th value for each alert condition according to their preferences. Success or failure of each sub-task in transportation application is dependent on the relative pose (location and orientation) and 3D geometry of dierent objects (e.g., robot, object, obstacles etc.), and how they change with time. We consider four major geometrical objects separately as each of them has a separate role in this operation. They are the robot, the gripper, the work-piece to be transported or manipulated, and obstacles which includes everything else in the workspace. We consider gripper separately from the robot since the gripper portion would actively manipulate the work-piece. Let R, G, O, W indicate the Cartesian space occupied by the mobile manipulator robot, gripper, obstacles, and the work-piece, respectively. These may change with time when their conguration change. Let us assume o s 2 O is the surface where W is to be picked from or placed on. X is the 3D space within the ngers of the gripper, which we denote as the grasping volume. If we have a multi-nger gripper, each nger sweeps through a maximum volume of space which can be denoted with X i . In our experiments, we use a two-nger parallel plate gripper, where each nger sweeps through a rectangular cuboid space which we indicate with X 1 and X 2 as shown in Fig. 7.3. All these items can be used to craft dierent failure conditions during the transportation operation to be done the robot. The 3D models for the robot and the gripper are available beforehand as provided by the manufacturers. These models and their pose and conguration provide R; G; X i at any given time during task execution. However, 3D models for the work-piece and workspace obstacles typically are not given, they need to be cap- tured by the robot before task planning. Then the system needs to segment the captured model using human operator's inputs (e.g., selection of the work-piece located in the workspace etc.), and identify W; O; o s , as needed for risk assessment. Once these models are registered, the system can generate robot motion plans, and estimate plan risk to generate alerts. The human operator can specify alert triggering conditions using these parameters and symbols beforehand, and can guide the system identifying the workspace related items (W; O; o s ) when the workspace model gets generated from the robot perception unit. The following enumerated sections will show how MTL can be used to model failure conditions during dierent steps of the operation. Here, A[ B indicates the total 3D space occupied by the two items, 136 A; B2fR; G; O; W; o s ; X 1 ; X 2 g. A\ B refers to the common space or surface occupied by both the items. It is non-empty only when the objects get into one another or they touch each other. Additionally, we assume that l A and q A are the location and orientation of object A. We can denote A as A(t) while denoting as a function of time if a formula requires. The following logic statements employ the operators nally in (, meaning \at some time the specied condition holds true") and globally in (, meaning \the specied condition holds true at all times"). 1. To pre-grasp pose: In this stage, the robot or gripper cannot collide or come in contact with the workspace obstacles or the work-piece of object to be manipulated. Failure occurs in case of collision. 1 : (R[ G)\ (O[ W)6=; 2. Grasping: For successful grasping, it needs to grasp the object within it's ngers. Before closing the gripper, we need to check whether some part of the work-piece is within the grasping volumes, so that the gripper does not just grasp air. Grasping air would be a failure condition which requires an alert. If toc is time of closing the gripper, this failure mode can be expressed as the following formula where T is logical True. 2 :: ( ( (W\ (X 1 [ X 2 )6=;) U ttoc T ) 3. Retrieval: A successful retrieval requires that the work-piece is released from the workspace surface (o s ) as the robot grasps and picks it up, otherwise it is a failure. 3a : W\ o s 6=; If failure mode from 3a does not happen, then failure can still happen from collision after release from the surface at t ==tor (time of release). In addition, the work-piece may need to be held and kept at a certain range of orientation (q 0 ) at all time. 137 3b : ttor (R[ G[ W)\ O6=; 3c :q W 6q 0 4. Transportation: As the robot navigates to the goal location for transporting the work-piece, collision must be avoided, and the orientation constraint should be met at all times. 4 : ( (R[ G[ W)\ O6=; )^ (q W 6q 0 ) 5. Placement: Collision and orientation condition should remain the same until the object is placed at the destination site. 5 : ( (R[ G[ W)\ O6=; )^ (q W 6q 0 ) 6. Release: The object needs to be released upon placement by opening the gripper. As the gripper opens it should avoid collision with the surrounding obstacles. 6 : (R[ G)\ (O[ W)6=; 7. Retraction: As the mobile manipulator retracts its arm, collision must be avoided. 7 : (R[ G)\ (O[ W)6=; This example with transportation task illustrates how alert conditions can be modelled with probabilistic MTL formula, by dividing complex task into sub-tasks, dening the required items and using them to craft failure/risk conditions. The process can be extended towards other applications involving mobile manipu- lators. Our example application includes commonly used operations like robot motion, object handling etc. We have expressed one of the most frequent failure modes collision, as well as some failure modes related to 138 object grasping and transportation. This demonstrates usability and potential of our approach for a wide variety of applications, hence making this approach exible and extensive. 7.5 Uncertainty Estimation in Environment Model Uncertainty in robot perception of the environment is one of the major challenges incurred in any application, especially in case of robotic manipulation. There is always some uncertainty in the data generated from the perception unit of the robot which can cause task failures. We estimate the uncertainty in the environment model coming from the perception sensor of the robot which we use later to estimate failure probability for alert generation. There have been many works on quantifying uncertainty in data from depth sensors like Lidar, Kinect etc. [155,213]. However, we needed a modular framework which can support a wide variety of sensors integrated with any mobile manipulator. Our previous works [12,16] highlights problems of using 3D data from sensor(s) located at the xed location(s). The major drawback is the lack of visibility from static sensor(s), which might unnecessarily shrink the free area around the workspace, make motion planning more challenging, and cause hazardous situations. We propose attaching a depth sensor near the end-eector of the mobile manipulator so that the robot can capture depth images from many dierent locations and viewing angles by moving it's arm. This approach signicantly improves the visibility of the work-space compared to a stationary sensor which can facilitate safer manipulation tasks. For such sensor setup, we have developed a sensor-agnostic method for estimating an uncertainty model of 3D environment which makes use of multiple measurements [39], and data comparison and fusion. The RGB-D camera mounted near the manipulator end-eector captures the workspace area of the robot while the manipulator moves and goes through a sequence of locations and orientation. Fig. 7.4 shows an image taken during our physical experiments as the manipulator was doing a scanning motion from the front of the workspace to generate the 3D model. We analyze multiple measurements for the 3D model of the environment to estimate the uncertainty. We aim to represent the 3D workspace environment using a nominal point cloud and an uncertainty map that maps the location uncertainty of each point in the nominal point cloud. We consider location uncertainty only along the surface normal direction at each point. We make this assumption because the uncertainty along normal direction can in ate or de ate the object as a 139 Figure 7.4: Robot scanning an environment to generate 3D uncertainty model using the RGB-D camera (RealSense D415) mounted at the end-eector. whole, which oers enough possibilities to check for our failure modes like collision. In our case, we do not need to distinct between dierent points on the surface of an object, rather we only require the overall 3D space occupied by it. Therefore, assuming uncertainty only along surface normal direction is reasonable. The uncertainty in the 3D point cloud model of the environment is contributed by the inherent inaccuracy present in most depth sensors and also the errors propagated in the robot links. As the robot executes its trajectory plan for scanning the workspace, the point clouds are captured continuously at 30 frames per second. We perform ltering in the incoming point clouds using the camera viewing angle and distance. We use these ltered clouds and the manipulator trajectory information to transform and fuse the point clouds, and generate the nominal point cloud. Due to the high bitrate nature of the incoming clouds we use specialized data structures to iteratively calculate the uncertainty. This method reduces the error in the nominal point cloud and the uncertainty model because of the large number of point clouds collected over time. The uncertainty in depth measurements of an RGB-D camera increases with the distance between the object and the sensor. RGB-D cameras also have minimum working distances due to the reliance on the disparity between features, which negatively impacts their eectiveness at identifying and measuring close 140 objects. Moreover, the data in a depth sensor has asymmetric error distributions with primary directions along the sensor lines of sight. We dene the sweet spot of a depth sensor as a cylindrical frustum created using the above points and an incoming 3D-point is said to lie in the sweet spot when it lies in this cylindrical frustum. In our method, we lter out those points from the incoming point clouds, that lie outside this frustum. Each ltered point cloud is then applied a transformation based on the corresponding manipulator pose, which brings the point cloud in the robot base coordinate frame. The ltered and transformed point clouds are used fused into generating the nominal point cloud as well as the associated uncertainty model. The camera uncertainty is present along the line connecting the origin of the camera origin and the point being viewed [41]. This is because the depth values are obtained by projecting an infrared pattern on the sur- face and measuring distortion. The uncertainty in determining the re ected ray results in range uncertainty along line of sight. In addition, since we create the point clouds of the environment by fusing measurements taken from various locations facing the work space, the errors in the robotic links are also propagated. We are interested to quantify the uncertainty in points on a surface along the surface normal directions. Since we require uncertainty along one particular direction, a narrow cylindrical geometry (cylinder axis aligned to the normal direction) around a nominal point can be suitable to capture its uncertainty in space along the line. We need to consider all points coming from all the ltered and transformed point clouds that lies within the cylinder while computing this uncertainty. Figure 7.5: Use of voxel based representation. (a) Occupancy grid of the environment. Green voxels are occupied and white voxels are empty. (b) To calculate the normal vector for a voxel, we nd it's plane parameters using PCA on neighboring points. (c) The eigen vector corresponding to the highest eigen value is used as the principal axis (red colored line) for the cylinder. Let, we are calculating the cylinder geometry for the blue voxel. We hash both the empty and occupied voxels that are lying inside the cylinder. 141 Figure 7.6: Estimation of each nominal point location along with its uncertainty level, using points inside the hypothetical cylinder (with principal axis along the normal direction of the voxel). (a) All points lying inside the cylinder is used to estimate the uncertainty model. (b) The points are projected on the normal vector, and the average location of them is considered as the nominal point, indicated by a green-colored symbol. (c) The standard deviation among the lengths of the blue-colored dashed lines provides of the zero-mean Gaussian distribution for uncertainty of the nominal point along the normal direction (red line). This is the uncertainty metric for the nominal point (green). Computational time is a big challenge in handling a large number of point clouds and generating an uncertainty model. We discretize the workspace into an occupancy grid, which makes it computationally easy to keep track of the point clouds being captured, ltered and transformed. To get the normal direction for a voxel V i , we rst estimate the plane parameters of the voxels mid point C i . We perform Pricipal Component Analysis (PCA) for the points in the local neighborhood, and use the eigenvector associated with the greatest eigenvalue as the normal vector or normal direction. Since we are doing this in an occupancy grid, the computation is very fast. We assume a hypothetical cylinder along this vector as the principal axis, centered around C i . The value of radius and height (h) of this cylinder are taken appropriately based on the desired resolution and the camera standard deviation along the depth direction of the camera. All the points that lie within the cylinder are then projected onto the principal axis and then averaged to get the centroid of the points in voxelV i . The distances of these projected points from the newly computed centroid, are then used to calculate the uncertainty. We assume zero-mean Gaussian distribution for uncertainty, and so we simply compute standard deviation of those distances for parameter of the Gaussian distribution. This process is depicted in Fig. 7.5 and 7.6. To make the overall pipeline fast, we evaluate this lazily once there is sucient points in the occupancy grid and after the normal direction is calculated, the keys for voxels along the positive and negative direction 142 of the normal centered atC i and at a distanceh=2 are cached for faster update. After the initial estimate of the centroid and the standard deviation, new readings are incorporated using the Welford's online algorithm for calculating variance. c k+1 i =c k i + (x j i c k i )=n (7.1) s k+1 i = ((x j i c k+1 i ) (x j i c k i )s k i )=n (7.2) n =n + 1 (7.3) Here,k is the iteration number, c i ands i is the centroid and variance that corresponds to voxel V i . n is the number of points that are inside the cylinder generated for V i . To do this process in real time without incurring a heavy memory requirement, we avoided using a dense or hierarchical data structure; instead chose a hashmap to store the occupied voxels. This eciently exploits the fact that most of the space in a regular voxel grid is either unobserved space or free space. Each voxel V i stores the centroid of the points along the surface normal, the uncertainty measurement, a list of voxels whose uncertainty measurement depends on V i and a buer to store unprocessed points in the voxel if we do not have sucient neighborhood voxels to calculate the normal. The updates are stopped when the scanning motion of the manipulator stops, and all the captured point clouds have been used. Finally, a point cloud (nominal) with meta data for uncertainty estimates is generated using the data for all the occupied voxels. We have veried the applicability and feasibility of our method by conducting some physical experiments. We used lawn-mowing patterns for scanning the environment from a vertical plane in front of the workspace. The data captured at 50 points along the path were used to generate the uncertainty 3D point cloud model of the environment. The point cloud fusion, comparison, and uncertainty model generation was completed within a few minutes. Fig. 7.7 shows the uncertainty model we generated by scanning the environment in Fig. 7.4. The gure shows nominal point cloud, and the color of each point shows its uncertainty level along the normal direction. The uncertainty parameter is , where is the standard deviation of Gaussian distribution. So, the actual location of each point is expected to lie within3 to 3 range along the surface normal direction, with 143 Figure 7.7: Representation of the uncertainty model generated for the scene in Fig. 7.4 during our experiments. The point cloud shows nominal point locations, and green-to-red color scheme shows the uncertainty level in each point along its normal direction. Greener points indicate lower uncertainty, and redder points higher. respect to the nominal location. Thus the spread of a point is 6. In our experiment we found the maximum to be 0:33 cm which gives the spread to be nearly 2 cm. However, the average spread across the scene was found to be around 1:5 cm. Fig. 7.7 also shows how the uncertainty varied across the scene. We found that nearer points have lower uncertainty than the points further away from the robot. This is expected according to the camera specication, since the nearer points are at the ideal distance from the camera, and increasing distance results in additional error. Another thing we observed is the change in uncertainty with respect to the surface normal direction of the points. The depth information (hence, location) of a point is expected to be more accurate when the surface normal is parallel to the camera's viewing direction. As Fig. 7.4 shows, our scanning motion mostly used forward-facing, but slightly tilted in the downward direction. Therefore, the surfaces that are vertical with respect to the ground, facing the robot, had the minimum uncertainty. Conversely, the surface along the ground (like the table's top surface) has higher level of uncertainty. The approach we have described for our uncertainty model generator module is sensor-agnostic and modular, as required by our system architecture. Our physical experiments demonstrate feasibility in terms 144 of application and computational time, and intuitive correctness in estimating the uncertainty model of the environment. Our alert generator module uses this uncertainty model to assess risk for executing any task plan under the uncertainty, and generates alerts as needed. 7.6 Estimating Task Failure and Alert Generation The alert generation module requires to estimate the probability of task failure as expressed by MTL formulae in our framework, and then issue alerts to human operators when the probability is higher than the specied level in an alert condition. The failure modes specied in the illustrative example in Section 7.4 for alert generation are based on the 3D spaces occupied dierent items (i.e., the robot, the gripper, the work-piece, and any obstacles), and the temporal changes in their poses. A task plan provides a sequence of robot poses{along with the grasped object after a successful grasping{with respect to time. Our system needs to compute the probability of failure for a particular plan, in order to check for the alert conditions to be true. The uncertainty model of the work-space in 3D (Section 7.5), generated by our system, is used in this probability computation. In general, people would prefer to execute plans where the risk level (or failure probability) is minimal, despite some unavoidable uncertainty in the work-space model and gripper pose. Based on the operator-specied alert triggering risk level, an alert is generated for a plan if the estimated risk exceeds the threshold. We specied three kinds of alert conditions for the sub-tasks of the object transport operation have associated failure modes. Let A and B indicate the 3D space occupied by any two objects, with associated orientations,q A andq B . W and X represent the 3D space of the object to be grasped and the grasping volume respectively. The three failure modes include: 1) collision (A[ B6=;), 2) missed grasping (W[ X =;), and 3) object orientation not maintained (q A 6q) 7.6.1 Collision Probability for a Task Plan Collision is the most common reason for task failure for almost any operation with a mobile manipulator. In our example, we have specied collision based failure mode for which probability of collision between two objects needs to be estimated when one object is moving. We assume, both the 3D models have uncertainty 145 which can contribute to a non-zero probability of collision during executing the motion plan, even when the nominal 3D models will never collide. Algorithm 5 Collision Probability Estimation between Two Objects, with One Object Moving 1: A: the moving object, B: the stationary object 2: : motion plan for A, where (t) = pose of A at time t, t2 [0;T ] 3: A;B : nominal point clouds of A, B, with associated uncertainty models 4: dm: maximum possible Cartesian displacement of A per unit time 5: : minimum distance between the nominal point clouds which ensures zero probability of collision despite the uncertainty in the 3D models 6: function Plan Collision Probability(A,B, , dm) 7: p 0 8: t 0 9: while tT do 10: A 0 transform (A;(t)) 11: B 0 portion of B within the region of interest around A 0 12: d minimum distance between A 0 ;B 13: if d then 14: p 0 Point Cloud Collision Probability(A 0 ;B 0 ) 15: if p 0:50 0.082 0.157 0.212 0.260 14.1% 22.54% 38.4% 49.6% 155 Table 7.2: In case of low-P2P-collision-probability randomly generated 1000 cases, failure in detecting any possible collision for using a limited number of samples (dierentN values when usingN 2 Monte-Carlo samples for uncertainty) Collision Percent of cases - failure to estimate Probability collision probability to be non-zero Range N = 300 N = 100 N = 50 N = 30 0:00 0:01 23.2% 44.0% 60.7% 73.5% 0:01 0:02 0.1% 4.8% 10.6% 23.2% 0:02 0:03 0.0% 0.9% 5.4% 12.5% In order to test feasibility of our method in terms of computational time, we tested a few cases in MATLAB. The moving and stationary objects were the gripper and the scene respectively, and we had suitable uncertainty models for the point clouds of both the objects. Uncertainty in gripper may arise from the error in manipulator joint angles, issues with the gripper attachment, or jerking related problems in mobile base or manipulator base. We used down-sampled point clouds where the distance between points is in the range of 3 5 mm. Every time a particular way-point of the gripper trajectory is tested, we crop out the scene points that lie within the region of interest based on the gripper pose. In our experiments, number of points in the down-sampled point-clouds of the gripper and the scene were between 1; 500 2; 500 and 30; 000 40; 000 respectively. Cropping of the scene point cloud (line 11 in Algorithm 5) reduced the size of the point cloud by at least 82%, bringing the number of points in the scene point cloud down to the range of 0 7; 000. For cases with moderate collision probability in our experiments, the number of points in the cropped scene point cloud was around 2; 000 4; 000, and even less negligible for lower collision risk cases. If people use any state-of-the-art manipulator planner to generate a plan, we do not expect the plan to have too high probability of collision. So, the above-mentioned size of the point clouds from out experiments should deem reasonable as we test for computational time of our approach. In our experiments using MATLAB on Intel Xeon CPU Quad-Core E5-1620 3.6 GHz processor with NVIDIA Quadro 600 and 16GB RAM, we found that the computational time of line 10 in Algorithm 5 is only a couple of milliseconds, and lines 11, 12 are done in tens of milliseconds. The main computational expense incurs in Point Cloud Collision Probability function call. k-NN-search (line 2 in algorithm 6) is done once, and it takes lower tens of milliseconds. The main challenge is performing P2P Collision 156 Probability for every pair of points found. Since all points might not have the same uncertainty level, therefore only computing the probability for the point pair with shortest distance does not work, computing probability has to be done for all the pairs. Our method for P2P collision probability computation requires sampling of one point, and use analytical formula for Gaussian distribution for the other point. To make the process simpler, one could do sampling for both the points, and perform deterministic collision check like Fig. 7.10, and compile the results for probability of collision. Let's denote our method as method-1, and the alternative methods as method- 2. Let's assume we are using N samples of each point's uncertainty in location. Our method requires N computations, whereas the alternative method would require NN orN 2 computations. Let's assume the time required for probability of P2P collision is t p , and computational time for each deterministic collision check ist d . So, the total time required for P2P collision probability computation using method-1 ist 1 = 2Nt p , and method-2 is t 2 = 2N 2 t d +. Here, is the time required for combining the deterministic results, and 2 in both formulas come from interchanging the points for putting the penetration cylinder. Clearly, the deterministic computation is extremely simple and fast, and therefore t d < t p . However, if Nt p t d , then t 1 <t 2 , and so method-1 will be faster for P2P collision probability computation. We found from our experiments that t p 40t d . So, for any N > 40, our method will be computationally faster. In case of sampling from a probability distribution, there is a high risk of inaccuracy in estimation. Using larger number of samples provides higher condence in the computed result, but that increases computational time. Therefore, the choice of N is crucial to ensure estimation quality. Since, our method uses sampling for only one point, instead of two in method-2, our method provides at least the same level of condence as method-2 for the same value of N. In order to get insights on choosing an appropriate value ofN, we perform some experiments for method-2 with a large number of randomly generated cases with point-pairs. For each case, we generate the locations of the points in a pair, and their normal directions. Some cases have high probability of collision, some have very low or zero. We accumulate the ndings from 1000 cases for the results presented in each of the rows in Table 7.1 and Table 7.2. We assumed thatN = 3000 provides 100% condence in estimating the probability of collision, which are denoted in column-1 in both the tables as the base values. When we use smaller N 157 values (N = 30; 50; 100; 300) for the same cases, clearly the computed probability of collision deviates from the base value found using N = 3000. Certainly one would like to go as close as possible to the base values to ensure high accuracy in estimation. The allowable error or inaccuracy level in probability estimation can be dierent for high probability cases versus low probability cases. Therefore, we separate cases for dierent ranges of probability in dierent rows in Table 7.1. We present the maximum error found among 1000 random cases, as well as the maximum percent error with respect to the base probability. It is clear that N 100 provides very high maximum error, hence unacceptable. N = 300 still incurs inaccuracy, but the level might be acceptable, depending on the situation. Table 7.1 shows that even N = 300 can still have an error of up to 0:08 in estimating a probability with base value higher than 0:50. So, clearly, N 300 should be used to perform good quality probability estimation. Since we are focusing on collision probability in our application, it is especially important to compute a non-zero probability of collision, even for cases with very low probability. In many cases, humans may want to get alerted for any plan with a non-zero probability of collision, or plans with collision probability greater than 0:05 or 0:01. Table 7.2 presents our ndings regarding collision detection for low collision probability ( :03) cases. The rst row shows that for cases with probability of collision less than :01, method-2 with N = 30 fails to detect any probability of collision for more than 22% of the cases. Even N = 100 fails more than 3% of the times, but N = 300 seems performing well. As we look into cases with slightly higher collision probability (0:01 0:03), we can see failure in detecting any collision decreases for all values of N. Please note that this table does not represent the accuracy level in probability estimates. Clearly, this table also shows thatN 100 can not be used since that could misrepresent the risk level to humans, and results in unsafe robot operations. Since our method will be computationally faster for any N > 40, and here we need to use N 100, preferably N 300, to generate reliable results, we can conclude that our method outperforms N 2 method in terms of performing superior estimation in less time. In our experiments using MATLAB, we managed to compute probability of collision for risky task plans in less than 10 seconds every time. If we use C++ 158 or more computationally ecient platform, we can certainty perform with even higher accuracy and less computational time. Thus, our method meets the requirement of generating alerts within seconds. Our collision probability estimation method is feasible in terms of computational time and quality of estimation. Using this method, collision probability for a task plan can be estimated which aids in generating alerts for one of the most common failure modes in mobile manipulator applications. 7.8 Visualizing Plan Risk Typically, a human-robot-interaction interface for a mobile manipulator provides visualization of the envi- ronment model (without uncertainty), some task goal or plan selection capability, and animation of a motion plan. However, we require more sophisticated visualization to support risk-aware decision making by human operators so that the data from our risk assessment and alert generation module can facilitate better perfor- mance. We need to graphically show potential failure modes and relate them to the source of uncertainty. We have added several features and settings in our system that can support operator's individual preference as well as situation-specic need, to improve human perception of plan risk. A plan visualizer is provided to show animations of the robot executing a particular plan selected by the operator before executing it. People can use any state-of-the-art planner to generate plans for a given goal. The operator can select a particular plan for which our system calculates the risk associated with the plan, generate alerts, and let the human see how the plan execution will look like. We use rviz, the 3D graphical interface of ROS, for this visualization. If the plan has some risk associated with it, we need to graphically represent (1) failure mode and risk level (probability), (2) uncertainty in the data involved in the failure, (3) role of the uncertainty in triggering failures. All these metrics need to be covered for ensuring good understanding of the risk by human operators. Each task plan includes the robot motion for perform the task (or sub-task). The manipulator motion is typically the most challenging since robot needs to avoid colliding with dierent obstacles in the workspace while physically interacting with certain objects, like grasping and retrieving an object or placing it. When the robot operator gives a particular goal, the manipulator planner generates one or more plans to execute 159 Figure 7.13: An instance in visualizing manipulator plan for grasping an object, where the green colored arm indicates the goal pose. [12] the task. The person can select a particular plan, and simulate and visualize the motion. Fig. 7.13 shows an instance of the animation of a manipulator motion to grasp an object. Each sub-task as dened in Section 7.4 has a dierent condition for task failure. Whenever the operator visualizes a plan, our alert generation module computes probabilistic risk and issues an alert when needed. In the animation of plan with an alert triggered, a yellow bubble appears around the region of failure when the failure occurs like shown in Fig. 7.14. Failure type and risk level are also provided to the human operator. As collision is perhaps the most common failure mode, we will illustrate dierent features that ensures eective graphical representation for collision based failures. Collision Visualization Settings: Collision probability is computed based on the robot motion plan and the uncertainty model of the scene and the gripper. If the estimated collision probability is greater than threshold value specied in the alert condition, an alert needs to be given. While or even before simulating a given plan, a pop-up warning message (Fig. 7.14) can be useful to attract the operator's attention. Ideally, the message should be simple, and easily interpretable. An alert sound along with the alert box can be even more eective. 160 Figure 7.14: Yellow bubble and red points are indicating collision between the gripper and the wall of 3D printer. An alert message pops up to indicate high risk (probability) of failure for a given task plan. An alert sound also gets played along with the pop-up message to attract human attention. The alert message pop-up indicates the failure mode. However, the operator needs to know the estimated risk level (here, collision probability), as well as, when, how and why the failure might be occurring. As collision can happen due to the uncertainty in environment and gripper model, there might be a non-zero probability of collision even if the nominal point clouds of the scene and gripper are not colliding. Alerts triggered in such a situation can be confusing to the operator where a plan that looks safe (during visualizing the plan) might still have a small likelihood for collision, depending on the uncertainty level. Therefore, it is extremely important to provide dierent features to visualize the collision state and assess the situation better. From the visualization of a plan execution, the operator can pause at any instance and adjust visualization settings for collision. We provide dierent features that can be enabled or disabled while visualizing a static collision state which are described and illustrated in this sub-section. 161 Figure 7.15: Brighter red colored are the points with higher probability of collision, and darker reds have lower risk. Figure 7.16: Two dierent colors, green and blue, to visualize collision region in the gripper and obstacles, respectively. • Yellow bubble: A yellow bubble or sphere enclosing the collision region of both objects is particularly useful during the animation. Even during a static state inspection, it still helps to draw attention to the colliding region as shown in Fig. 7.14. • Color gradient on points in collision: Coloring the points in collision (non-zero collision probability) with a bright color, like red, can be useful. If the colliding regions in any object is relatively large, the operator may also want to dierentiate between highly probable region and low probable region 162 Figure 7.17: Choosing the right color scheme to visualize collision region is important. (a) If one color is used which is already dominant in one of the objects, it loses clarity. (b) If one non-dominant color is used, but the colliding objects are too close, its becomes hard to distinguish between the two objects. (c) Using two dierent colors on the objects, and using non-dominant color can provide better clarity. Figure 7.18: 3 arrows, along surface normal direction, extending from the collision region helps to show the extent of location uncertainty of the points. for collision. Color gradient is a great way to achieve that. In Fig. 7.15 brighter red stands for higher probability of collision whereas darker red color is for lower probability. / • Dierent colors in two objects: The operator may prefer two dierent colors for the two objects in collision like shown in Fig. 7.16. This might be particularly useful when the nominal point clouds are too close to each other. The left two images in Fig. 7.17 show that it is very dicult to inspect if we use the same color when the objects are nearly touching each other, whereas the right-most image shows some distinction. 163 Figure 7.19: In ation of point clouds: (a) with no in ation, (b) in ation of all points by • Using non-dominant colors: If the color used to highlight the points in collision is already dominant on one of the objects, then the operator would not be able to distinguish the region of collision properly (Fig. 7.17 (a)). Using colors that are not dominant on the objects is important to ensure clarity as illustrated in Fig. 7.17. • Hiding non-collision objects: Some time it might be useful to hide all other objects in the scene which are not in collision. Other objects in the background may hamper the inspection. • 3 arrows from region of collision: When we are visualizing the region in collision with the nominal point clouds, it seems unclear how far these colored points can extend outwards from the nominal surface. Since the points can go 3 inwards or outwards, we can use some outward arrows of length 3 to show the maximum extent. We use sparsely spaced arrows, instead of adding arrows to every point, to ensure clarity. The operator can enable or disable the arrows of each object individually as enabling both of them might look confusing as illustrated in Fig. 7.18. This feature is particularly useful when there is some gap between the nominal point clouds to see the extent the uncertainty in collision regions. If the gap is too small, the arrows will go into the other object, and so visually would not be very helpful. • In ating point clouds: Another way to show the extent of the uncertainty of the points is by in ating the object by; 2 or 3. In ation by 3 state will denitely show touch or intrusion of one object into 164 other, for a state with a non-zero probability of collision. With increasing in ation, the operator can view how the distance between the objects are decreasing, and that might provide a good understanding of the situation. Fig. 7.19 shows an example of how in ation by makes the objects look closer to each other. In ating also helps only when there is a visible gap in nominal point clouds. Otherwise, the two will just penetrate each other in in ated forms. These features ensure that all three visualization metrics are supported in our system in case of collision. The yellow bubble attracts attention on where the failure mode is occurring, and failure type and risk level are provided to human operators. In ating point clouds shows the uncertainty level in the colliding objects. All other features or settings help in understanding the role of the uncertainty in triggering failures. 7.9 Results Figure 7.20: The scenes we used in our experiments for grasping and retrieving an object. The objects to be grasped are marked with the green arrows. We have conducted some physical experiments with our mobile manipulator to test dierent modules of our alert generation framework. We mainly performed grasping and retrieval of objects during these experiments. We used two dierent cluttered environments, shown in Fig. 7.20. Our robot picked up the hand sanitizer bottle from scene-1, and a 3D-printed red object from scene-2. 165 For each scene, rst, the manipulator did a scanning motion from the front of the workspace to generate 3D model of the environment (Fig. 7.4). The RealSense RGB-D camera mounted on the manipulator end- eector captured 50 point clouds from dierent locations and orientations, which are ltered and then fused together. Our uncertainty model generator analysed the data to get a nominal point cloud, and uncertainty model for each scene. Fig. 7.7 illustrates the uncertainty model we computed for scene-1 with a green-to-red color gradient scheme. Each point has a dierent level of uncertainty which can be recognized by its color in the gure. Figure 7.21: Robot executing a safe plan for retrieving an object from inside the 3D printer in scene-1. We used the generated uncertainty model for each scene to compute plan risk and generate alerts for dierent task-plans. We experimented with multiple task-plans for executing sub-tasks 1 3, i.e., going to pre-grasp pose, grasping the object, and then retrieval of the object. We took three manipulator motion plans for each scene, one very safe plan, one plan with extremely small probability of collision, and one very risky plan. Safe plans for both scenes resulted in zero probability of failure in our risk calculation. We executed the safe plan on the actual robot and the robot performed the grasped and retrieved the object perfectly (Fig. 7.21). For the second category of task-plan for each scene, our alert generation system calculated 166 probability of collision to be below 0:012. Even though the probability of collision was greater than zero, since they were very small numbers, we inspected the plans' risk in our visualizer, and then executed the plans on the robot. The robot gripper went very close to an obstacle, but did not cause a collision. Lastly, we had one very bad plan for each scene, for which our alert generation module computed probability of collision to be 0.6, 0.43. In this case, we only observed the task execution animation and static collision states in our visualizer. 7.10 Summary A model-based, risk identication and management framework is presented in which dierent hazards, such as collision, in semi-autonomous mobile manipulator tasks are addressed. These models, based on volumetric sensing of the robot's work volume, are analysed using an MTL assessment of the mobile manipulator's sub- tasks, and leveraged to provide timely, actionable information to a remote operator. We have proposed a modular architecture which allows integration of our alert generation module with any mobile manipulator hardware and software suite. We have demonstrated how risk based alert triggering conditions can be expressed in probabilistic MTL framework. We have presented a sensor-agnostic way of generating uncertainty model of the 3D environment. We have developed a computationally ecient method for estimating probability of collision (and other failure mode) using uncertainty estimate of 3D models of moving objects, given any task plan. Finally, we have described how alerts can be generated based on estimated failure probabilities and human-specied alert conditions, and how risk related information can be graphically presented to human operators eectively. We have veried feasibility of our approach for alert generation by conducting some physical experiments. 167 Chapter 8 Conclusions This chapter presents the expected intellectual contributions and anticipated benets from the work proposed in this dissertation, and future research directions. 8.1 Intellectual Contributions The dissertation makes the following contributions toward automated generation of alerts to improve human decision-making where humans are in the supervisory role in robotic operations. • This dissertation proposes an alert generation framework where alert conditions are formulated mathe- matically. The framework is exible and extensible and is modular in nature so that it can be integrated in case of any application with human-robot teaming. • It describes computational techniques in discrete event simulation paradigm which allows a large num- ber of large-scale multi-robot mission simulations to be completed in a short amount of time. The approach ensures computational eciency while maintaining mission prediction accuracy. This way, mission prediction-based alerts are generated in a timely manner which is necessary for many time- critical and challenging mission scenarios. • This document presents a preliminary assessment of prediction-based alerts with ndings from a human subject study. It demonstrates how alerts can help humans identify potential contingencies better, 168 which is often dicult or not possible without alerts. Alert are shown to help in improving decision- making in terms of addressing contingencies. • The dissertation also describes a robot-tasking suggestion generation approach that aims to handle the potential mission contingencies and improve performance. The novel heuristics-based approach aims to promote faster mission completion while incorporating task criticality and incomplete tasks. It also promotes information propagation in missions with constrained communication and makes suggestions after analyzing the risks and benets of certain probabilistic actions. The approach has been demonstrated as suciently fast in computation and enhancing mission performance compared to baseline approaches. • Alerts are meant to get a user's attention to important or severely-consequential matters, and sugges- tions are the next stage in cognitive aids which can directly recommend decisions, fully or partially. The two technologies of alert generation and suggestion generation presented here are combined, and a comprehensive human subject study is conducted to evaluate the eects of the entire alert framework. The data collected from the user study is analyzed to gain useful insights and test several hypotheses. It is shown that cognitive aids in the form of alerts and suggestions result in better decisions with improved mission performance, faster decision-making by humans, and faster completion of higher- priority tasks by the robots. The study also shows that an individual's trust in automation positively correlates to mission performance with alerts, and only a minimal training overhead is required for integrating the alert framework. • This dissertation also presents alert generation toward task-level risk identication and management in human-supervised mobile manipulation operations. A modular architecture and sensor-agnostic methods are proposed which allow integration of the alert framework with any mobile manipulator hardware and software suite. This paper demonstrates how the probability of several failure modes occurring with a particular task plan can be computed in an ecient manner. It also shows how alerts and risk-related information can be graphically presented to human supervisors in manipulation tasks. 169 8.2 Anticipated Benets With the advancement of articial intelligence and robotic technology, more and more robots are being used in a wide variety of applications. This dissertation mainly focuses on application domains related to disaster rescue and humanitarian assistance. These applications are great for deploying robots as we can limit the exposure to risky situations for humans operating in such scenarios. However, due to the inherent unpredictability and uncertainty involved with the robots and such missions, current robotic deployment is still very limited. Regardless, humans have to have decision-making authority in safety-critical missions. This dissertation aims to develop cognitive aids for humans, in the form of alerts and suggestions. We believe it can encourage the future deployment of robots under human supervision in such scenarios and make it safer and more feasible. The dissertation presents some computation speed-up techniques in the discrete event simulation paradigm which can be applicable to any application where lots of simulations need to be performed for a team of agents roaming around in large geographical areas. This work focuses on multi-robot operations in large- scale search and rescue or similar applications. The alert generation architecture proposed in this document is feasible and exible enough to be easily incorporated and used in a wide variety of applications. The multi-robot task allocation approach is proposed to be used for suggestion generation in human- supervised applications. However, the technology is generic enough to be applied in any multi-robot task allocation problem that has similar characteristics, requirements, and constraints, even if there is no human involved. The comparison with baseline approaches in terms of mission performance shows the applicability of the method in general. The main anticipated benet of this work lies in improving human decision-making when they are su- pervising or commanding robots in challenging operations. The human subject study carried out as a part of this work clearly demonstrates how these alerts and suggestions can enhance human decision-making. Humans are inherently inadequate at processing a lot of information, propagating uncertainty, make a prob- abilistic assessment of pros and cons. Cognitive overload and stress can make the job even more dicult. Therefore, cognitive aids, such as alerts and suggestions, can certainly improve human performance in these decision-making roles with robots. 170 Lastly, supervising manipulation tasks with mobile manipulation robots come with its unique challenges, where risk assessment and alert generation for executing a task plan can prevent accidents. The proposed method can make manipulation using semi-autonomous robots much safer. 8.3 Future Directions The present work can be extended in the following directions in the future. • The rapid advancement in articial intelligence and computational capability will enable us to deploy AI-driven robotic agents on a massive scale in the future. There are a variety of robotic agents, such as unmanned ground vehicles, drones, boats, legged robots, etc., each with unique capabilities and limitations. The robots can signicantly vary in scale or size, currently ranging from nanometers to tens of meters, and the range will keep expanding. All these robots will have fundamentally dierent capabilities and will require to work together to achieve common goals. Firstly, the required team size in future missions will go extremely large. Secondly, There will be a huge variability among the robots and the widely diering attributes within the team will need to be eciently utilized. There might be multiple humans involved which will add to the heterogeneity of the overall team. Thirdly, the team structure might require to be more hierarchical to make use of the large and diversied team of agents. Overall, future missions will get extremely complex on all fronts. The scope of mission tasks will widen a lot, more complicated tasks will be performed by autonomous agents, including tasks where multiple agents would need to physically collaborate with each other. The operational environment might range from air, water, and ground to a combination of all kinds, spanning geographical regions beyond any limit. All this will require new alert generation methodologies, the present technology with simulation- based assessment will not work. The alert generation framework might need to be distributed in nature to accommodate the dimensionality and complexity of future missions with human-robot teams. Novel simulation techniques will be required for ensuring feasibility in terms of computational time and estimation accuracy. Machine learning-based techniques can aid in making mission predictions, alert generation, identifying useful alerts, and prioritization across dierent alerts. Generative AI technology 171 might be useful in improving the robustness of the alert system by becoming more adept at handling new scenarios and alleviating human workload further by automating alert conguration according to preferences specic to each user. Future research will pave the way to handling all these aspects and developing alert generation frameworks for a variety of missions. • The ever-expanding team size and heterogeneity in multi-robot applications will require brand-new approaches to the task allocation problem. Manually designed heuristics will not be sucient in the future. The use of machine learning techniques for developing the guiding heuristics in task allocation problems is an obvious future direction. Neural network, active learning, reinforcement learning, genetic algorithms - each method has its own merits toward specic aspects of task scheduling. In fact, decision-making in complex future missions would go much beyond a multi-agent task assignment problem. There will be scenarios where contingencies occur at high frequencies amid rapidly changing mission conditions. Contingencies will become more complicated which would require more advanced solutions. Machine learning-based approaches might be useful in the automatic generation of newer and better contingency resolution strategies. Learning techniques with the history of human interventions will ensure improved contingency handling. • Future research is needed to develop interaction techniques for humans to eectively supervise robot teams and make decisions. Operational performance can be enhanced by studying the agent's behavior, identifying undesirable behaviors, and addressing them. The increasing complexity of future missions will require the agents to interact in many dierent ways. This interaction might be information sharing, intellectual interaction, or physical. A large heterogeneous multi-agent team, involving AI- driven agents and humans, would require signicant research towards developing eective interactions. • The benets of cognitive aids, such as alerts and decision recommendations, greatly depend on the user interface used by the decision-maker. A great interface that can present all information in the best possible way and the interactive features are optimally integrated can amplify the merits of the overall alert system. The interface will not be limited to a display or buttons and joysticks. Signicant research is currently being done in dierent modes of interfaces, such as augmented reality, virtual 172 reality, neural interface, etc. The research trend is moving from single-modal interfaces to multi-modal interfaces which are enabling complex operations. Humans with disability are becoming more capable of participating and contributing in meaningful roles. Therefore, interface design is an essential future research direction. 173 References [1] Gazebo: Physics parameters. http://gazebosim.org/tutorials?tut=physics_params&cat= physics. Accessed: 2021-03-03. [2] Ali Agha, Kyohei Otsu, Benjamin Morrell, David D Fan, Rohan Thakker, Angel Santamaria-Navarro, Sung-Kyun Kim, Amanda Bouman, Xianmei Lei, Jerey Edlund, et al. Nebula: Quest for robotic au- tonomy in challenging environments; team costar at the darpa subterranean challenge. arXiv preprint arXiv:2103.11470, 2021. [3] Iretiayo Akinola, Jacob Varley, Boyuan Chen, and Peter K Allen. Workspace aware online grasp planning. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2917{2924. IEEE, 2018. [4] Sarah Al-Hussaini, Neel Dhanaraj, Jason M Gregory, Rex J. Joseph, Shantanu Thakar, , Brual C. Shah, Jeremy A. Marvel, and Satyandra K Gupta. Seeking human help to manage plan failure risks in semi-autonomous mobile manipulation. ASME Journal of Computing and Information Science in Engineering (JCISE), 2022. [5] Sarah Al-Hussaini, Jason M Gregory, Neel Dhanaraj, and Satyandra K Gupta. A Simulation-Based Framework for Generating Alerts for Human-Supervised Multi-Robot Teams in Challenging Environ- ments . In IEEE International Conference on Safety, Security, and Rescue Robotics (SSRR), October 2021. [6] Sarah Al-Hussaini, Jason M Gregory, Yuxiang Guan, and Satyandra K Gupta. Generating Alerts to Assist With Task Assignments in Human-Supervised Multi-Robot Teams Operating in Challenging Environments. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, U.S.A, October 2020. [7] Sarah Al-Hussaini, Jason M. Gregory, and Satyandra K. Gupta. Generation of context-dependent policies for robot rescue decision-making in multi-robot teams. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, October 2018. [8] Sarah Al-Hussaini, Jason M. Gregory, and Satyandra K. Gupta. A policy synthesis-based framework for robot rescue decision-making in multi-robot exploration of disaster sites. In IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), Philadelphia, PA, U.S.A., August 2018. [9] Sarah Al-Hussaini, Jason M. Gregory, and Satyandra K. Gupta. Generating task reallocation sug- gestions to handle contingencies in human-supervised multi-robot missions. IEEE Transactions on Automation Science and Engineering (TASE), 2023. [10] Sarah Al-Hussaini, Jason M Gregory, Shaurya Shriyam, and Satyandra K Gupta. An alert-generation framework for improving resiliency in human-supervised, multi-agent teams. In Articial Intelligence for Human-Robot Interaction Symposium (AI-HRI), AAAI Fall Symposium Series, Arlington, VA, U.S.A, November 2019. [11] Sarah Al-Hussaini, Yuxiang Guan, Jason M Gregory, Kimberly Pollard, Peter Khooshabeh, and Satyandra K Gupta. Assessing the impact of alerts on the human supervisor's decision-making per- formance in multi-robot missions. Manuscript under review. 174 [12] Sarah Al-Hussaini, Shantanu Thakar, Hyojyong Kim, Pradeep Rajendran, Brual C. Shah, Jeremy A. Marvel, and Satyandra K Gupta. Human-supervised semi-autonomous mobile manipulators for safely and eciently executing machine tending tasks. In Articial Intelligence for Human-Robot Interaction Symposium (AI-HRI), AAAI Fall Symposium Series, November 2020. [13] Stefano Di Alesio, Lionel C Briand, Shiva Nejati, and Arnaud Gotlieb. Combining genetic algorithms and constraint programming to support stress testing of task deadlines. ACM Transactions on Software Engineering and Methodology (TOSEM), 25(1):1{37, 2015. [14] Daniel Altho, James J Kuner, Dirk Wollherr, and Martin Buss. Safety assessment of robot tra- jectories for navigation in uncertain and dynamic environments. Autonomous Robots, 32(3):285{302, 2012. [15] Daniel Altho, Dirk Wollherr, and Martin Buss. Safety assessment of trajectories for navigation in uncertain and dynamic environments. In 2011 IEEE International Conference on Robotics and Automation, pages 5407{5412. IEEE, 2011. [16] Vivek Annem, Pradeep Rajendran, Shantanu Thakar, and Satyandra K. Gupta. Towards remote teleoperation of a semi-autonomous mobile manipulator system in machine tending tasks. In ASME Manufacturing Science and Engineering Conference (MSEC), Erie, USA, June 2019. [17] Shady Attia, Elisabeth Gratia, Andr e De Herde, and Jan LM Hensen. Simulation-based decision support tool for early stages of zero-energy building design. Energy and buildings, 49:2{15, 2012. [18] Terje Aven. On the meaning of a black swan in a risk context. Safety science, 57:44{51, 2013. [19] Vahid Azizi, Andrew Kimmel, Kostas Bekris, and Mubbasir Kapadia. Geometric reachability analysis for grasp planning in cluttered scenes for varying end-eectors. In 2017 13th IEEE Conference on Automation Science and Engineering (CASE), pages 764{769. IEEE, 2017. [20] Michael Baker, Robert Casey, Brenden Keyes, and Holly A Yanco. Improved interfaces for human- robot interaction in urban search and rescue. In 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No. 04CH37583), volume 3, pages 2960{2965. IEEE, 2004. [21] Michael Baker and Holly A Yanco. Autonomy mode suggestions for improving human-robot inter- action. In 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No. 04CH37583), volume 3, pages 2948{2953. IEEE, 2004. [22] Christopher Banks, Sean Wilson, Samuel Coogan, and Magnus Egerstedt. Multi-agent task allocation using cross-entropy temporal logic optimization. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 7712{7718. IEEE, 2020. [23] Micha l Barci s, Agata Barci s, and Hermann Hellwagner. Information distribution in multi-robot sys- tems: Utility-based evaluation model. Sensors, 20(3):710, 2020. [24] Michael J Barnes, Jessie Y Chen, Florian Jentsch, Tal Oron-Gilad, Elizabeth Redden, Linda Elliott, and Arthur W Evans III. Designing for humans in autonomous systems: Military applications. Tech- nical report, ARMY RESEARCH LAB ABERDEEN, 2014. [25] Frederic Becker, Julian Skirzy nski, Bas van Opheusden, and Falk Lieder. Boosting human decision- making with ai-generated decision aids. arXiv preprint arXiv:2203.02776, 2022. [26] Tamer Boyaci, Caner Canyakmaz, and Francis deVericourt. Human and machine: The impact of machine input on decision-making under cognitive limitations. 2020. [27] Jennifer L Burke, Robin R Murphy, Michael D Coovert, and Dawn L Riddle. Moonlight in miami: Field study of human-robot interaction in the context of an urban search and rescue disaster response training exercise. Human{Computer Interaction, 19(1-2):85{116, 2004. 175 [28] Jonathan Cacace, Alberto Finzi, and Vincenzo Lippiello. Implicit robot selection for human multi- robot interaction in search and rescue missions. In 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pages 803{808. IEEE, 2016. [29] Joyce Y Chai, Rui Fang, Changsong Liu, and Lanbo She. Collaborative language grounding toward situated human-robot dialogue. AI Magazine, 37(4):32{45, 2016. [30] Imen Chaouch, Olfa Belkahla Driss, and Khaled Ghedira. A novel dynamic assignment rule for the distributed job shop scheduling problem using a hybrid ant-based algorithm. Applied Intelligence, 49(5):1903{1924, 2019. [31] Jonathan Chase, Tran Phong, Kang Long, Tony Le, and Hoong Chuin Lau. Grand-vision: An intel- ligent system for optimized deployment scheduling of law enforcement agents. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 31, pages 459{467, 2021. [32] Maria Mikela Chatzimichailidou and Ioannis M Dokas. Introducing risksoap to communicate the distributed situation awareness of a system about safety issues: an application to a robotic system. Ergonomics, 59(3):409{422, 2016. [33] Jessie Y Chen, Katelyn Procci, Michael Boyce, Julia Wright, Andre Garcia, and Michael Barnes. Situation awareness-based agent transparency. Technical report, Army research lab aberdeen proving ground md human research and engineering . . . , 2014. [34] Jessie YC Chen. Human-autonomy teaming in military settings. Theoretical issues in ergonomics science, 19(3):255{258, 2018. [35] Jessie YC Chen, Shan G Lakhmani, Kimberly Stowers, Anthony R Selkowitz, Julia L Wright, and Michael Barnes. Situation awareness-based agent transparency and human-autonomy teaming eec- tiveness. Theoretical issues in ergonomics science, 19(3):259{282, 2018. [36] Min Chen, Stefanos Nikolaidis, Harold Soh, David Hsu, and Siddhartha Srinivasa. Planning with trust for human-robot collaboration. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, pages 307{315, 2018. [37] Wayne Chi, Jagriti Agrawal, Steve Chien, Elyse Fosse, and Usha Guduri. Optimizing parameters for uncertain execution and rescheduling robustness. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 29, pages 501{509, 2019. [38] Shih-Yi Chien, Yi-Ling Lin, Pei-Ju Lee, Shuguang Han, Michael Lewis, and Katia Sycara. Attention allocation for human multi-robot control: Cognitive analysis based on behavior data and hidden states. International Journal of Human-Computer Studies, 117:30{44, 2018. [39] JongOh Choi, Euijin Hwang, H-Y So, and Byungjoo Kim. An uncertainty evaluation for multiple measurements by gum. Accreditation and quality assurance, 8(1):13{15, 2003. [40] Raaele Colombo, Francesco Gennari, Vivek Annem, Pradeep Rajendran, Shantanu Thakar, Luca Bascetta, and Satyandra K. Gupta. Parameterized model predictive control of a nonholonomic mobile manipulator: A terminal constraint-free approach. In IEEE International Conference on Automation Science and Engineering (CASE), Vancouver, Canada, August 2019. [41] Brian Curless and Marc Levoy. A volumetric method for building complex models from range images. In John Fujii, editor, SIGGRAPH, pages 303{312. ACM, 1996. [42] Torbjrn S Dahl, Maja Matari c, and Gaurav S Sukhatme. Multi-robot task allocation through vacancy chain scheduling. Robotics and Autonomous Systems, 57(6-7):674{687, 2009. [43] Bruno Damas, Pedro Lima, et al. Stochastic discrete event model of a multi-robot team playing an adversarial game. In 5th IFAC/EURON Symposium on Intelligent Autonomous Vehicles-IAV2004, 2004. 176 [44] Mirella Santos Pessoa de Melo, Jos e Gomes da Silva Neto, Pedro Jorge Lima da Silva, Jo~ ao Marcelo Xavier Natario Teixeira, and Veronica Teichrieb. Analysis and comparison of robotics 3d simulators. In 2019 21st Symposium on Virtual and Augmented Reality (SVR), pages 242{251. IEEE, 2019. [45] Ewart de Visser and Raja Parasuraman. Adaptive aiding of human-robot teaming: Eects of imperfect automation on performance, trust, and workload. Journal of Cognitive Engineering and Decision Making, 5(2):209{231, 2011. [46] Arwen H DeCostanza, Amar R Marathe, Addison Bohannon, A William Evans, Edward T Palazzolo, Jason S Metcalfe, and Kaleb McDowell. Enhancing human-agent teaming with individualized, adap- tive technologies: A discussion of critical scientic questions. Technical report, US Army Research Laboratory Aberdeen Proving Ground United States, 2018. [47] Neel Dhanaraj, Yeo Jung Yoon, Rishi Malhan, Prahar M. Bhatt, Shantanu Thakar, and Satyandra K. Gupta. A mobile manipulator system for accurate and ecient spraying on large surfaces. Procedia Computer Science, 200:1528{1539, 2022. 3rd International Conference on Industry 4.0 and Smart Manufacturing. [48] M Bernardine Dias, Balajee Kannan, Brett Browning, E Jones, Brenna Argall, M Freddie Dias, Marc Zinck, Manuela M Veloso, and Anthony Stentz. Sliding autonomy for peer-to-peer human-robot teams. In Proceedings of the international conference on intelligent autonomous systems, pages 332{341, 2008. [49] Thomas Dietrich, Silvia Krug, and Armin Zimmermann. A discrete event simulation and evaluation framework for multi uav system maintenance processes. In 2017 IEEE international systems engineer- ing symposium (ISSE), pages 1{6. IEEE, 2017. [50] Thomas Diller, George Helmrich, Sharon Dunning, Stephanie Cox, April Buchanan, and Scott Shap- pell. The human factors analysis classication system (hfacs) applied to health care. American Journal of Medical Quality, 29(3):181{190, 2014. PMID: 23814026. [51] Xuchu Dennis Ding, Brendan Englot, Alessandro Pinto, Alberto Speranzon, and Amit Surana. Hier- archical multi-objective planning: From mission specications to contingency management. In 2014 IEEE international conference on robotics and automation (ICRA), pages 3735{3742. IEEE, 2014. [52] Amir Dini, Cornelia Murko, Saeed Yahyanejad, Ursula Augsd orfer, Michael Hofbaur, and Lucas Paletta. Measurement and prediction of situation awareness in human-robot interaction based on a framework of probabilistic attention. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4354{4361. IEEE, 2017. [53] Ilias El Makrini, Kelly Merckaert, Joris De Winter, Dirk Lefeber, and Bram Vanderborght. Task allo- cation for improved ergonomics in human-robot collaborative assembly. Interaction Studies, 20(1):102{ 133, 2019. [54] Mica R Endsley. Level of automation forms a key aspect of autonomy design. Journal of Cognitive Engineering and Decision Making, 12(1):29{34, 2018. [55] Eduardo Feo Flushing, Luca M Gambardella, and Gianni A Di Caro. Simultaneous task allocation, data routing, and transmission scheduling in mobile multi-robot teams. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1861{1868. IEEE, 2017. [56] Terrence Fong, Charles Thorpe, and Charles Baur. Multi-robot remote driving with collaborative control. IEEE Transactions on Industrial Electronics, 50(4):699{704, 2003. [57] Dieter Fox, Wolfram Burgard, and Sebastian Thrun. The dynamic window approach to collision avoidance. IEEE Robotics & Automation Magazine, 4(1):23{33, 1997. [58] Jerry L Franke, Stephen M Jameson, Rosemary D Paradis, and Robert J Szczerba. Hierarchical contingency management system for mission planners, December 13 2011. US Patent 8,078,319. 177 [59] Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning, volume 1. Springer series in statistics New York, 2001. [60] Nicholas R Gans and John G Rogers. Cooperative multirobot systems for military applications. Cur- rent Robotics Reports, pages 1{7, 2021. [61] Fei Gao and Missy Cummings. Using discrete event simulation to model multi-robot multi-operator teamwork. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 56, pages 2093{2097. SAGE Publications Sage CA: Los Angeles, CA, 2012. [62] Iv an Garc a-Magari~ no, Alma G omez-Rodr guez, Juan C Gonz alez-Moreno, and Guillermo Palacios- Navarro. Peabs: a process for developing ecient agent-based simulators. Engineering Applications of Articial Intelligence, 46:104{112, 2015. [63] Brian P Gerkey and Maja J Matari c. A formal analysis and taxonomy of task allocation in multi-robot systems. The International journal of robotics research, 23(9):939{954, 2004. [64] Felix Gervits, Dean Thurston, Ravenna Thielstrom, Terry Fong, Quinn Pham, and Matthias Scheutz. Toward genuine robot teammates: Improving human-robot team performance using robot shared men- tal models. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, pages 429{437, 2020. [65] Mohamed Gomaa and Amal Gomaa. The eect of time pressure, task complexity and litigation risk on auditors' reliance on decision aids. 2018. [66] Kevin Gomes, Danelle Cline, Duane Edgington, Michael Godin, Thom Maughan, Mike McCann, Tom O'Reilly, Fred Bahr, Francisco Chavez, Monique Messi e, et al. Odss: A decision support system for ocean exploration. In 2013 IEEE 29th International Conference on Data Engineering Workshops (ICDEW), pages 200{211. IEEE, 2013. [67] Jason Gregory, Jonathan Fink, Ethan Stump, Jerey Twigg, John Rogers, David Baran, Nicholas Fung, and Stuart Young. Application of Multi-Robot Systems to Disaster-Relief Scenarios with Limited Communication. Field and Service Robotics: Results of the 10th International Conference, pages 639{ 653, 2016. [68] Jason M Gregory, Sarah Al-Hussaini, Ali-akbar Agha-mohammadi, and Satyandra K Gupta. Tax- onomy of a decision support system for adaptive experimental design in eld robotics. In Articial Intelligence for Human-Robot Interaction Symposium (AI-HRI), AAAI Fall Symposium Series, Ar- lington, VA, U.S.A, November 2022. [69] Jason M Gregory, Sarah Al-Hussaini, and Satyandra K Gupta. Heuristics-based multi-agent task allocation for resilient operations. In 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pages 1{8. IEEE, 2019. [70] Jason M Gregory, Iain Brookshaw, Jonathan Fink, and Satyandra K Gupta. An investigation of goal assignment for a heterogeneous robotic team to enable resilient disaster-site exploration. In 2017 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), pages 133{140. IEEE, 2017. [71] Jason M Gregory, Daniel Sahu, Eli Lancaster, Felix Sanchez, Trevor Rocks, Brian Kaukeinen, Jonathan Fink, and Satyandra K Gupta. Active Learning for Testing and Evaluation in Field Robotics: A Case Study in Autonomous, O-Road Navigation. In 2022 International Conference on Robotics and Automation (ICRA), pages 8217{8223. IEEE, 2022. [72] Robert S. Gutzwiller, Erin K. Chiou, Scotty D. Craig, Christina M. Lewis, Glenn J. Lematta, and Chi-Ping Hsiung. Positive bias in the `trust in automated systems survey'? an examination of the jian et al. (2000) scale. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 63(1):217{221, 2019. 178 [73] Svyatoslav Guznov, J Lyons, Marc Pfahler, A Heironimus, Montana Woolley, Jeremy Friedman, and A Neimeier. Robot transparency and team orientation eects on human{robot teaming. International Journal of Human{Computer Interaction, 36(7):650{660, 2020. [74] Lydia Habib, Marie-Pierre Pacaux-Lemoine, and Patrick Millot. Human-robots team cooperation in crisis management mission. In 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pages 3219{3224. IEEE, 2018. [75] Maaike Harbers, Karel van den Bosch, and John-Jules Meyer. Design and evaluation of explainable bdi agents. In 2010 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, volume 2, pages 125{132. IEEE, 2010. [76] James Robert Harbin, Simos Gerasimou, Nikolaos Matragkas, Athanasios Zolotas, and Radu Calinescu. Model-driven simulation-based analysis for multi-robot systems. In MODELS 2021: ACM/IEEE 24th International Conference on Model Driven Engineering Languages and Systems (MODELS). York, 2021. [77] Ahnaf Rashik Hassan and Abdulhamit Subasi. A decision support system for automated identication of sleep stages from single-channel eeg signals. Knowledge-Based Systems, 128:115{124, 2017. [78] Wei He, Zhijun Li, and CL Philip Chen. A survey of human-centered intelligent robots: issues and challenges. IEEE/CAA Journal of Automatica Sinica, 4(4):602{609, 2017. [79] Guy Homan and Xuan Zhao. A primer for conducting experiments in human{robot interaction. ACM Transactions on Human-Robot Interaction (THRI), 10(1):1{31, 2020. [80] Yuuji Hosoda, Hiroshi Yamamoto, Makoto Hattori, Hiroshi Sakairi, Tarou Iwamoto, Masataka Oowada, Akihiro Kanno, and Yuuji Saitou. 'swan': a robot for nuclear disaster prevention support. Advanced Robotics, 16(6):485{488, 2002. [81] Hsu-Chieh Hu and Stephen F Smith. Learning model parameters for decentralized schedule-driven trac control. In Proceedings of the International Conference on Automated Planning and Scheduling, volume 30, pages 531{539, 2020. [82] Xun Huan and Youssef M Marzouk. Simulation-based optimal bayesian experimental design for non- linear systems. Journal of Computational Physics, 232(1):288{317, 2013. [83] M. Hybinette, E. Kraemer, Y. Xiong, G. Matthews, and J. Ahmed. Sassy: A design for a scalable agent-based simulation system using a distributed discrete event infrastructure. In Proceedings of the 2006 Winter Simulation Conference, pages 926{933, Dec 2006. [84] Raa Inam, Klaus Raizer, Alberto Hata, Ricardo Souza, Elena Forsman, Enyu Cao, and Shaolei Wang. Risk assessment for human-robot collaboration in an automated warehouse scenario. In 2018 IEEE 23rd International Conference on Emerging Technologies and Factory Automation (ETFA), volume 1, pages 743{751. IEEE, 2018. [85] International Organization for Standardization. Systems and software engineering { Systems and software Quality Requirements and Evaluation (SQuaRE) { System and software quality models. Standard, International Organization for Standardization /International Electrotechnical Commission, Geneva, CH, 2011. [86] Florian Jentsch. Human-robot interactions in future military operations. CRC Press, 2016. [87] Jiun-Yin Jian, Ann M Bisantz, and Colin G Drury. Foundations for an empirically determined scale of trust in automated systems. International journal of cognitive ergonomics, 4(1):53{71, 2000. [88] Shiyuan Jin, Guy Schiavone, and Damla Turgut. A performance study of multiprocessor task schedul- ing algorithms. The Journal of Supercomputing, 43(1):77{97, 2008. 179 [89] David B Kaber. Issues in human{automation interaction modeling: Presumptive aspects of frameworks of types and levels of automation. Journal of Cognitive Engineering and Decision Making, 12(1):7{24, 2018. [90] A. M. Kabir, S. Thakar, P. M. Bhatt, R. K. Malhan, P. Rajendran, B. C. Shah, and S. K. Gupta. Incorporating motion planning feasibility considerations during task-agent assignment to perform com- plex tasks using mobile-manipulators. In IEEE International Conference on Robotics and Automation, Paris, France, June 2020. [91] Ariyan Kabir, Alec Kanyuck, Rishi K. Malhan, Aniruddha V. Shembekar, Shantanu Thakar, Brual C. Shah, and Satyandra K. Gupta. Generation of synchronized conguration space trajectories of multi- robot systems. In IEEE International Conference on Robotics and Automation (ICRA), Montreal, Canada, May 2019. [92] Ariyan M Kabir, Alec Kanyuck, Rishi K Malhan, Aniruddha V Shembekar, Shantanu Thakar, Brual C Shah, and Satyandra K Gupta. Generation of synchronized conguration space trajectories of multi- robot systems. In 2019 International Conference on Robotics and Automation (ICRA), pages 8683{ 8690. IEEE, 2019. [93] Ariyan M Kabir, Shantanu Thakar, Prahar M Bhatt, Rishi K Malhan, Pradeep Rajendran, Brual C Shah, and Satyandra K Gupta. Incorporating motion planning feasibility considerations during task- agent assignment to perform complex tasks using mobile manipulators. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 5663{5670. IEEE, 2020. [94] Ariyan M Kabir, Shantanu Thakar, Rishi K Malhan, Aniruddha V Shembekar, Brual C Shah, and Satyandra K Gupta. Generation of synchronized conguration space trajectories with workspace path constraints for an ensemble of robots. The International Journal of Robotics Research, 40(2-3):651{678, 2021. [95] Mrinal Kalakrishnan, Sachin Chitta, Evangelos Theodorou, Peter Pastor, and Stefan Schaal. Stomp: Stochastic trajectory optimization for motion planning. In 2011 IEEE international conference on robotics and automation, pages 4569{4574. IEEE, 2011. [96] Arnold Kamis, Marios Koufaris, and Tziporah Stern. Using an attribute-based decision support system for user-customized products online: an experimental investigation. MIS quarterly, pages 159{177, 2008. [97] Kapil D Katyal, Christopher Y Brown, Steven A Hechtman, Matthew P Para, Timothy G McGee, Kevin C Wolfe, Ryan J Murphy, Michael DM Kutzer, Edward W Tunstel, Michael P McLoughlin, et al. Approaches to robotic teleoperation in a disaster scenario: From supervised autonomy to direct control. In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1874{ 1881. IEEE, 2014. [98] Giora Keinan. Decision making under stress: scanning of alternatives under controllable and uncon- trollable threats. Journal of personality and social psychology, 52(3):639, 1987. [99] Hyungil Kim, Joseph L Gabbard, Alexandre Miranda Anon, and Teruhisa Misu. Driver behavior and performance with augmented reality pedestrian collision warning: An outdoor user study. IEEE transactions on visualization and computer graphics, 24(4):1515{1524, 2018. [100] Wansoo Kim, Jinoh Lee, Luka Peternel, Nikos Tsagarakis, and Arash Ajoudani. Anticipatory robot assistance for the prevention of human static joint overloading in human{robot collaboration. IEEE robotics and automation letters, 3(1):68{75, 2017. [101] Sven Koenig, Pinar Keskinocak, and Craig Tovey. Progress on agent coordination with cooperative auctions. In Proceedings of the AAAI Conference on Articial Intelligence, volume 24, 2010. [102] Savas Konur. A survey on temporal logics for specifying and verifying real-time systems. Frontiers of Computer Science, 7(3):370{403, 2013. 180 [103] Savas Konur. A survey on temporal logics for specifying and verifying real-time systems. Frontiers of Computer Science, 7(3):370{403, 2013. [104] Tom a s Kot and Petr Nov ak. Application of virtual reality in teleoperation of the military mobile robotic system taros. International journal of advanced robotic systems, 15(1):1729881417751545, 2018. [105] Kentaro Kozai and Manabu Hashimoto. Determining robot grasping-parameters by estimating" pick- ing risk". In 2018 International Workshop on Advanced Image Technology (IWAIT), pages 1{4. IEEE, 2018. [106] Geert-Jan M Kruij, M Jan cek, Shanker Keshavdas, Benoit Larochelle, Hendrik Zender, Nanja JJM Smets, Tina Mioch, Mark A Neerincx, JV Diggelen, Francis Colas, et al. Experience in system design for human-robot teaming in urban search and rescue. In Field and Service Robotics, pages 111{125. Springer, 2014. [107] James J Kuner and Steven M LaValle. Rrt-connect: An ecient approach to single-query path planning. In Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065), volume 2, pages 995{1001. IEEE, 2000. [108] Michael J Kuhlman, Dylan Jones, Donald A Sofge, Georey A Hollinger, and Satyandra K Gupta. Collaborating underwater vehicles conducting large-scale geospatial tasks. IEEE Journal of Oceanic Engineering, 46(3):785{807, 2021. [109] Michael J Kuhlman, Michael W Otte, Donald Sofge, and Satyandra K Gupta. Maximizing mutual information for multipass target search in changing environments. In 2017 IEEE International Con- ference on Robotics and Automation (ICRA), pages 4383{4390. IEEE, 2017. [110] Michael J Kuhlman, Michael W Otte, Donald Sofge, and Satyandra K Gupta. Multipass target search in natural environments. Sensors, 17(11):2514, 2017. [111] Michael J Kuhlman, Petr Svec, Krishnanand N Kaipa, Donald Sofge, and Satyandra K Gupta. Physics- aware informative coverage planning for autonomous vehicles. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 4741{4746. IEEE, 2014. [112] Mathieu Labb e and Fran cois Michaud. Rtab-map as an open-source lidar and visual simultaneous localization and mapping library for large-scale and long-term online operation. Journal of Field Robotics, 36(2):416{446, 2019. [113] Dominic Larkin, Michael Novitzky, Jinho Kim, and Christopher M Korpela. Atak integration through ros for autonomous air-ground team. In 2021 International Conference on Unmanned Aircraft Systems (ICUAS), pages 1116{1122. IEEE, 2021. [114] Mikko Lauri and Risto Ritala. Planning for robotic exploration based on forward simulation. Robotics and Autonomous Systems, 83:15{31, 2016. [115] Steven M LaValle. From dynamic programming to rrts: Algorithmic design of feasible trajectories. In Control Problems in Robotics, pages 19{37. Springer, 2003. [116] Christian Lebiere and John R Anderson. Cognitive constraints on decision making under uncertainty. Frontiers in psychology, 2:305, 2011. [117] Steven James Levine and Brian Charles Williams. Concurrent plan recognition and execution for human-robot teams. In Twenty-Fourth International Conference on Automated Planning and Schedul- ing, 2014. [118] Hongyi Liu and Lihui Wang. Collision-free human-robot collaboration based on context awareness. Robotics and Computer-Integrated Manufacturing, 67:101997, 2021. 181 [119] Xiang Liu, Rui Lin Ma, Jingwen Zhao, Jia Ling Song, Jian Quan Zhang, and Shuo Hong Wang. A clinical decision support system for predicting cirrhosis stages via high frequency ultrasound images. Expert Systems with Applications, 175:114680, 2021. [120] Yugang Liu and Goldie Nejat. Multirobot cooperative learning for semiautonomous control in urban search and rescue applications. Journal of Field Robotics, 33(4):512{536, 2016. [121] Giacomo Lunghi, Raul Marin, Mario Di Castro, Alessandro Masi, and Pedro J Sanz. Multimodal human-robot interface for accessible remote robotic interventions in hazardous environments. IEEE Access, 7:127290{127319, 2019. [122] Lanssie Mingyue Ma, Terrence Fong, Mark J Micire, Yun Kyung Kim, and Karen Feigh. Human-robot teaming: Concepts and components for design. In Field and service robotics, pages 649{663. Springer, 2018. [123] Sotiris Makris, Panagiotis Karagiannis, Spyridon Koukas, and Aleksandros-Stereos Matthaiakis. Aug- mented reality system for operator support in human{robot collaborative assembly. CIRP Annals, 65(1):61{64, 2016. [124] Carmen Marcher, Andrea Giusti, and Dominik T Matt. On the design of a decision support system for robotic equipment adoption in construction processes. Applied Sciences, 11(23):11415, 2021. [125] Marcos Maroto-G omez, Alvaro Castro-Gonz alez, Jos e Carlos Castillo, Mar a Malfaz, and Miguel Angel Salichs. An adaptive decision-making system supported on user preference predictions for human{robot interactive communication. User Modeling and User-Adapted Interaction, pages 1{45, 2022. [126] James R Marsden, Ramakrishnan Pakath, and Kustim Wibowo. Decision making under time pressure with dierent information sources and performance-based nancial incentives: part 3. Decision support systems, 42(1):186{203, 2006. [127] Naresh Marturi, Marek Kopicki, Alireza Rastegarpanah, Vijaykumar Rajasekaran, Maxime Adjigble, Rustam Stolkin, Ale s Leonardis, and Yasemin Bekiroglu. Dynamic grasp and trajectory planning for moving objects. Autonomous Robots, 43(5):1241{1256, 2019. [128] Laura R Marusich, Norbou Buchler, and Jonathan Z Bakdash. Human limits to cognitive information fusion in a military decision-making task. Technical report, Army Research Lab, Aberdeen Proving Ground, MD, 2014. [129] Jeremy A Marvel and Roger Bostelman. A cross-domain survey of metrics for modelling and evaluating collisions. International Journal of Advanced Robotic Systems, 11(9):142, 2014. [130] Jeremy A Marvel, Joe Falco, and Ilari Marstio. Characterizing task-based human{robot collaboration safety in manufacturing. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 45(2):260{ 275, 2014. [131] Siddharth Mayya and Magnus Egerstedt. Safe open-loop strategies for handling intermittent commu- nications in multi-robot systems. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 5818{5823. IEEE, 2017. [132] Steve McGuire, P Michael Furlong, Christoer Heckman, Simon Julier, Daniel Szar, and Nisar Ahmed. Failure is not an option: Policy learning for adaptive recovery in space operations. IEEE Robotics and Automation Letters, 3(3):1639{1646, 2018. [133] Mitchell McIntire, Ernesto Nunes, and Maria Gini. Iterated multi-robot auctions for precedence- constrained task scheduling. In Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, pages 1078{1086, 2016. [134] Abdullah Mohammed, Bernard Schmidt, and Lihui Wang. Active collision avoidance for human{robot collaboration driven by vision sensors. International Journal of Computer Integrated Manufacturing, 30(9):970{980, 2017. 182 [135] Carlos Morato, Krishnanand N Kaipa, Boxuan Zhao, and Satyandra K Gupta. Toward safe human robot collaboration by using multiple kinects based real-time human tracking. Journal of Computing and Information Science in Engineering, 14(1), 2014. [136] Kathleen L Mosier, Ute Fischer, Barbara K Burian, and Janeen A Kochan. Autonomous, context- sensitive, task management systems and decision support tools i: Human-autonomy teaming funda- mentals and state of the art. 2017. [137] S Meysam Mousavi and Reza Tavakkoli-Moghaddam. A hybrid simulated annealing algorithm for loca- tion and routing scheduling problems with cross-docking in the supply chain. Journal of Manufacturing Systems, 32(2):335{347, 2013. [138] Galen E Mullins and Satyandra K Gupta. Adversarial blocking techniques for autonomous surface vehicles using model-predictive motion goal computation. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2272{2278. IEEE, 2015. [139] Galen E Mullins, Paul G Stankiewicz, and Satyandra K Gupta. Automated generation of diverse and challenging scenarios for test and evaluation of autonomous vehicles. In 2017 IEEE International Conference on Robotics and Automation (ICRA), pages 1443{1450. IEEE, 2017. [140] Galen E Mullins, Paul G Stankiewicz, R Chad Hawthorne, and Satyandra K Gupta. Adaptive gener- ation of challenging scenarios for testing and evaluation of autonomous vehicles. Journal of Systems and Software, 137:197{215, 2018. [141] Robin R Murphy. Disaster robotics. MIT press, 2014. [142] Matthieu Myrsky, Heikki Nikula, Seppo Sierla, Jari Saarinen, Nikolaos Papakonstantinou, Ville Kyrki, and Bryan O'Halloran. Simulation-based risk assessment of robot eets in ooded environments. In 2013 IEEE 18th Conference on Emerging Technologies & Factory Automation (ETFA), pages 1{8. IEEE, 2013. [143] CE Nehme, Jacob W Crandall, and ML Cummings. Using discrete-event simulation to model sit- uational awareness of unmanned-vehicle operators. In Virginia Modeling, Analysis and Simulation Center Capstone Conference. Norfolk, VA. Citeseer, 2008. [144] Farzad Niroui, Yugang Liu, Rak Bichay, Elias Barker, Carlos Elchami, Owen Gillett, Maurizio Fico- celli, and Goldie Nejat. A graphical user interface for multi-robot control in urban search and rescue applications. In 2016 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), pages 217{222. IEEE, 2016. [145] Illah R Nourbakhsh, Katia Sycara, Mary Koes, Mark Yong, Michael Lewis, and Steve Burion. Human- robot teaming for search and rescue. IEEE Pervasive Computing, (1):72{78, 2005. [146] Ernesto Nunes, Marie Manner, Hakim Mitiche, and Maria Gini. A taxonomy for task allocation problems with temporal and ordering constraints. Robotics and Autonomous Systems, 90:55{70, 2017. [147] Olessia Ogorodnikova. Human weaknesses and strengths in collaboration with robots. Periodica Polytechnica Mechanical Engineering, 52(1):25{33, 2008. [148] Pedro A Ortega and Alan A Stocker. Human decision-making under limited time. Advances in Neural Information Processing Systems, 29, 2016. [149] Michael Otte, Michael Kuhlman, and Donald Sofge. Multi-robot task allocation with auctions in harsh communication environments. In 2017 International Symposium on Multi-Robot and Multi- Agent Systems (MRS), pages 32{39. IEEE, 2017. [150] Michael Otte, Michael J Kuhlman, and Donald Sofge. Auctions for multi-robot task allocation in communication limited environments. Autonomous Robots, 44(3):547{584, 2020. 183 [151] Jo el Ouaknine and James Worrell. Some recent results in metric temporal logic. In International Conference on Formal Modeling and Analysis of Timed Systems, pages 1{13. Springer, 2008. [152] A Ozgelen and Elizabeth I Sklar. An approach to supervisory control of multi-robot teams in dynamic domains. In Conference Towards Autonomous Robotic Systems, pages 198{203. Springer, 2015. [153] Jia Pan, Sachin Chitta, and Dinesh Manocha. Fcl: A general purpose library for collision and proximity queries. In 2012 IEEE International Conference on Robotics and Automation, pages 3859{3866. IEEE, 2012. [154] Tommaso Pardi, Rustam Stolkin, et al. Choosing grasps to enable collision-free post-grasp manipu- lations. In 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), pages 299{305. IEEE, 2018. [155] Jae-Han Park, Yong-Deuk Shin, Ji-Hun Bae, and Moon-Hong Baeg. Spatial uncertainty model for visual features using a kinect™ sensor. Sensors, 12(7):8640{8662, 2012. [156] Alessandro Pellegrini, Sebastiano Peluso, Francesco Quaglia, and Roberto Vitali. Transparent spec- ulative parallelization of discrete event simulation applications using global variables. International Journal of Parallel Programming, 44(6):1200{1247, 2016. [157] Megan A Perrin, Laura DiGrande, Katherine Wheeler, Lorna Thorpe, Mark Farfel, and Robert Brack- bill. Dierences in ptsd prevalence and associated risk factors among world trade center disaster rescue and recovery workers. American Journal of Psychiatry, 164(9):1385{1394, 2007. [158] Jerey R Peters and Luca F Bertuccelli. Robust task scheduling for multi-operator supervisory control missions. Journal of Aerospace Information Systems, 13(10):393{406, 2016. [159] Gloria Phillips-Wren and Monica Adya. Decision making under stress: The role of information over- load, time pressure, complexity, and uncertainty. Journal of Decision Systems, 29(sup1):213{225, 2020. [160] Carlo Pinciroli, Vito Trianni, Rehan O'Grady, Giovanni Pini, Arne Brutschy, Manuele Brambilla, Nithin Mathews, Eliseo Ferrante, Gianni Di Caro, Frederick Ducatelle, et al. Argos: a modular, parallel, multi-engine simulator for multi-robot systems. Swarm intelligence, 6(4):271{295, 2012. [161] Lenka Pitonakova, Manuel Giuliani, Anthony Pipe, and Alan Wineld. Feature and performance com- parison of the v-rep, gazebo and argos robot simulators. In Annual Conference Towards Autonomous Robotic Systems, pages 357{368. Springer, 2018. [162] Rey Pocius, Naghmeh Zamani, Heather Culbertson, and Stefanos Nikolaidis. Communicating robot goals via haptic feedback in manipulation tasks. In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, pages 591{593, 2020. [163] Anatoliy I Povoroznyuk, Anna E Filatova, Georgij R Mumladze, Alexandra S Zlepko, Dmytro H Shtofel, Andrii I Bezuglyi, Waldemar W ojcik, and Ulzhalgas Zhunissova. Formalization of the stages of diagnostic and therapeutic measures in decision support systems in medicine. In Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2019, volume 11176, pages 866{873. SPIE, 2019. [164] Eric Raboin, Petr Svec, Dana Nau, and Satyandra K Gupta. Model-predictive target defense by team of unmanned surface vehicles operating in uncertain environments. In 2013 IEEE International Conference on Robotics and Automation (ICRA), pages 3517{3522. IEEE, 2013. [165] Eric Raboin, Petr Svec, Dana S Nau, and Satyandra K Gupta. Model-predictive asset guarding by team of autonomous surface vehicles in environment with civilian boats. Autonomous Robots, 38(3):261{282, 2015. 184 [166] Pradeep Rajendran, Shantanu Thakar, Prahar M Bhatt, Ariyan M Kabir, and Satyandra K Gupta. Strategies for speeding up manipulator path planning to nd high quality paths in cluttered environ- ments. Journal of Computing and Information Science in Engineering, pages 1{27, 2020. [167] Pradeep Rajendran, Shantanu Thakar, and Satyandra K. Gupta. User-guided path planning for re- dundant manipulators in highly constrained work environments. In IEEE International Conference on Automation Science and Engineering (CASE), Vancouver, Canada, August 2019. [168] Pradeep Rajendran, Shantanu Thakar, Ariyan M. Kabir, Brual C. Shah, and S. K. Gupta. Context- dependent search for generating paths for redundant manipulators in cluttered environments. In IEEE International Conference on Intelligent Robots and Systems (IROS), Macau, China, November 2019. [169] Ramya Ramakrishnan, Chongjie Zhang, and Julie Shah. Perturbation training for human-robot teams. Journal of Articial Intelligence Research, 59:495{541, 2017. [170] Sarvapali D Ramchurn, Trung Dong Huynh, Feng Wu, Yukki Ikuno, Jack Flann, Luc Moreau, Joel E Fischer, Wenchao Jiang, Tom Rodden, Edwin Simpson, et al. A disaster response system based on human-agent collectives. Journal of Articial Intelligence Research, 57:661{708, 2016. [171] Jens Rasmussen. Human errors. a taxonomy for describing human malfunction in industrial installa- tions. Journal of occupational accidents, 4(2-4):311{333, 1982. [172] Nathan Ratli, Matt Zucker, J Andrew Bagnell, and Siddhartha Srinivasa. Chomp: Gradient opti- mization techniques for ecient motion planning. In 2009 IEEE International Conference on Robotics and Automation, pages 489{494. IEEE, 2009. [173] Yara Rizk, Mariette Awad, and Edward W Tunstel. Cooperative heterogeneous multi-robot systems: A survey. ACM Computing Surveys (CSUR), 52(2):1{31, 2019. [174] Robotic Industries Association. RIA TR15.306: Task-Based Risk Assessment Methodology. Standard, Robotic Industries Association, 2016. [175] Juan Rold an, Elena Pe~ na-Tapia, Andr es Mart n-Barrio, Miguel Olivares-M endez, Jaime Del Cerro, and Antonio Barrientos. Multi-robot interfaces and operator situational awareness: Study of the impact of immersion and prediction. Sensors, 17(8):1720, 2017. [176] Juan Jes us Rold an, Elena Pe~ na-Tapia, Pablo Garcia-Aunon, Jaime Del Cerro, and Antonio Barrien- tos. Bringing adaptive and immersive interfaces to real-world multi-robot scenarios: Application to surveillance and intervention in infrastructures. Ieee Access, 7:86319{86335, 2019. [177] Juan Jes us Rold an, Elena Pe~ na-Tapia, David Garz on-Ramos, Jorge de Le on, Mario Garz on, Jaime del Cerro, and Antonio Barrientos. Multi-robot systems, virtual reality and ros: Developing a new generation of operator interfaces. In Robot Operating System (ROS), pages 29{64. Springer, 2019. [178] Shady Salama and Amr B Eltawil. A decision support system architecture based on simulation opti- mization for cyber-physical systems. Procedia Manufacturing, 26:1147{1158, 2018. [179] Lindsay Sanneman and Julie A Shah. A situation awareness-based framework for design and evaluation of explainable ai. In International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems, pages 94{110. Springer, 2020. [180] Kristin E Schaefer, Edward R Straub, Jessie YC Chen, Joe Putney, and Arthur W Evans III. Com- municating intent to develop shared situation awareness and engender trust in human-agent teams. Cognitive Systems Research, 46:26{39, 2017. [181] Brent Schlotfeldt, Vasileios Tzoumas, Dinesh Thakur, and George J Pappas. Resilient active informa- tion gathering with mobile robots. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4309{4316. IEEE, 2018. 185 [182] Anthony R Selkowitz, Shan G Lakhmani, and Jessie YC Chen. Using agent transparency to support situation awareness of the autonomous squad member. Cognitive Systems Research, 46:13{25, 2017. [183] Scott Shappell, Cristy Detwiler, Kali Holcomb, Carla Hackworth, Albert Boquet, and Douglas A Wiegmann. Human error and commercial aviation accidents: An analysis using the human factors analysis and classication system. Human Factors, 49(2), 2007. [184] Scott A Shappell and DOUGLAS A WEIGMANN. Human factors analysis and classication system. Flight Safety Digest, 2001. [185] Raymond Sheh and Isaac Monteath. Introspectively assessing failures through explainable articial intelligence. In IROS Workshop on Introspective Methods for Reliable Autonomy, pages 40{47. iliad- project. eu Vancouver, Canada, 2017. [186] Sarah M Sherwood. The eect of task load, automation reliability, and environment complexity on uav supervisory control performance. 2018. [187] Iurii E Shishkin, Aleksandr N Grekov, and Vladimir V Nikishin. Intelligent decision support sys- tem for detection of anomalies and unmanned surface vehicle inertial navigation correction. In 2019 International Russian Automation Conference (RusAutoCon), pages 1{6. IEEE, 2019. [188] Vikram Shree, Beatriz Asfora, Rachel Zheng, Samantha Hong, Jacopo Ban, and Mark Campbell. Exploiting natural language for ecient risk-aware multi-robot sar planning. IEEE Robotics and Automation Letters, 6(2):3152{3159, 2021. [189] Shaurya Shriyam and Satyandra K. Gupta. Incorporating potential contingency tasks in multi-robot mission planning. In IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, May 2018. [190] Shaurya Shriyam and Satyandra K Gupta. Task assignment and scheduling for mobile robot teams. In International Design Engineering Technical Conferences and Computers and Information in Engi- neering Conference, volume 51807, page V05AT07A075. American Society of Mechanical Engineers, 2018. [191] Shaurya Shriyam and Satyandra K Gupta. Incorporation of contingency tasks in task allocation for multirobot teams. IEEE Transactions on Automation Science and Engineering, 17(2):809{822, 2019. [192] Shaurya Shriyam and Satyandra K Gupta. Modeling and verication of contingency resolution strate- gies for multi-robot missions using temporal logic. International Journal of Advanced Robotic Systems, 16(6):1729881419885697, 2019. [193] Shaurya Shriyam, Brual C Shah, and Satyandra K Gupta. On-line task decomposition for collaborative surveillance of marine environment by a team of unmanned surface vehicles. In International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, volume 58189, page V05BT08A019. American Society of Mechanical Engineers, 2017. [194] Pan Shu, Chen Min, Indu Bodala, Stefanos Nikolaidis, David Hsu, and Harold Soh. Human trust in robot capabilities across tasks. In Companion of the 2018 ACM/IEEE international conference on human-robot interaction, pages 241{242, 2018. [195] Dani ele Thomassin Singh. Incorporating cognitive aids into decision support systems: the case of the strategy execution process. Decision support systems, 24(2):145{163, 1998. [196] David Sirkin, Nikolas Martelaro, Mishel Johns, and Wendy Ju. Toward measurement of situation awareness in autonomous vehicles. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pages 405{415, 2017. [197] Kai-Tai Song, Sin-Yi Jiang, and Ming-Han Lin. Interactive teleoperation of a mobile manipulator using a shared-control approach. IEEE Transactions on Human-Machine Systems, 46(6):834{845, 2016. 186 [198] Prasanna Sridhar, Shahab Sheikh-Bahaei, Shan Xia, and Mo Jamshidi. Multi-agent simulation using discrete event and soft-computing methodologies. In SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme-System Security and Assurance (Cat. No. 03CH37483), volume 2, pages 1711{1716. IEEE, 2003. [199] Petr Stibinger, George Broughton, Filip Majer, Zden ek Rozsyp alek, Anthony Wang, Kshitij Jindal, Alex Zhou, Dinesh Thakur, Giuseppe Loianno, Tom a s Krajn k, et al. Mobile manipulator for au- tonomous localization, grasping and precise placement of construction material in a semi-structured environment. IEEE Robotics and Automation Letters, 6(2):2595{2602, 2021. [200] Ioan A Sucan, Mark Moll, and Lydia E Kavraki. The open motion planning library. IEEE Robotics & Automation Magazine, 19(4):72{82, 2012. [201] Balakumar Sundaralingam and Tucker Hermans. Relaxed-rigidity constraints: kinematic trajectory optimization and collision avoidance for in-grasp manipulation. Autonomous Robots, 43(2):469{483, 2019. [202] Aaquib Tabrez, Matthew B Luebbers, and Bradley Hayes. A survey of mental modeling techniques in human{robot teaming. Current Robotics Reports, pages 1{9, 2020. [203] Shantanu Thakar, Liwei. Fang, Brual Shah, and Satyandra K Gupta. Towards time-optimal trajec- tory planning for pick-and-transport operation with a mobile manipulator. In IEEE International Conference on Automation Science and Engineering (CASE), Munich, Germany, August 2018. [204] Shantanu Thakar, Ariyan Kabir, Prahar Bhatt, Rishi Malhan, Pradeep Rajendran, Brual Shah, and S. K. Gupta. Task assignment and motion planning for bi-manual mobile manipulation. In IEEE In- ternational Conference on Automation Science and Engineering (CASE), Vancouver, Canada, August 2019. [205] Shantanu Thakar, Pradeep Rajendran, Ariyan M. Kabir, and Satyandra K Gupta. Manipulator mo- tion planning for part pickup and transport operations from a moving base. IEEE Transactions on Automation Science and Engineering, 2020. [206] Shantanu Thakar, Pradeep Rajendran, Hyojeong Kim, Ariyan M Kabir, and Satyandra K Gupta. Accelerating bi-directional sampling-based search for motion planning of non-holonomic mobile ma- nipulators. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, USA, October 2020. [207] Shantanu Thakar, Srivatsan Srinivasan, Sarah Al-Hussaini, Prahar M. Bhatt, Pradeep Rajendran, Yeo Jung Yoon, Neel Dhanaraj, Rishi K. Malhan, Matthias Schmid, Venkat N. Krovi, and Satyandra K. Gupta. A Survey of Wheeled Mobile Manipulation: A Decision-Making Perspective. Journal of Mechanisms and Robotics, 15(2), 07 2022. 020801. [208] Nava Tintarev and Judith Mastho. Explaining recommendations: Design and evaluation. In Recom- mender Systems Handbook, pages 353{382. Springer, 2015. [209] Haluk Topcuoglu, Salim Hariri, and Min-You Wu. Performance-eective and low-complexity task scheduling for heterogeneous computing. IEEE transactions on parallel and distributed systems, 13(3):260{274, 2002. [210] Jerey N Twigg, Jason M Gregory, and Jonathan R Fink. Towards online characterization of au- tonomously navigating robots in unstructured environments. In 2016 IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS), pages 1198{1205. IEEE, 2016. [211] Chinonso Udokporo, Anthony Anosike, and Ming Lim. A decision-support framework for lean, agile and green practices in product life cycle stages. Production Planning & Control, 32(10):789{810, 2021. 187 [212] Robert Valner, Veiko Vunder, Andy Zelenak, Mitch Pryor, Alvo Aabloo, and Karl Kruusam ae. In- tuitive `human-on-the-loop'interface for tele-operating remote mobile manipulator robots. In Interna- tional symposium on articial intelligence, robotics, and automation in space (i-SAIRAS), pages 1{8, 2018. [213] Wouter Van Gansbeke, Davy Neven, Bert De Brabandere, and Luc Van Gool. Sparse and noisy lidar completion with rgb guidance and uncertainty. In 2019 16th international conference on machine vision applications (MVA), pages 1{6. IEEE, 2019. [214] Sergey Vasilyev, Vladimir Slabunov, Oleg Voevodin, and Alexandra Slabunova. Development of a decision support system at the stages of pre-design studies and design of irrigation systems based on idefo functional modelling methodology. Irrigation and Drainage, 69(4):546{558, 2020. [215] Edward Venator, Gregory S Lee, and Wyatt Newman. Hardware and software architecture of abby: An industrial mobile manipulator. In 2013 IEEE International Conference on Automation Science and Engineering (CASE), pages 324{329. IEEE, 2013. [216] Gregory W Vogl, Brian A Weiss, and Moneer Helu. A review of diagnostic and prognostic capabilities and best practices for manufacturing. Journal of Intelligent Manufacturing, 30(1):79{95, 2019. [217] Neng Wan, Lei Li, Chunming Ye, and Bo Wang. Risk assessment in intelligent manufacturing process: A case study of an optical cable automatic arranging robot. IEEE Access, 7:105892{105901, 2019. [218] Zheyuan Wang and Matthew Gombolay. Learning scheduling policies for multi-robot coordination with graph attention networks. IEEE Robotics and Automation Letters, 5(3):4509{4516, 2020. [219] Rebecca Wiczorek, Dietrich Manzey, and Anna Zirk. Benets of decision-support by likelihood versus binary alarm systems: Does the number of stages make a dierence? In Proceedings of the Hhuman Factors and Ergonomics Society Annual Meeting, volume 58, pages 380{384. SAGE Publications Sage CA: Los Angeles, CA, 2014. [220] R Williams. Collaborative multi-robot multi-human teams in search and rescue. In Proceedings of the International ISCRAM Conference, volume 17, 2020. [221] Choon Yue Wong and Gerald Seet. Workload, awareness and automation in multiple-robot supervision. International Journal of Advanced Robotic Systems, 14(3):1729881417710463, 2017. [222] Julia L Wright, Jessie YC Chen, and Michael J Barnes. Human{automation interaction for multi- ple robot control: the eect of varying automation assistance and individual dierences on operator performance. Ergonomics, 61(8):1033{1045, 2018. [223] Xuesu Xiao, Jan Dufek, and Robin R Murphy. Robot risk-awareness by formal risk reasoning and planning. IEEE Robotics and Automation Letters, 5(2):2856{2863, 2020. [224] Guang-Hui Xu, Feng Qi, Qiang Lai, and Herbert Ho-Ching Iu. Fixed time synchronization control for bilateral teleoperation mobile manipulator with nonholonomic constraint and time delay. IEEE Transactions on Circuits and Systems II: Express Briefs, 67(12):3452{3456, 2020. [225] Takashi Yamamoto, Koji Terada, Akiyoshi Ochiai, Fuminori Saito, Yoshiaki Asahara, and Kazuto Murase. Development of the research platform of a domestic mobile manipulator utilized for interna- tional competition and eld test. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7675{7682. IEEE, 2018. [226] Fan Yang and Nilanjan Chakraborty. Chance constrained simultaneous path planning and task as- signment for multiple robots with stochastic path costs. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pages 6661{6667. IEEE, 2020. [227] Bin Zhang, Liang Tang, Jonathan DeCastro, Michael Roemer, and Kai Goebel. Autonomous vehicle battery state-of-charge prognostics enhanced mission planning. International Journal of Prognostics and Health Management, 5(2), 2014. 188 [228] Cong Zhang, Wen Song, Zhiguang Cao, Jie Zhang, Puay Siew Tan, and Xu Chi. Learning to dispatch for job shop scheduling via deep reinforcement learning. Advances in Neural Information Processing Systems, 33:1621{1632, 2020. [229] Yiqi Zhang, Changxu Wu, and Jingyan Wan. Mathematical modeling of the eects of speech warning characteristics on human performance and its application in transportation cyberphysical systems. IEEE Transactions on Intelligent Transportation Systems, 17(11):3062{3074, 2016. [230] Yu Zhang and Lynne E Parker. Multi-robot task scheduling. In 2013 IEEE International Conference on Robotics and Automation, pages 2992{2998. IEEE, 2013. [231] Kai Zhou, Gerhard Ebenhofer, Christian Eitzinger, Uwe Zimmermann, Christoph Walter, Jos e Saenz, Luis P erez Casta~ no, Manuel Alejandro Fern andez Hern andez, and Jos e Navarro Oriol. Mobile ma- nipulator is coming to aerospace manufacturing industry. In 2014 IEEE international symposium on robotic and sensors environments (ROSE) proceedings, pages 94{99. IEEE, 2014. 189

Abstract (if available)

Abstract Human supervision is essential for deploying multi-robot teams in challenging missions, such as military applications and disaster rescue operations. Humans need to have high-level decision-making authority because of the safety-critical decisions and life-or-death situations in these missions. The major challenges are robotic failures, uncertainty in navigation and other task performances, and delayed and intermittent data flow due to restrictive communications. Whenever some mission update gets available to humans, they need to predict the upcoming and unknown mission states, adjust their plan rapidly, and retask robots if necessary. Since humans in supervisory roles in such missions are already under tremendous stress and emotional fatigue, they might not make the best decisions. My dissertation aims to provide computational foundations for risk assessment, alert generation, and robot tasking suggestion generation to assist human supervisors. Firstly, an alert generation framework is presented where the alert conditions are mathematically modeled to offer human users flexibility and extensibility. Secondly, a forward simulation-based approach is developed to estimate the probability of mission events under uncertainty in a computationally efficient manner. Next, a heuristics-based approach is presented for automated suggestion generation for robot retasking decisions to handle potential contingencies. Fourth, a human subject study is performed to evaluate the usefulness of the alerts and suggestions as cognitive aids in enabling faster and better decision-making by humans. Finally, human supervision is required in task-level operations with mobile manipulators in the presence of uncertainty, and this dissertation presents a hardware or software-agnostic framework for risk assessment, alert generation, and risk visualization for humans. I anticipate that this work will help improve human decision-making in supervising challenging operations involving robots.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Contingency handling in mission planning for multi-robot teams

PDF

Quality diversity scenario generation for human robot interaction

PDF

Decision support systems for adaptive experimental design of autonomous, off-road ground vehicles

PDF

Decentralized real-time trajectory planning for multi-robot navigation in cluttered environments

PDF

Robot trajectory generation and placement under motion constraints

PDF

Multiparty human-robot interaction: methods for facilitating social support

PDF

Speeding up trajectory planning for autonomous robots operating in complex environments

PDF

Coordinating social communication in human-robot task collaborations

PDF

Nonverbal communication for non-humanoid robots

PDF

Target assignment and path planning for navigation tasks with teams of agents

PDF

Efficiently learning human preferences for proactive robot assistance in assembly tasks

PDF

Process planning for robotic additive manufacturing

PDF

Robust loop closures for multi-robot SLAM in unstructured environments

PDF

Planning for mobile manipulation

PDF

Trajectory planning for manipulators performing complex tasks

PDF

Information theoretical action selection

PDF

Rethinking perception-action loops via interactive perception and learned representations

PDF

Artificial Decision Intelligence: integrating deep learning and combinatorial optimization

PDF

Data-driven acquisition of closed-loop robotic skills

PDF

Advancing robot autonomy for long-horizon tasks

Asset Metadata

Creator Al-Hussaini, Sarah (author)

Core Title Automated alert generation to improve decision-making in human robot teams

School Viterbi School of Engineering

Degree Doctor of Philosophy

Degree Program Computer Science

Degree Conferral Date 2023-05

Publication Date 02/22/2023

Defense Date 12/06/2022

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag artificial intelligence,human-robot interactions,intelligent decision support system,multi-robot mission,OAI-PMH Harvest

Format theses (aat)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Gupta, Satyandra K. (committee chair)

Creator Email salhussa@usc.edu,st.alhussaini@gmail.com

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC112732975

Unique identifier UC112732975

Identifier etd-AlHussaini-11476.pdf (filename)

Legacy Identifier etd-AlHussaini-11476

Document Type Dissertation

Format theses (aat)

Rights Al-Hussaini, Sarah

Internet Media Type application/pdf

Type texts

Source 20230222-usctheses-batch-1007 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu