Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Contingency handling in mission planning for multi-robot teams
(USC Thesis Other)
Contingency handling in mission planning for multi-robot teams
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
CONTINGENCY HANDLING IN MISSION PLANNING FOR MULTI-ROBOT TEAMS by Shaurya Shriyam A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulllment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (Mechanical Engineering) December 2019 Copyright 2019 Shaurya Shriyam Dedication This dissertation is dedicated To my parents, Shreyash, school teachers, and college professors. ii Acknowledgements Doing a Ph.D. degree is an excellent academic journey, and I am very thankful to the University of Maryland, College Park, and the University of Southern California for providing me with an opportunity to experience it. The robotic labs I worked within both the universities had state-of-the-art robots and greatly stimulated my research progress. I would like to thank my advisor, Professor S. K. Gupta, for his constant motivation and terric problem-solving skills, and guiding me through the challenges that I faced during my doctoral program. One faces the regular challenges during the completion of a doctoral program such as coming up with new ideas, producing results worthy of publication, and so on. My sincere appreciation goes to my supervisor for his excellent supervision during these ve years. I would like to thank my committee members, Professor Azad Madni, and Professor Yan Jin, for serving in my committee. They provided helpful advice to improve my dissertation. Specically, their guidance was useful for me to improve the structure and writing style of this dissertation I would like to acknowledge the nancial support provided by the National Science Foundation (NSF) towards the completion of my research goals accumulating into this nal dissertation. Although Robotic Smart Assistant for Manufacturing project is not part of this dissertation, it was jointly done with Professor Krishna Kaipa, and it was a great experience to work with the Baxter robot. I would like to thank all of my labmates with whom I interacted and worked during my research program. The discussions that I had with Brual, Ariyan, Pradeep, Shantanu, Rishi, Jason, Josh, iii Sarah, Aniruddha, Nithyanand, Akshay, and Srudeep signicantly helped improve my research work. Finally, I would like to thank my family for their support across multiple continents, especially to my Mom who always had faith in me, and to my Dad who always gave me incredible words of encouragement, and to my Brother who took great interest in my research work and gave interesting feedback. iv Table of Contents Dedication ii Acknowledgements iii List Of Tables viii List Of Figures ix Abstract xii Chapter 1: Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Goal and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.3 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Chapter 2: Literature Review 11 2.1 Mission Planning Using Model Checking . . . . . . . . . . . . . . . . . . . . . . . . 11 2.1.1 Multi-robot Mission Modeling for Model Checking . . . . . . . . . . . . . . 11 2.1.2 Model Checking Tools for Verication and Evaluation . . . . . . . . . . . . 16 2.1.3 Handling Stochastic Events . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.1.4 Contingency Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 2.2 Multi-robot Task Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.3 Decomposition of Exploration Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Chapter 3: Modelling and Verication of Contingency Resolution Strategies for Multi-robot Missions Using Temporal Logic 42 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3 Overview of Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.3.1 Nominal Mission Modeling for Deterministic Transitions . . . . . . . . . . . 44 3.3.2 Nominal Mission Modeling for Probabilistic Transitions . . . . . . . . . . . 47 3.3.3 Modeling of Contingency Resolution Strategies . . . . . . . . . . . . . . . . 49 3.3.4 Model Checking and Verication . . . . . . . . . . . . . . . . . . . . . . . . 51 3.3.4.1 Deterministic Transition Systems . . . . . . . . . . . . . . . . . . 51 3.3.4.2 Probabilistic Transition Systems . . . . . . . . . . . . . . . . . . . 52 3.4 Task Network Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 v Chapter 4: Multi-USV Case Study 56 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.3 Mission Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 4.4 Model Checking and Verication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Chapter 5: Multi-robot Assembly Case Study 76 5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 5.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.3 Overview of Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.4 Modeling of Assembly Cell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.4.1 Nominal Operation Description . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.4.2 State Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.4.3 Transition Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 5.4.4 Nominal Operation Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . 89 5.4.5 Contingency Resolution Strategies . . . . . . . . . . . . . . . . . . . . . . . 90 5.5 Model Checking and Verication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.5.1 Run-time Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.5.2 Assembly Cell Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 5.5.3 Aggregate Analysis of Large-scale Assembly Operations . . . . . . . . . . . 99 5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Chapter 6: Incorporation of Contingency Tasks in Nominal Task Allocation 104 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 6.2.1 Environment Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.2.2 Resource Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.2.3 Task Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.2.3.1 Mission Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.2.3.2 Contingency Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 6.2.4 Nominal Mission Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 6.3 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 6.4 Task Schedule Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 6.4.1 Task Execution by Partial Teams . . . . . . . . . . . . . . . . . . . . . . . . 123 6.4.2 Handling Divisible Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 6.4.3 Task Scheduling Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 6.4.3.1 Heuristics Employed . . . . . . . . . . . . . . . . . . . . . . . . . . 129 6.4.3.2 Task Allocation Without Task Division . . . . . . . . . . . . . . . 131 6.4.3.3 Task Allocation with Task Division . . . . . . . . . . . . . . . . . 133 6.4.4 Accounting for Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 6.5 Incorporating Potential Contingency Tasks . . . . . . . . . . . . . . . . . . . . . . 138 6.5.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 6.5.2 Search-space Pruning Heuristic . . . . . . . . . . . . . . . . . . . . . . . . . 141 6.5.3 Analysis of Pruning Heuristic . . . . . . . . . . . . . . . . . . . . . . . . . . 142 6.5.4 Contingency Handling Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 143 6.6 Results and Illustrations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 6.6.1 Multi-robot Task Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 6.6.2 Contingency Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 6.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 vi Chapter 7: Decomposition of Collaborative Surveillance Tasks with Uncertainty in Environmental Conditions and Robot Availability 164 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164 7.2 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 7.2.1 Denitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 7.2.2 State Space Representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 7.2.3 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 7.3 Overview of Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 7.3.1 Partitioning the Region of Interest . . . . . . . . . . . . . . . . . . . . . . . 170 7.3.2 Optimizing Partition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 7.4 Area Partitioning Using Known Velocity-map . . . . . . . . . . . . . . . . . . . . . 180 7.4.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 7.4.2 Computational Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186 7.5 Area Partitioning Using Unknown Velocity-map . . . . . . . . . . . . . . . . . . . . 190 7.5.1 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 7.5.2 Computational Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191 7.6 Area Partitioning Under Variable Robot Availability . . . . . . . . . . . . . . . . . 193 7.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Chapter 8: Conclusions 196 8.1 Intellectual Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 8.1.1 Modelling and Verication of Contingency Resolution Strategies for Multi- robot Missions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197 8.1.2 Incorporation of Contingency Tasks in Nominal Task Allocation . . . . . . 198 8.1.3 Decomposition of Collaborative Surveillance Tasks . . . . . . . . . . . . . . 198 8.2 Anticipated Benets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198 8.3 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Reference List 201 vii List Of Tables 4.1 Runtime statistics collected for dierent problem sizes . . . . . . . . . . . . . . . . 74 5.1 Eect of initial part buer on production time . . . . . . . . . . . . . . . . . . . . 93 5.2 Eect of contingency resolution probability on production time . . . . . . . . . . . 93 5.3 Computational performance of s 1 with varying parameters . . . . . . . . . . . . . . 94 5.4 Computational performance of s 2 with varying parameters . . . . . . . . . . . . . . 95 5.5 Computational performance of s 3 with varying parameters . . . . . . . . . . . . . . 96 5.6 Eect of human characteristics on production time for strategy s 1 . . . . . . . . . 96 5.7 Eect of human characteristics on production time for strategy A . . . . . . . . . . 96 6.1 Computational tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 6.2 Motion tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 6.3 Overview of strategies used for task allocation . . . . . . . . . . . . . . . . . . . . . 132 6.4 Dierent types of penalty associated with available actions for contingency tasks . 139 6.5 Analysis of various task networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 6.6 Benchmarking strategy S 2 against S 6 and S 7 . Table entries are mission makespan in minutes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 6.7 Description of scenarios used for experiments in Table 6.8 . . . . . . . . . . . . . . 149 6.8 Results for ten simulation runs with task division . . . . . . . . . . . . . . . . . . . 150 6.9 Eect of uncertainty on mission makespan . . . . . . . . . . . . . . . . . . . . . . . 155 viii List Of Figures 1.1 This is an outline of the research work investigated in the dissertation. . . . . . . . 10 3.1 Characterization of multi-USV escorting mission scenario (Not drawn to scale) . . 44 4.1 Characterization of multi-USV escorting mission scenario (Not drawn to scale) . . 57 4.2 Kripke modeling for the ship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 4.3 Kripke modeling for a USV in the absence of contingencies of type A and B . . . . 64 4.4 Strategy for handling communication failure . . . . . . . . . . . . . . . . . . . . . . 65 4.5 Kripke modeling for a USV with three way-points only . . . . . . . . . . . . . . . . 66 4.6 Resolution strategy for contingencies of type A . . . . . . . . . . . . . . . . . . . . 69 5.1 Three manipulators assigned for cooperative assembly operation . . . . . . . . . . 78 5.2 Mobile manipulator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 5.3 CAD model for the assembled satellite mock-up . . . . . . . . . . . . . . . . . . . . 79 5.4 Exploded view of the CAD model for the satellite's mock-up . . . . . . . . . . . . 80 5.5 Schematic representation of multi-robot assembly cell . . . . . . . . . . . . . . . . 81 5.6 Probabilistic transition model for assembly robot . . . . . . . . . . . . . . . . . . . 84 5.7 Schematic representation of assembly processes in station S i for i > 1 for strat- egy s 1 . The rectangles represent storage containers, and the rounded rectangles represent processes involved during assembly. . . . . . . . . . . . . . . . . . . . . . 87 5.8 Eect of dierent strategies on expected production time . . . . . . . . . . . . . . 95 5.9 Eect of dierent combinations of stations on expected production time. . . . . . . 99 5.10 Scenario for multiple assembly cells interacting with each other . . . . . . . . . . . 100 ix 6.1 Example involving USVs where a contingency task impacts the mission . . . . . . 105 6.2 Example of task precedence graph . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 6.3 General rule for graph renement to incorporate a contingency task . . . . . . . . 118 6.4 Rule for graph renement when the contingency task was not properly incorporated in the nominal mission plan and encountered during mission execution . . . . . . . 118 6.5 Overview of mission planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 6.6 Task networks based on neural networks used for simulation experiments in Table 6.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 6.7 Performance of list scheduling heuristics compared to MinStepSum and MinInter- fere from [1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 6.8 Application of strategyS 4 on task networks (a,c), and corresponding robot sched- ules (b,d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151 6.9 Application of strategyS 4 on task networks (a,c), and corresponding robot sched- ules (b,d) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 6.10 Eect on performance of strategies by varying task duration compared to traveling duration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 6.11 Computational performance of strategyS 4 . . . . . . . . . . . . . . . . . . . . . . . 154 6.12 (a) Mission scenario, (b) Robot schedules obtained without considering contingency tasks, (c) Incorporating reported contingency tasks in robot schedules, (d) Replan- ning schedules for robots when probability of contingency tasks exceed desired limits157 6.13 (a) Mission scenario, (b) Robot schedules obtained without considering contingency tasks, (c) Incorporating reported contingency tasks in robot schedules, (d) Replan- ning schedules for robots when probability of contingency tasks exceed desired limits158 6.14 Comparison of time spent during task execution and otherwise for problems of (a) Fig. 6.12a, (b) Fig. 6.13a . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 6.15 Comparison of conservative (C), reactive (R) and proactive (P ) approaches . . . . 160 6.16 Variation of running time with number of concurrently reported contingency tasks for strategyS 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 6.17 Replanning trigger conditions obtained from sensitivity analysis . . . . . . . . . . . 162 7.1 This is an illustration of a port scenario where a region has been demarcated for exploration by multiple USVs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 7.2 (a) This image shows the example scenario discussed in Figure 7.3, (b) Contour- lines based velocity-map is used to model low speeds for USVs near land. . . . . . 173 x 7.3 In the above images, the brown cells are outside the region of interest, the light blue cells represent the free, unexplored region and the dark blue cells represent regions already claimed by an USV for exploration. (c) The green cells represent the frontier cells for iteration 29. The scheme used for the above exploration is explained in detail in Section 7.3.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 7.4 (a-b) Uniform, constant velocity-map, (c-d) Linear velocity-map . . . . . . . . . . 184 7.5 (a) Linearly graded velocity-map, (b-d) Application on manually created regions. . 185 7.6 Application of the linearly graded velocity-map on regions based on real maps. . . 186 7.7 (a) Polar velocity-map, (b-d) Application on manually created regions. . . . . . . . 187 7.8 Application of the polar velocity-map on regions based on real maps. . . . . . . . . 188 7.9 (a) Contour-lines based velocity-map, (b) Corresponding area partition . . . . . . . 189 7.10 (a) This image shows the range of times required by the USVs for exploration of given regions from over 200 simulations in the form of box-plot, (b) Here, the variation of computational time required for generating area partitions with respect to the number of USVs involved in exploration is shown. . . . . . . . . . . . . . . . 190 7.11 In the absence of correct velocity-map, one can either use (a) constant map, or (b) noisy map; and record the time taken to complete partitioning the region based on how much time into the area partitioning process the correct map becomes available. The observed trend is explained in detail in Section 7.5. . . . . . . . . . 192 7.12 Eect of a robot becoming unavailable in the middle of task execution on the time to partition a (a) convex , and (b) nonconvex region . . . . . . . . . . . . . . . . . 194 xi Abstract In order to be deployed in real-world applications, autonomous multi-robot systems need to be able to handle contingency events. Many multi-robot missions are time-critical, so using overly conservative methods to handle contingencies will not be acceptable in such applications. Robots themselves may be prone to unreliable task execution and may require human assistance for handling such contingencies, which must be properly planned and coordinated. This dissertation models the handling of contingency events in the form of execution of contingency tasks. When an unexpected event occurs that adversely impacts the mission work ow, then the multi-robot system, with or without the guidance of human operators, must perform contingency tasks to ensure the safety and completion of the mission. Contingency tasks can arise due to robot failure occurring due to battery failure or communication lapse. Contingency tasks may also occur explicitly as an additional task that becomes a bottleneck for nominal mission execution. For example, if an autonomous vehicle were to run out of fuel, the vehicle must be taken to a lling center to refuel it. Additional tasks that may be required for mission completion are referred to as contingency tasks. Contingency tasks may also arise due to uncertain operating conditions or bad information about operating conditions. The general approach to handling contingencies in robotics has been to design failure recovery protocols which kick in when a specic contingency occurs. However, such a reactive approach does not work well in time-critical missions. This dissertation presents the computational foundations for proactively handling contingen- cies encountered during multi-robot missions. A formal model checking framework is developed to evaluate the adequacy of contingency resolution strategies that may be encountered during xii multi-robot missions. In this dissertation, the developed framework was successfully applied to an assembly line setup and a ship escorting mission which used unmanned surface vehicles, and the performance of the strategies designed to handle dierent contingencies was evaluated. This dissertation also presents a proactive approach for incorporating contingency tasks in nominal mission planning. It was found through simulation experiments that the proactive approach out- performed the reactive and conservative approaches. Finally, the dissertation proposes a method for task decomposition required for completing collaborative surveillance tasks assigned to mul- tiple robots under the conditions of uncertain information and varying robot availability. It was found that the developed methods for task decomposition were robust against these uncertainties and were able to successfully guide the robots to successful task completion while minimizing the time taken. xiii Chapter 1 Introduction 1.1 Motivation In recent years, unmanned robots have been increasingly used for a variety of operations, such as inspection of agricultural elds, search and rescue, surveillance, and defense applications [2{5]. In many of the applications, deploying a team of robots instead of a single robot leads to automation of mundane and critical activities which leads to higher eciency, increased productivity, safer working conditions, robust operations and reduced costs in the long-term. Multi-robot teams are also increasingly being used to provide support in complex industrial operations [6{12]. Most of these applications require the execution of spatially separated tasks. Multi-robot teams are well-suited to such applications because they can execute multiple such tasks in parallel. Multiple robots can divide each task into sub-tasks, which can then be independently accomplished by each robot or the robots can work as a team and assist each other with dierent sub-tasks. Researchers have extensively examined the area of collaborative robotics [13{19]. Even in manufacturing applications, active research is going on to incorporate multi-robot setups in factory oors to boost operational eciency and bring down costs [20{28]. The motivation for modeling such multi-robot missions is due to the missions involving com- plex sequences of survey-like tasks involving unmanned surface and ground vehicles. Teams (or 1 coalitions) of multiple robots are being extensively employed for automation projects and complex missions in domains such as factory cells, outdoor survey/inspection missions, and search-and- rescue missions. Multi-robot missions require a set of objectives to be fullled. This in turn translates into a sequence of tasks that must be executed by each robot. Each robot plans its trajectories to execute these tasks. Such nominal mission planning techniques have been investigated in detail and generally yield good results. However, in more realistic scenarios where nominal mission plans have to be modied due to changes in the work environment, traditional methods of nominal mission planning do not perform very well. Some unexpected event may occur causing bottleneck which requires an additional task to be executed to safely resume the mission. It may also happen that robots may fail to perform a task that they were assigned to nish and hence must again attempt to execute the task. Sud- den changes in the environmental information associated with the mission scenario may require improving the current mission execution plan to ensure better performance. Mission execution must also be robust to varying availability of robots assigned to a mission. Such bad and unex- pected events are referred to as contingencies. This dissertation is focused mainly on handling contingency situations for multi-robot systems eectively. The developed methods should be able to handle contingencies in real-time so that multi-robot teams can be deployed for challenging missions. The developed methods should proactively account for potential contingencies to ensure that plans can eciently deal with contingencies. There are several civilian applications where proactive handling of contingencies for multi- robot autonomous systems can signicantly bring down mission costs while boosting the guarantee of safety for mission infrastructure as well as humans involved, thereby leading to the more rapid deployment of autonomous systems in diverse areas. Active research is taking place in making the robots better prepared to handle unfavourable conditions that arise during mission execution [29{33]. There are several multi-robot applications [34] which may potentially benet 2 from better and proactive handling of contingencies. A few representative applications are enlisted below: 1. Infrastructure security: Many critical facilities require protection and constant pa- trolling. However, hiring enough human guards is just not economically viable in the long-term [35]. Neither are robotic technologies advanced enough to completely hand over the guarding and patrolling jobs to autonomous systems. Human operators embedded in multi-robot systems provide the best solution. However, there are possibilities for several unexpected adverse conditions that may interrupt the security missions of the robots [36]. Improved methods of handling contingencies prove to be very benecial. 2. Hazardous Waste Cleanup: Sending humans for nuclear waste cleanup puts the lives of the workers at risk due to radioactive exposure and is highly unethical, if not inhumane. In the Fukushima Daiichi nuclear disaster, elderly pensioners volunteered to participate in the cleanup of the power station, thus saving young men from receiving a fatal dose of radiation. Given the prevalence of nuclear plants throughout the world and the trend getting stronger and stronger with more countries enthusiastically signing up for the program, a concerted and serious eort must be directed to automating nuclear plant operations. This may require handling situations such as unexpected communication failures and unexpected failure of robot's batteries [37]. 3. Urban search and rescue: In dangerous disaster scenarios, there is extreme time pressure on the rescue workers and dogs to locate injured civilians and rescue them. If robots can replace humans and dogs, it will reduce the risk to workers' lives. But there is a risk of gas leaks and explosions happening unexpectedly or debris falling from a construction. There is signicant research eort underway to achieve robotic search and rescue [38]. 4. Management and transportation of large containers: In ports, airports, and railway yards, there is a frequent requirement of transporting heavy containers from one location to 3 another. Since these places are unstructured environments, there is a lot of potential for un- expected events to interrupt or disrupt the mission. If a team of robots is to be employed to conduct all container-related tasks, then safety features in the form of contingency handling is a must [39,40]. 5. Space exploration: NASA has laid down exciting visions for space exploration in the near future. One of its objectives is to increase the use of robots for construction, repair, and maintenance purposes in space stations and spaceships. Robots may be used additionally to provide assistance or companionship to the space crew. However, this is quite challenging because these robots have to operate in a sensitive environment. Any chemical leak, elec- tronics failure, computer failure, or collision with big space debris could be fatal for the space crew and cause a loss of several billion dollars. However, research progress is continually being made in this sphere [41]. However, incorporating contingencies in multi-robot mission planning suers from several chal- lenges: 1. There is the modeling challenge to decide what information to abstract out or hide from ex- plicitly considering in the mission model so that the eects of contingencies can be analyzed without making the model computationally very expensive. 2. If failures occur during multi-robot missions, then they may result in human injury or damage to expensive equipment. Thus, the contingency planning methods must also seek to guarantee safe mission completion. 3. It is challenging to use a model checking framework to assess the adequacy of contingency resolution strategies for multi-robot mission because it is computationally more time-taking than simulation testing. However, to seek robust guarantees in a rigorously proved mathe- matical framework, one should make eective use of the model checking toolkits developed by the formal methods community. 4 4. It is dicult to allocate tasks to robots under uncertainty when disrupting events such as contingencies are expected. On top of that, wishing to minimize the expected mission completion time in a computationally ecient manner requires developing heuristics that guide the solution search properly. 5. It must be taken into account how robots distribute a task assigned to them in an ecient manner. Use of computationally heavy methods may provide the optimal decomposition of a task among multiple robots but would be so time-taking that they cannot be deployed online. So a planning approach must be able to handle adverse eects like the uncertainty of task execution and completion, or varying robot availability, in a faster way even if it implies that only a near-optimal distribution of tasks among robots can be obtained. 1.2 Goal and Scope The research presented in this dissertation is concerned with developing proactive approaches to handling contingencies for multi-robot teams that have been assigned to a complex mission. The research work presented in this dissertation focuses on contingencies that can be anticipated but their occurrence cannot be predicted in advance. The methods described in the rest of the work are designed for robot team sizes ranging from two to twenty robots. The main research issues this work addresses are as follows: 1. Modelling and verication of contingency resolution strategies for multi-robot missions: An approach for evaluating contingency resolution strategies using temporal logic is pre- sented. A framework is presented for nominal mission modeling, designing contingency resolution strategies and evaluating their eectiveness for the mission. The approach in this dissertation focuses on leveraging the use of model checkers to the domain of multi-robot missions to assess the adequacy of contingency resolution strategies to minimize the adverse 5 eects of contingencies on the mission execution. The missions are allowed to have deter- ministic as well as probabilistic transitions. The use of formal methods and model checking software packages allows the development of autonomous systems with guarantees against faulty behaviors and inconsistencies. Such autonomous systems may be deployed on industrial scales and would make processes more ecient and streamlined. For this purpose, two completely dierent case studies are chosen and the application of model checking software is shown to provide guarantees of safety and success for complex multi-agent missions. A nominal mission model is proposed and contingency resolution strategies are developed in both scenarios. Then the results obtained via model checkers are analyzed. It is not possible to generate such logical results as well as expected values of critical mission variables so easily in any other way than modeling a mission as a formal transition system and encoding it in a model checker. This is because nominal mission specications, contingency models, and contingency resolution strategies may need to be changed repeatedly to decide what works best. A methodology that al- lows exibility in eciently evaluating these possibilities in a mathematically rigorous way is bound to outperform conventional methods like running heavy simulations in memory- intensive software tools. These methods can be exploited to quickly evaluate contingency resolution strategies and screen them for their performance. Once some good ways have been identied to handle contingencies for a mission, then one can resort to simulation tech- niques to study the strategies in ner details. Thus, there will be no need for eliminating simulation testing paradigm. The two case studies are as follows. The rst one considers the escorting of a ship in a port where multiple contingencies may occur concurrently and analyze the proposed contingency resolution strategies. The second one considers a manufacturing scenario where multiple assembly stations collaborate to create a product. In this case, assembly operations may 6 fail, and human intervention is needed to complete the product. Several dierent strategies are investigated and their eectiveness is assessed based on mission characteristics. This modeling framework is extended to a production line set up across multiple robotic assembly cells by integrating probabilistic modeling and temporal logic verication tech- niques. A production line is a complex system comprising of multiple assembly cells which in turn is composed of multiple assembly stations. The performance of the proposed oper- ations plan is evaluated by decomposing the whole system into smaller subsystems which are analyzed separately. This may help in designing more optimized systems. An assembly cell is modeled in PRISM software, which is a commonly used probabilistic model checking tool. Contingencies may arise in each cell during the production operations and are resolved by humans or the assembly robots themselves. Multiple cells operating cooperatively to nish the production of parts are then simulated. Formal analysis is performed for each cell, which is a subsystem of the production line and then the aggregate behavior of the resulting production line is studied. Based on the feedback from PRISM analysis of individual cell behavior, it can be determined how to change cell settings to achieve the best production performance. 2. Incorporation of Contingency Tasks in Nominal Task Allocation: Complex logistics support missions require execution of spatially separated information gath- ering and situational awareness tasks. Mobile robot teams can play an important role in the automated execution of these tasks to reduce mission completion time. Planning strategies for such missions must take into account the formation of eective coalitions among avail- able robots and assignment of tasks to robots to minimize the expected mission completion time. The occurrence of unexpected situations that adversely interfere with the execution of the mission may require the execution of contingency tasks so that the originally planned tasks may proceed with minimal disruption. 7 The work reported in this dissertation will be useful for deploying multi-robot teams to sup- port complex logistics missions spread over a large area where the robots must be prepared to handle contingencies that can adversely impact the mission. The proposed proactive approach can be used to handle contingencies in information gathering, surveillance, guard- ing, and situational awareness tasks to support safe and secure transportation of important assets through sensitive areas. A port operation is used as an illustrative example where unmanned surface and aerial vehicles can be useful in ensuring the safety and security of the ports. This is a computationally challenging problem. This work proposes heuristics algorithms to solve the task allocation problem among many dierent agents eciently. The approach presented in this work integrates the available information regarding mission and contingencies along with the resource constraints to plan the mission execution. Initially reported potential contingency tasks may not always aect mission tasks owing to the uncertainty in the mission environment. When potential contingency tasks are re- ported, the planner updates its existing plan to minimize the expected mission completion time based on the probability of these contingency tasks impacting the mission, their im- pact on the mission and other task characteristics. Various heuristic-based strategies are described to compute task-allocations for robots for mission execution. Simulation exper- iments are performed to compare them and the computational performance of the best performing strategy is analyzed. It is shown that the proactive approach to contingency task management outperforms both the conservative and reactive approaches. 3. Decomposition of Collaborative Surveillance Tasks: This dissertation introduces an approach for decomposing exploration tasks among multiple Unmanned Surface Vehicles (USVs) in congested regions. To ensure the eective distribution of the workload, the algorithm has to consider the eects of the environmental constraints on the USVs. The performance of a USV is in uenced by the surface currents, the risk of 8 collision with the civilian trac, and varying depths due to tides, and weather. The team of USVs needs to explore a certain region of the harbor, and an algorithm must be developed to decompose the region of interest into multiple sub-regions. The algorithm overlays a 2D grid upon a given map to convert it to an occupancy grid and then proceeds to partition the region of interest among the multiple USVs assigned to explore the region. During partitioning, the rate at which each USV can travel varies with the applicable speed limits at the location. The objective is to minimize the time taken for the last USV to nish exploring the assigned area. The particle swarm optimization method is used to compute the optimal region partitions. The method is veried by running simulations in dierent test environments. The performance of the developed method is also analyzed in environments where speed restrictions are not known in advance. 1.3 Overview The outline of this dissertation can be brie y summed up using the block diagram in Figure 1.1. To incorporate contingencies in nominal mission planning, three stages are considered: 1. The given mission model, usually provided in English, must be translated into a formal model. The modeling must also include the contingency model describing the contingencies that may occur and the nature of their occurrence. Using this information, contingency resolution strategies are designed manually. The objective is to select the most eective contingency resolution strategies by verifying their desired properties. For this purpose, model checking software packages are used. The results of model checking may also be used to modify the strategies to improve their expected performance. 2. A nominal mission planner is also developed in parallel that takes in the nominal mission model and generates a task precedence graph comprising of the order in which dierent tasks must be performed by the available robots. But the execution of dierent tasks must be 9 Figure 1.1: This is an outline of the research work investigated in the dissertation. allocated to the agents and scheduled in such a way so that the expected mission completion time is minimized. Concurrently, the contingency information must be taken into account and proactively integrated into the task network. The resulting task allocation then species which task must be executed in what order by which robots. 3. When multi-robot tasks are scheduled for execution, then multiple robots are also assigned to the task. These robots must coordinate to nish the task optimally and eciently. Con- tingencies may arise due to imperfect information, or bad information about the operating conditions of the task, or robots becoming suddenly unavailable for task execution due to being diverted to another task of higher priority. This entire information must be incorpo- rated to decompose the task among several robots for ecient execution without interference and bottlenecks. Such a task decomposition results in task schedules for individual robots. These individual robot task allocations are easier to implement and can be monitored by onboard computers. 10 Chapter 2 Literature Review This chapter provides a literature review of methods used for multi-robot mission planning that is used in this dissertation for incorporating contingencies. The methods discussed here are those from the domains of task allocation, scheduling, temporal logic based model verication, and task decomposition. 2.1 Mission Planning Using Model Checking 2.1.1 Multi-robot Mission Modeling for Model Checking Model checking essentially helps in verifying the robustness of missions against contingency scenar- ios. Other researchers have investigated model checking for formulating constraints for automated multi-agent systems and then checking for any inconsistency [42,43]. Most model checking work in robotics generally focuses on properties involving temporal logic because these are the most commonly queried and desirable properties for ascertaining the robustness and correctness of the system in the robotics community. Temporal logic properties are widely expressed using compu- tation tree logic (CTL), linear temporal logic (LTL), or a combination of both. For stochastic processes, system specications make use of probabilistic temporal logic like Probabilistic Com- putation Tree Logic (PCTL). 11 Bolton et al. [44] survey the usage of formal model verication methods in improving human- automation interaction (HAI). Such autonomous systems may suer failures due to being brittle, and hence, unexpected scenarios may overwhelm the human operator. The cause of such failures lies mostly in the design of the automation software. Identifying shortcomings in the design of such automation systems and their interfaces for human operators allows automation to become more robust. A major problem is associated with mode confusion where human fails to keep track of the mode in which the device or the automation system is performing. Over more extended periods of times, it leads to human getting confused and adopting random guessing strategy or some other incoherent strategy to keep guiding/controlling the system. This may lead to the surprise of the human operator eventually and cause disruption in the work output. It is therefore essential to take into account the human mental model by explicitly or implicitly including it in the transition system representation of the human-automation interface. The paper lays out the dierent kinds of formal properties generally veried by other researchers. Temporal logic-based model verication methods oer several benets when deployed for plan- ning at a higher level of abstraction, viz. the task and mission planning layer of autonomous multi-robot systems. Pant et al. [45] use Signal Temporal Logic (STL) for specifying missions involving multiple drones that must take o and land from designated spots, must monitor the environment, reach a location while avoiding obstacles, perform periodic surveillance and data collection, collect sensor data from equipment tted across a smart city. A simulation was carried out using up to 16 drones and actual experiments using two drones to check out the applicabil- ity of their method. They use unconstrained optimal control techniques to compute trajectories that are dynamically feasible and can be tracked eciently using low-level o-the-shelf position and attitude controllers. They formulated their optimization problem using CasADi symbolic framework designed for automatic dierentiation and optimal control and used the HSL MA27 FORTRAN routine with IPOPT for solving the nonlinear program. They compared their method 12 with another STL-based trajectory generator called BluSTL, which took more time to solve the problem. Liu et al. [46] encode mission specication for multi-agent systems with wireless communica- tion constraints using Signal Temporal Logic (STL) and spatial-temporal logic (SpaTeL) formulas. Spatial-Temporal Logic (SpaTeL) is a unication of Signal Temporal Logic (STL) and Tree Spa- tial Superposition Logic (TSSL). SpaTeL is capable of describing high-level spatial patterns that change over time. The robot environment is a 2D grid area which is represented using a quad- tree structure. A MILP optimization program is used to develop a distributed model predictive control (MPC) based strategy to satisfy the mission constraints. In their second work [47], they require multi-agent systems with inter-agent communication to satisfy motion specication in the form of signal temporal logic formulas. It is the task of a local motion controller to ensure that its associated agent satises certain user-dened local specications while avoiding collisions with other agents and obstacles. It is also important to maintain high communication quality among the agents. A Gaussian fading model is used to model wireless communication signals. The sig- nal temporal logic specications and wireless communication conditions are expressed as mixed integer-linear (MIL) constraints in terms of agents' state variables and communication signal sta- tus. Many simulations carried out using MATLAB, and AMPL/Gurobi show that their strategy improves the communication quality without obstacle collision. Nikou et al. [48] perform controller synthesis for multi-agent path planning where Metric Interval Temporal Logic specications are given. Each agent has a dynamic model capturing the coupling eects as well as control inputs. A ve-step controller synthesis algorithm is presented, which is tested out in simulation for time-bounded constraints of traveling to a region. Zhou et al. [49] present an optimization-based method for planning paths of a mobile robot for which bounded temporal constraints are specied using metric temporal logic (MTL). The metric temporal logic formulas are translated into mixed integer linear constraints and then solved using YALMIP- CPLEX software. They provided simulation examples that included static as well as moving 13 obstacles based on quadrotors and cars. High-level strategies provided by non-experts are encoded as preferences using the Planning Domain Denition Language (PDDL) 3:0 language by Kim et al. [50] and used by automated planners. MacKenzie et al. [51] use behavior-based modeling of multi-agent intelligent systems using societal agent theory. Firstly, the primitive robot behaviors are abstracted into individual agents, and then assemblages are created out of them. Assemblages are groups of fundamental behaviors and coordination mechanisms among these agents to ensure coherent behavior. The Conguration Description Language (CDL) is developed to encode these agent behaviors and makes use of their recursive nature. It also allows the same agents to be used in dierent missions using dierent robots. The transition from one state to another is modeled using a nite state automaton to select valid and feasible states that may be accessed by robots. Lomuscio and Raimondi [52] consider deterministic, non-deterministic, and other classes of interpreted systems which are formal models to dene multi-agent systems and interpret the semantics of Alternating-time Temporal Logic (ATL) operators in their context. The rst class of agents may guess moves, the second class consists of deterministic agents, and the third class of agents chooses to perform specic actions consistently. Finally, they present a model checking algorithm based on Ordered Binary Decision Diagrams (OBDDs) which users can apply to their problems using the MCMAS (Model Checker for Multi- Agent Systems) software. They used MCMAS to solve games such as Nim, a simple card game, and a scenario involving the failure of the coyote to catch the roadrunner inspired from the popular cartoon, The Road Runner Show. Sahin et al. [53] developed a temporal logic called counting linear temporal logic plus (cLTL+) to model the collective behavior of multi-agent systems when heterogeneous dynamics is allowed. They formulate an optimization problem and use Gurobi to compute the trajectories for dierent robots. Then they studied asynchronous trajectories and found their method to work for bounded asynchrony with appropriate modications. 14 DeCastro et al. [54] combine Reactive Mission Planning using Linear Temporal Logic, and Local Motion Planning using convex optimization while enforcing the dynamic constraints of the robot and guaranteeing collision avoidance in 2D and 3D workspaces. The reactive mission planner synthesizes a correct-by-construction controller based on the high-level linear temporal logic specications of motion sequencing. Then physical experiments with two centrally-controlled Nao robots and a dynamic obstacle (KUKA's youBot) were carried out to demonstrate deadlock resolution. Specically, one of the Nao robots reversed its direction due to an impending deadlock with youBot. Lindemann et al. [55] allow users to dene a performance function in terms of a desired tem- poral behavior which the multi-robot system should satisfy based on a continuous state feedback control law. The Signal Temporal Logic (STL) based formalism is used to prescribe constraints for the multi-robot system. The developed control law is robust to disturbances and satises the signal temporal logic specications within user-dened robustness. They tackle specications which require certain conditions to be met within certain time intervals or sequential specications, meaning that a sequence of conditions will be satised in prescribed time intervals. A non-linear multi-robot system exhibiting single integrator dynamics while using a consensus protocol to stay close to each other is used to demonstrate tasks like formation control, sequencing, and dispersion. Ke et al. [56] model the behaviors of a ground control station, unmanned ground vehicles, micro aerial vehicles, and high-level unmanned aerial vehicles using Kripke structures. Then they verify the mission model's validity using model checkers like SPIN. They also integrate the SPIN model checker to assist in motion planning by generating counterexamples. Model checking techniques are very well-suited for state space search problems and by mod- eling path planning problems as state space search problems, they are eectively able to utilize linear temporal logic based model checking software packages to not only verify missions but also generate motion plans. 15 2.1.2 Model Checking Tools for Verication and Evaluation There are several model checking software available online, e.g., NuSMV, SPIN, PRISM that automatically verify user properties for a given system model. Generally, system models have to be coded in a programming language specic to the model checking being used. For SPIN model checker, one would have to code the model in Promela language. For NuSMV model checker, one would code in the SMV specication language. PRISM model checker also has its programming language. Many such software frameworks have been used for robotic planning. Fraser et al. [57] provide an extensive survey of model checkers, their comparisons and reasons for their eectiveness. Fainekos et al. [58] used NuSMV for path generation and MATLAB for controller synthesis to plan robot motion based on LTL constraints. Lahijanian et al. [59] designed the Robotic InDoor Environment (RIDE) simulator to move an iRobot Create mobile platform autonomously through corridors and recongurable intersections using PCTL-based MDP framework. Finucane et al. [60] designed the LTLMoP (Linear Temporal Logic MissiOn Planning) toolkit for designing, testing, and implementing hybrid controllers generated automatically from task specications written in Structured English or Temporal Logic. Konur et al. [61] used PRISM model-checker for verication of global robotic swarm behavior. UPPAAL model checker is an integrated tool environment for modeling, simulation, and verication of real-time systems modeled as a collection of non-deterministic processes and has been used for various robotic applications also [62{64]. Karaman and Frazzoli et al. [65] solved the LTL based multi-vehicle routing problem by converting the LTL constraints into Mixed-Integer Linear Programming (MILP) problems. LTL formulas may also be converted to an automaton whose product is then taken with robot automa- ton and then automata graph search methods are constructed to plan robot motion [66,67]. Shoukry et al. [68] solve a multi-robot path planning problem by computing collision-free trajectory controllers. They use a satisability modulo convex (SMC) programming approach 16 to decompose the problem into smaller sub-problems via Boolean constraints. They compared their method with sampling-based motion planners and outperformed them signicantly for single robot reach-and-avoid specications. They even simulated multi-robot scenarios where multiple targets had to be visited innitely often. Kamali et al. [69] perform formal verication of safety properties for automotive platooning using the Agent Java Pathnder (AJPF) model-checker. Machin et al. [70] developed a safety monitoring framework based on the NuSMV model checker for autonomous systems. Liu et al. [71] subject an unmanned ground vehicle to trac rules in the form of linear tem- poral logic specications. Environmental conditions are used to trigger the behavior execution of the unmanned ground vehicle. The unmanned ground vehicle is modeled as a reactive system with its behaviors encoded as a transition system, and the linear temporal logic mission specica- tions are used to compute the resulting automata. Simulations were carried out in Python-based, open-source Linear Temporal Logic MissiOn Planning (LTLMoP) to plan the motion of the un- manned ground vehicle for reaching its goal location while reacting to the environmental factors appropriately. Mehta et al. [72] design a robot compiler system which requires a user to input the functional specication of desired robotic behavior in Structured English. This requires some technical skill on the part of the user, but this is overall a very low requirement. That is then mapped into a linear temporal logic formula which is used to perform controller synthesis. They also provide a library of robot building blocks from which the system lters certain blocks for each proposition. The system also allows for faulty linear temporal logic specications, which it corrects based on behavioral con icts. Two robots, one line-following, and another wall- following were made by using this system, and these robots accomplished their tasks while being assisted with sensors. Choi et al. [73] present Kripke structures for three missions and verify their critical properties using the model checker called MCMAS (Model Checker for Multi-Agent Systems). An unmanned 17 ground vehicle is supposed to reach a goal location and is assisted by an unmanned aerial vehi- cle. Such a mission is made more complicated by adding multiple unmanned aerial vehicles and unmanned ground vehicles. There is also a ground control station that initiates the mission and guides the robots until the mission ends using images transmitted to it. The ground control station takes safety precautions like instructing the unmanned aerial vehicles to land once the communication signal has been lost. Humphrey [74] model the escorting of a VIP vehicle through a road network using multiple unmanned aerial vehicles. A human operator lays down the procedure through which the un- manned aerial vehicles explore the roads for safety and guide the VIP vehicle towards its target. This model of mission execution is coded up in Promela language. The conditions that should hold for the given mission execution are specied using the language of linear temporal logic. SPIN model checker processes the linear temporal logic specications and the Promela model to verify whether the properties hold or not. If the properties do not hold, then a counterexample is produced, which may be used to debug the mission execution plan further. Rothwell et al. [75] present a model checking software framework called Veriable Task As- signment and Scheduling Controller (VTASC) which takes in mission specications in natural language as input and converts it to temporal logic formula like linear temporal logic (LTL), com- putation tree logic (CTL) and CTL's extension, real-time computation tree logic (RTCTL). Then a model checking software called NuSMV is used to check whether the user-dened properties are satised during mission execution plans as dened by the operators. A graphical user interface (GUI) is used where users create the temporal logic specications in the English language. The temporal logic translator must process the nal English specica- tion and input it to the NuSMV if and only if it passes grammatical checks. They compared VTSAC against two other such software tools: Specication Pattern Instantiation and Derivation EnviRonment (SPIDER) and Linear Temporal Logic for MissiOn Planning (LTLMoP). 18 Alonso-Mora et al. [76] propose an integrated mission and motion planning framework where the mission planning is done oine, and the motion plan is computed at runtime. The mis- sion planner takes as input the linear temporal logic specications and generates the automaton representing the mission. The localized motion planner uses a convex optimization based ap- proach. They also describe the deadlock resolution strategy to resolve deadlocks by modifying the specication, if needed, to generate a feasible plan that may then be executed. Bank et al. [77] proposed an autonomy framework for smart manufacturing systems using linear temporal logic and its corresponding model checkers like NuSMV. However, NuSMV solvers may be very time-consuming because they exhaustively search the state space. To solve this problem, they considered a toy problem where blocks have to be appropriately rearranged on a 2D grid. Then they reduced the exploration space with a divide-and-conquer approach. Rather than solving for the entire grid area at once, smaller areas were considered one by one and then solved using NuSMV. This led to very high speed-ups as NuSMV is very fast in solving small problem sets. Their simulations showed that they were able to achieve linear computational time performance. Schillinger et al. [67] present a linear temporal logic based planning approach for on-demand multi-robot missions called simultaneous task allocation and planning (STAP) which constructs a team model for multi-robot missions to decompose the mission into more straightforward tasks and allocates them to dierent robots. Given a linear temporal logic mission specication for the mission, they generate an optimal task decomposition for the mission regardless of whether the explicit tasks are provided as input or not. The resource constraints like limited battery power are also taken into consideration for each robot. Several missions arising in oce scenarios are then simulated in the Robot Operating System (ROS) to validate and analyze the performance of their algorithm against the classical approach of planning for task costs rst. In their second work [78], they consider multi-robot mission scenarios in indoor oce spaces. They simulated two robots in an oce space which have to carry drinks from one location to another while avoiding 19 certain areas when carrying drinks. They also ran physical experiments with two robots. The mission constraints were encoded into a linear temporal logic formula. FlexBE software is used to generate complex robot behaviors using GUI facilities. It is also well-integrated with ROS. Spot is used to translate the linear temporal logic formula to an automaton. Then planning is done to minimize the maximal team cost incurred to nish the mission by using a label-setting approach for multi-objective planning problem. Maly et al. [79] present a multi-layer planning framework for a mobile robot along with replanning capabilities when obstacles are encountered. The high-level layer takes as input a linear temporal logic specication for the robot to satisfy. The robot is a second-order car with hybrid dynamics, and the linear temporal logic specication species how a particular oce area must be explored. A product automaton is formed out of the robot dynamics and linear temporal logic specications. The high-level layer generates plans for which continuous robot trajectories are computed using a low-level planner. Graph-based distances to an accepting state in the automaton are used to satisfy the specication as closely as possible. Saha et al. [80] study multi-robot mission settings whose specications are given using linear temporal logic based safety conditions concerning obstacle avoidance and collision avoidance. The problem is rst converted to a form understood by a Satisability Modulo Theories (SMT) based problem-solver called Z3 which then generates a trajectory for the robot by solving the system of constraints denoted by the linear temporal logic safety properties. Then simulation experiments for a group of quadrotors amid static obstacles were carried out successfully. Guo et al. [81] combine motion planning and action planning to tackle tasks involving visiting regions sequentially and performing actions at various locations. The task specications are given in the formalism of linear temporal logic. The robot automaton is constructed using an abstracted robot mobility model and its action map. A product automaton is then formed with the B uchi automaton representing the linear temporal logic specications. Then a graph-search algorithm is used to compute the optimal task sequence using which the hybrid controllers are synthesized. 20 Li et al. [82] have developed a model checking framework called PAT: Process Analysis Toolkit. They used this and other model checkers like Spin, NuSMV to solve some classical planning prob- lems like bridge crossing and sliding game problems. They found their model checker even to outperform state-of-the-art planners designed for these problems. PAT is highly ecient, ready to be used out of the box and modular. It was demonstrated to automate the municipal trans- portation management system by simplifying fare collection and notifying users about important information customized to their history. 2.1.3 Handling Stochastic Events Contingencies arise in robotic systems because robotic behaviors as well as environment responses are probabilistic [83] and multi-robot systems may generally be modeled as Markov Decision Pro- cesses (MDPs) [84{87]. Cizelj et al. [88] present an approach to control a vehicle in a hostile environment while considering static obstacles as well as moving adversaries against whom the vehicle must protect itself from collisions. The vehicle mission model, as well as the eect of adversaries on the vehicle, is modeled as a Markov decision process. The mission objective is to start from a certain region, reach another region to pick up an item and drop it o at another des- ignated region. They adopt the Probabilistic Computation Tree Logic (PCTL) control synthesis approach to plan the mission for the vehicle and use the o-the-shelf PCTL model-checking tool PRISM for this purpose. Ding et al. [89] present probabilistically guaranteed optimal control policies over innite time horizons for a Markov decision process with linear temporal logic constraints. The policy seeks to minimize the average cost per cycle (ACPC) starting from the initial state. Rigorous conditions for policy optimality are theoretically proved, and a dynamic programming algorithm is developed. This is demonstrated in MATLAB with and without linear temporal logic constraints for a robot navigating an environment and performing persistent tasks such as environmental monitoring and data gathering. 21 Faruq et al. [90] develop simultaneous task allocation and planning for multi-robot systems which are operating in uncertain environments and prone to failures. The robotic system is modeled as a Markov decision process. They propose a sequential model where each robot is considered independently one by one. Switch transitions are added to combine the individual robot models into a team Markov decision process. The goal is to nd a policy that maximizes the probability of satisfying prescribed co-safe linear temporal logic based task specications without violating safety specications also. They also develop the feature of allowing a task to be reallocated to another robot when its assigned robot fails. For this purpose, they synchronize the robot actions within the range of policies computed by their sequential model. They study the eect of increasing the number of failure points on computational time and reallocation frequency. Feng et al. [91] consider a road network surveillance problem involving an unmanned aerial vehicle and a human operator. Human operator assists the unmanned aerial vehicle to visit newer way-points based on the drone's existing state and also instructs the unmanned aerial vehicle whether to continue loitering over a way-point to improve upon the sensory information that has already been gathered. The human operator's performance is modeled in terms of his/her prociency, workload, and fatigue. They model this mission as a Markov decision process and implement it in the PRISM model checker. They study the eect of model parameters on the minimization of expected mission completion time. Then they model the same mission as a stochastic two-player game and study the trade-os between expected mission completion time and expected number of restricted operating zones visited by the unmanned aerial vehicle. Kloetzer and Mahulea [92] consider a partitioned environment in which one or more robots must survey regions of interest which appear and disappear probabilistically. The automaton building and graph search approaches are used to nd robot trajectories that maximize the prob- ability of satisfying the task of surveying regions of interest. This task is specied as a linear 22 temporal logic formula. They also seek to minimize the traveled distance. All robot collisions are ignored during planning to avoid extra computational costs. Such collisions are resolved during trajectory following stage. Chen et al. [93] use automata learning technique in uncertain, dynamic, and partially known environments to optimize surveillance missions. An indoor environment with doors is considered. Based on environmental data, the robot must learn how to incorporate the doors in its motion planning. Based on the learned door models, optimal control strategies are designed for the surveillance mission, which is modeled as a temporal logic game. The control policy is proven to be very close to the truly optimal one when unknown environmental information is fully learned. 2.1.4 Contingency Management Failure recovery through contingency management [94{96] is a crucial component of multi-robot systems that lead to their autonomy in real-sense because unexpected and uncertain contingencies are bound to arise and may result in failure of robotic operations. Christensen et al. [97] developed a method to detect failures of robotic sensors and actuators by observing changes in the sensory data that is recorded in real-time. For this purpose, faults are caused by software programming in the system, and their eects on sensory data are learned through neural networks. McGuire et al. [98] used contextual multi-arm bandit algorithms to help the robots learn how to select the best assistants to recover from failures while accounting for uncertain/dynamic operating conditions and assistant capabilities. In many works on multi-robot systems, failures arising due to communication failures, lack of situational awareness, sensor failures, and localiza- tion failures are studied [99{101]. Tang et al. [102] incorporated the advancement in prognostics and health management systems to develop a proactive and automated contingency management system that takes into account 23 prognosis information for mission replanning and failure recovery. They demonstrated this frame- work on a Pioneer 3-AT ground robot by using NASA's Hybrid Diagnostic Engine (HyDE) as the fault analysis engine and particle ltering-based prognosis algorithms [103]. 2.2 Multi-robot Task Allocation There is an abundance of multi-robot task allocation (MRTA) in the robotics literature [104{107]. Classical approaches have relied on mixed integer linear programming (MILP) techniques [108{ 111]. Zhang et al. [112] study the problem of automation in an assembly line with robots and human workers. They solve a MILP problem to compute the initial agent placement. The required tasks are then assigned and scheduled on these agents using hill-climbing and Tercio algorithms, respectively. Scheduling of activities on transportation and routing networks using constraint programming (CP) approach has been reported heavily [113,114]. In general, task scheduling problems in multi- robot scenarios can also be formulated as a constraint satisfaction problem making it amenable to the methods of constraint programming. Booth et al. [115] show that CP outperforms MILP for task allocation and scheduling problems relevant to retirement homes. Novas and Henning [116] use CP to automate manufacturing, traveling, and storage activities in exible manufacturing systems (FMSs) while accounting for tools, machines, and robot constraints. Evolutionary techniques have also been eectively utilized in solving MRTA and scheduling problems. Godinho Filho et al. [117] have surveyed the increasing application of genetic algo- rithms in scheduling activities in FMSs while [118] contains an application of particle swarm optimization algorithm to FMS scheduling problem. 24 Mousavi et al. [119] have employed a hybrid method combining simulated annealing and tabu search to solve supply chain scheduling formulated as a two-stage MILP problem. Multi-robot scheduling problems share several similarities to multi-processor scheduling prob- lems and hence processor scheduling algorithms are heavily applied [1,120]. Application of proces- sor scheduling algorithms mainly involves the use of dierent heuristics to schedule robot coalitions on tasks greedily [121,122]. Mission scenarios involving mobile robots must take into account the temporospatial nature of tasks which also have inter-task precedence constraints [123,124]. Task allocation and scheduling algorithms also need to take into account the uncertainty associated with task execution durations and robot traveling times due to the robots' and tasks' characteristics and their interactions with the environment [125,126]. Peters and Bertuccelli [127] developed a MILP framework to compute robust schedules for collaborative human-UAV missions. Gombolay et al. [128] present Tercio algorithm which solves a simplied MILP problem to generate agent-task allocations from which a near-optimal sequence satisfying temporal and spatial constraints is computed using dynamic priority heuristics. Owing to inter-robot communication failures and imperfect situational awareness across all the robots engaged in the mission, decentralized approaches have become a popular method of handling these challenges. These mainly consist of auction/market-based methods or their variants. Liu and Shell [129] developed a market strategy for MRTA problems where prices are strategically controlled to guide the purchasing behavior of the bidders and achieved global optimality. Wicke et al. [130] proposed a specic MRTA algorithm inspired by bounty-hunting practices robust to loss of agents and other noises while accounting for non-exclusivity of task assignments. Otte et al. [131] proposed a generalized Prim allocation auction algorithm to study MRTA problem with lossy communication channels. Another successful multi-assignment approach of combining auction and consensus methods results in the consensus-based bundle algorithm (CBBA) [132]. Zhao et al. [133] developed a 25 distributed algorithm to allocate tasks among heterogeneous unmanned vehicles for search and rescue missions by maintaining signicance values of tasks across all robots which performs much better than the CBBA. Das et al. [134] developed a distributed, market-based MRTA algorithm called Consensus Based Parallel Auction and Execution (CBPAE) for dynamic allocation of tasks in health-care facilities where the next task is not assigned to robots till they have nished their current task. It was shown that CBPAE was much quicker than CBBA. When the numbers of robots are more than the number of feasible tasks and multiple robots are allowed to work on a task simultaneously, a crucial factor in the MRTA problem is to form eective teams (or coalitions) among the robots and assign them to appropriate tasks [135{138]. This problem becomes even more complicated when heterogeneous robots are considered. Guerrero and Oliver [139] model the physical interference caused when employing a team of robots to nish a task to compute the task execution time and use auctions to determine the eective coalition sizes. Gunn and Anderson [140] study dynamic team formation when a robot gets lost or separated from the team in urban search and rescue missions. Cases with replacement robots and failures of leader robots are also analyzed. Hybrid methods try to make use of dierent parts of these primary methods to improve com- putational performance and solution quality. Lim et al. [141] designed four hybrid evolutionary techniques by putting together dierential evolution, articial bee colony, genetic and particle swarm optimization algorithms in dierent combinations. They were applied to optimize layouts for multi-robot cellular manufacturing systems. Lu et al. [142] proposed a hybrid multi-objective backtracking search algorithm to solve permutation ow-shop scheduling problem by considering both make-span and energy consumption. Heuristic schedulers are generally combined with MILP models to converge to solutions quickly [143,144]. A unied framework for combining MILP and CP models achieves faster convergence to high-quality schedules [145,146]. Evolutionary techniques can be incorporated into the scheduling procedure to obtain better results [147,148]. 26 In processor scheduling literature, problems dealing with precedence-constrained have been addressed, but those methods have not been explored at length in the robotics community. One of the work in robotics used in this dissertation for benchmarking purpose is [1], which assumes that the set of admissible coalitions is already available and does not consider precedence constraints. But one could also enhance features of the mission model, such that it allows fractional task completion, task interruption, and accounts for uncertainty. Ou et al. [149] present an online task allocation method for multi-gantry work cell in which a machine requires the gantry system to supply it with a part from its upstream buer and send the processed part to downstream buer. The goal is to maximize throughput, so the authors dene a metric called permanent production loss to track the system's performance. Machines must not be allowed to be idle, and production systems must be robust to disruption events. This is accounted for by the concept of opportunity windows. Flushing et al. [111] solve a multi-robot task allocation scenario where a mobile ad hoc network (MANET) needs to be set up and maintained. This requires computing data routing policies and data transmission schedules. They design a mixed-integer linear program (MILP) that allows a trade-o between task completion and communication performance. However, the mixed-integer linear program solutions only consider the total delivery ratio for the transmitted data, and therefore, an extra LP-based renement mixed-integer linear program stage is also added to include another important metric, viz. transmission delays. Whitbrook et al. [150] developed a distributed performance impact (PI) algorithm to allocate time-critical tasks among multiple heterogeneous robots. This paper suggests a useful extension to the PI algorithm to speed it up and solve more complex problems. Benchmarking it against the consensus-based bundle algorithm and baseline PI algorithm shows that it outperforms both. The baseline PI algorithm runs the risk of not exploring the search space fully. To tackle this issue, the task selection procedure is chosen to be an appropriate combination of the PI selection procedure and either the -greedy action selection method or the Boltzmann Softmax action 27 selection method. Simulations were carried out for search and rescue missions which comprise of vehicles, helicopters and unmanned aerial vehicles that must reach the victims to provide the resources they are carrying. The number of tasks is double the number of these vehicles. Average time to nish the rescue mission was reduced by as up to 8% using these modications. Nam and Shell et al. [105] model resource contention among multiple robots during task allocation. There is interference happening when a team of robots shares the same resource to perform a task. The authors model the performance degradation in task execution when multiple robots contend for the same resource and physically interfere with each other. These robots may perform the same tasks in dierent manners. Based on how the robots end up deciding to execute each task, a convex or linear penalization function is added to the objective function to be minimized. This problem is solved in two phases. In the rst phase of optimization, the integral constraints of the linear programming problem are relaxed, and fractional solutions are found using the interior point method. In the second phase, these fractional solutions are then rounded to integral solutions using the multi-choice Hungarian algorithm and multi-choice Murthy's ranking algorithm. Physical experiments, as well as simulations, were carried out where globally optimal solutions are found quickly. Shiroma and Campos [151] tackle the problem of task allocation in robots capable of doing multiple tasks simultaneously and ensuring coordination among the formed teams. They call this algorithm CoMutaR, which stands for coalition formation based on multi-tasking robots. They use schema theory and represent tasks as a set of motor schemas. This allows them to decompose the tasks into smaller robot-independent tasks such that the information collected incrementally by the system may be used by all the robots freely to perform their remaining tasks. Resources cannot be freely shared among robots. Coalitions among robots are formed using a single-round auction algorithm. Simulation experiments are carried out using player/stage/gazebo platform in ROS. Loosely coupled tasks like area surveillance and transportation are considered as well as tightly coupled tasks like box pushing. 28 Turpin et al. [152] solve the problem of concurrently assigning goal locations to robots and then plan collision-free trajectories for each robot. It considers an obstacle-free environment, and collisions among robots are prevented using online avoidance algorithms. The authors rst introduce a centralized algorithm which is tested out on a team of quadrotors. The minimum clearance between quadrotors was shown to be safe. The assignment was done to minimize the square of the minimum distance traveled using the Hungarian algorithm. Then collision-free trajectories were computed using the calculus of variations. Also, a decentralized version of the algorithm was designed to perform local replanning based on inter-robot communications, which resulted in sub-optimal but safe trajectories. Nunes and Gini [153] design an auction algorithm to allocate tasks with time windows among cooperative robots. The time windows specify the time range during which the tasks must be executed, and the tasks are allowed to overlap. Each robot models its temporal constraints using a simple temporal network. This is the variation they introduced in the sequential single- item auction algorithm and call it the temporal-sequential single-item auction algorithm. Their algorithm minimizes mainly the mission makespan in conjunction with the total distance traveled. Their algorithm outperforms a greedy algorithm and the consensus-based bundle auction during simulation in the JADE multi-agent platform. Maoudj et al. [154] solve the problem of allocating and scheduling tasks for robots such that the total mission completion time is minimized. The allocation of tasks among robots is done using a negotiation based approach similar to the contract net protocol. Thereafter, priority-based scheduling rules that respect task constraints are used to minimize the time spent being idle by robots. They demonstrated their methods using a pick-transport-place operation involving two manipulators and two mobile manipulators. Kanakia et al. [155] model robotic reghting and ant foraging tasks as global games where each agent's action depends on others as well as an underlying signal. Also, there is imperfect information and uncertainty associated with this scenario. Here, the authors show the application 29 of continuous threshold functions to allocate tasks among robots which leads to ecient coordi- nation among the swarm members and show how this leads to Bayesian Nash equilibria without communication among the agents. Additionally, it is shown that this method works even if the agents do not have a behavioral model of each other. Chopra and Egerstedt [156] consider several spatially distributed service requests with time instants when the tasks must be executed are also specied. Each such request requires a set of skills. Only the robots which have at least one skill in common with the skill set required are allowed to travel to the task location and execute it. This problem is formulated and solved as a combinatorial optimization problem. Its solution is validated using MATLAB simulation of a robotic music wall where dierent robots traverse over a wall to play a song, and each robot can play dierent instruments. Deng et al. [157] solve task allocation and path planning problem for multiple autonomous underwater vehicles (AUVs). They use a Mobile Ad Hoc Network (MANET) simulator for testing purpose. They adopt a location aided auction algorithm to assign tasks to robots and benchmark it against generic auctioning. Some autonomous underwater vehicles are good at searching vast areas of water while others are good at identifying the nature of the target. For planning trajectories of multiple autonomous underwater vehicles, they adopt a multi- objective feedback-based dynamic system that generates an action determination grid map to guide the autonomous underwater vehicle motions. This map encodes the combined benet at each cell due to multiple objective functions, and it is used in conjunction with a vehicle distribution grid map that stores the location of other vehicles. They simulate two dierent acoustic noise conditions for testing their model. One models autonomous underwater vehicles in shallow waters over a sandy bottom in calm seas while the other simulates the autonomous underwater vehicles operating in shallow water reef in rough seas. They achieve this by varying the power spectral density of the background noise and the Nakagami fading coecient. 30 Mosteo et al. [158] consider a multi-robot routing problem with limited communication ability. The robots cannot break the network connectivity among robots via radio waves while executing their allocated tasks. Whenever a communication link appears to be breaking, a motion-control mechanism based on spring action is activated to ensure that radio communication is not lost. Four task allocation algorithms are proposed that optimize over the total distance traveled, mission makespan, or the average task completion time. They use greedy and auction algorithms, and another one based on the traveling salesman problem. Then they also code an algorithm that allocates tasks by imitating the clock sweeping motion in polar coordinates. Kaddouh et al. [159] present a mission involving multiple unmanned aerial vehicles where vehicle teams are dynamically created and recongured to solve dierent parts of the mission. There are constraints associated with the dierent locations where the unmanned aerial vehicles are supposed to perform the tasks, and these tasks comprise time windows, refueling, or trans- portation services. Multiple requests are allowed to be combined into one task to reduce costs. The resource allocation problem is formulated as a mixed integer linear program (MILP). Their solutions outperformed those produced by greedy and bundle methods for eectively allocating resources and unmanned aerial vehicles for the tasks and reducing take-o and landing costs. There was a trade-o between faster mission completion and cheaper mission completion, which they try to optimize. Ban et al. [160] deal with a scenario involving a set of mobile robots with limited-range omnidirectional sensors tasked with monitoring other mobile targets. The robots are modeled to have Bayesian beliefs regarding their targets and the environment which they exploit to monitor all the targets more fairly. A mixed integer linear program is formulated to maximize the amount of average monitoring done for the targets as well as to minimize the standard deviation of the detection rate of the targets. The latter optimization is done to ensure fairness in the sense that lower uncertainty means the targets were monitored equally. The mixed integer linear program is used to monitor for a specic interval of time after which robot beliefs are updated, and the mixed 31 integer linear program is then solved again. The simulation experiments varied replanning times and communication ranges to study their eects. Both centralized and distributed architectures were also implemented. The distributed architecture proved to be better overall due to its low computational cost. Real robot experiments were shown to report a deviation from the simulations due to collision avoidance routines and robots getting stuck in congested and narrow channels. 2.3 Decomposition of Exploration Tasks A lot of work has been done to eciently explore unknown regions based on dierent constraints using multiple robots. Also, in the general context of multi-robot tasks, there is a heavy emphasis on decomposing the main tasks eectively among the robots. Task decomposition thus arises naturally into a host of robotics applications, including industrial manufacturing. Tham and Prager [161] use task decomposition methods to develop modular approaches to learning manipulator tasks through their sub-tasks using reinforcement learning. Since the state space and action space are both vast, they used a special type of neural network called a cerebellar model arithmetic computer (CMAC) based on the mammalian cerebellum. Wu et al. [162] studied the specic example of mold manufacturing operation as representative of multi-site product manufacturing. The operation is decomposed into tasks that can be managed individually within a network of manufacturing rms. A graph algorithm is presented based on a bidding process to assign the tasks to dierent contenders. Gindy and Ratchev [163] represented process capabilities of a facility using resource elements. Conventionally, sequences in which components visited dierent machines in a facility was used for this purpose. Rather than that, authors propose that machining operations be used to classify dierent components and then based on these classied component groups, machines are chosen. These resource elements are used to decompose a manufacturing facility into machining cells which 32 can operate autonomously. It avoids machine repetition, thus avoiding bottlenecks and is more compact. Whiteson et al. [164] study the soccer drill called keepaway which is a complex multi-agent task. In this drill, the players assigned as keepers aim to learn good performances for the tasks. But since the task is so complex, the idea of decomposing the task based on decision trees is used. Then the learning can be done over the simpler subtasks and learned models could be combined to master the original complex task. The authors use three learning methods with varying degree of human assistance so that somehow humans are not constraining the search space of good solutions. They show that when the player assigned as a taker in the keepaway game is allowed to run after the ball with very high speeds, only the decomposition-enabled techniques can learn the winning strategies. Stone and Veloso [165] design a distributed autonomous system for robotic soccer games allowing for periodic team synchronization. Tasks are decomposed into exible roles. Agents are allowed to create formations. Each agent could change its roles within formations, thus allowing exible teamwork. It was shown that the exible teams outperformed rigid teams. Formation switching was also allowed, and it helped to maximize the number of goals scored. Su and Zheng [166] worked on coordinating the motion of a legged mobile platform and a robot arm mounted on top of it, which is a challenging problem. They developed three dierent task decomposition approaches for reaching a given goal end-eector pose. The task of orienting the robot is decomposed to orienting the two robots separately in a dened sequence, thus making the problem easier to solve. The rst two approaches do motion using both the platform and the arm. The third approach exclusively uses the platform. As expected, the approach that uses the arm more is better because arm manipulation consumes less energy than platform motion. Tan et al. [167] facilitated a cable assembly operation using human-robot collaboration. The hierarchical task analysis technique is used to decompose the entire assembly operations into tasks assigned to only the human worker or both robots and human worker. When a task is assigned 33 to both of them, then the sub-tasks are partitioned between them and sequenced properly. This reduces the mental workload of the human worker and increases worker eciency by reducing faults during assembly and reducing the time taken for assembly. Huckaby and Christensen [168] use a taxonomy based on the decomposition of complex airplane assembly tasks into primitive robot skills. But this is easily generalizable to other manufacturing tasks. Perception tasks which will be done by vision systems are also incorporated in the taxonomy. The problem of decomposing exploration tasks among multiple USVs is closely linked to the problem of area partitioning. Hert et al. [169] dene boundary points that should lie in the partitioned polygons. Bast et al. [170] compute partitions of equal areas by merging and dividing convex pieces obtained originally via convex decomposition of the polygon such as in [171]. Bullo et al. [172] derive spatially distributed algorithms to compute a convex and equitable partition of a convex polygon based on power diagram with additional features involving the use of Voronoi diagrams. These works furnish mathematically robust solutions to the area partitioning problem. There is an abundance of literature that focuses on the area partitioning problem in physical robot settings. Chand and Carnegie [173] developed a hierarchy to utilize the available robots maximally. The robots are divided into three classes: manager robots have high processing and communication abilities. The planner robots have medium range communication abilities but low sensing and actuation abilities. The explorer robots have low communication abilities but high sensing and actuation abilities. Each robot gets scores for the task it performs, and when these task scores are used as feedback, it leads to signicant improvement in some cases. Ziparo et al. [174] consider scenarios where large teams of robots are coordinated for explo- ration problem using RFID tags. The RFID information helps to avoid collisions during local region exploration. A global monitoring process based on sequential planning is used to start 34 explorations at dierent regions to increase the explored area. This outperforms task assignment techniques because it avoids con icts. Visser and Slamet [175] study frontier-based exploration in oce-like space. The USARSim framework is used to simulate the oce space, obstacles, and robots in rescue scenarios and fron- tiers are assigned to robots with maximum utility based on information gain and probability of maintaining communication links. Planning is done by estimating the probability of communi- cation success signicantly longer into the future motion. And it is shown that when robots are allowed to share their maps, they avoid traveling the explored corridors doubly. Matignon et al. [176] attempt to solve CAROTTE challenge where an unknown environment has to be explored and objects known a priori to the robots have to be identied in the area. For some of these objects, robots may also have extra tasks associated with them. This problem is formulated as a decentralized Markov Decision Process (MDP), but since it is challenging to solve, the authors instead consider using a set of individual MDPs. A distributed value function is used to take the interaction among robots into account and maximize the area coverage of the team with minimum interactions. Yuan et al. [177] perform cooperative multi-robot exploration by expanding individual fron- tiers and selecting next target destinations to maximize information gain while minimizing travel cost and satisfying communication-limit constraints. They benchmarked their technique against the approach that automatically selects the nearest frontier cell. Their approach nishes faster and explores more area. Low et al. [178] use robotic simulator webots to simulate a team of 4 robots getting assigned a region where mineral data must be collected, which amounts to exploring a wide area which has only a few mineral hotspots. An adaptive sampling based exploration algorithm is designed, which leads to lower energy consumption, lower mission time, and higher hotspot sampling. Adaptive samplers use estimators, which can be highly biased by data sampled at hotspots. So they use Rao-Blackewellized estimators to handle the bias leading to higher chances of uncovering less likely 35 hotspots. Low et al. [179] solve the problem of performing both wide area coverage and hotspot sampling together. They use non-myopic and adaptive strategy to choose the next exploration locations based on data already observed. The new locations are sampled for exploration to minimize spatial mapping uncertainty. Their approach outperforms greedy and non-adaptive strategies. Ahmadi and Stone [180] deployed a team of robots to continuously monitor an area based on the changing importance of dierent locations. Simulation experiments that were carried out show how area gets partitioned with respect to conventional static partitioning algorithms. Their method also accommodates the addition or deletion of robots. They consider robots at dierent speeds also. Due to the changing importance of dierent locations, they are visited with non-uniform frequencies. Zhao et al. [181] proposed a multi-robot exploration approach that uses a market-based ap- proach for assigning frontiers to robots. Potential elds are used to coordinate the robots using virtual forces exerted on them. To avoid local minima, the potential eld is not always kept active. Basilico and Amigoni [182] consider an unexplored region with unknown targets inside this re- gion for which the paper shows the eciency of using Multi-Criteria Decision-Making techniques. The multiple criteria considered for frontier selection are information gain, the probability of communication link working, and traveling cost. To combine their scores into one global utility function, they make use of Choquet integrals computed using Kappalab R package. Their algo- rithm was benchmarked against other algorithms and shown to perform better, especially when diverse decision choices were available with sharply varying rewards. Solanas and Garcia [183] partition the unexplored area periodically using a K-means clustering algorithm and then assign the partitions to robots closest to their centroids. They have bench- marked this method against Burgard's multi-robot exploration work. This algorithm focuses on spreading out robots to cover the area and performs better than Burgard's method. But it is not signicantly more ecient than Burgard's method. 36 Wurm et al. [184] use Voronoi graphs to partition indoor environments into segments. Then the cost of sending each robot to the frontier targets of each segment is computed, and the Hungarian algorithm is used to assign robots to their optimal segments. Simulation experiments show that it is faster than frontier-based coordination. Real experiments using two identical Pioneer II robots in oce-like environments were done to show that the environment was explored completely and without any interference between robots. Pei et al. [185] consider robots exploring an environment such that connectivity is maintained and bandwidth is available to transmit video/audio feed from any explored region at any time back to the central command. They apply the set cover problem, Steiner tree problem, and the linear bottleneck assignment problem to solve such constrained exploration tasks. They compare their methods against two communication-limited exploration techniques. With respect to exploration time, they outperform the other two overall. Their method also guarantees better connectivity. Simmons et al. [186] work on frontier-based exploration via distributed bidding among robots. But the system is not fully distributed. Higher eciency can be achieved using a central plan- ner that synthesizes the local data accrued by all robots. The frontiers are expanded based on information gain. A team of two Pioneer AT robots and an Urbie robot was used to test their algorithm's robustness and was found to reduce exploration time and increase mapping accuracy. Doniec et al. [187] formulated frontier-based exploration problem by multiple robots as a constraints satisfaction problem and solved it by using a distributed and asynchronous algorithm. Constraints are placed so that communication links do not break and robots explore the unknown area more by not overlapping their sensing regions. Simulations were carried out in NetLogo to observe the algorithmic performance with multiple robots while allowing for obstacles and dierent communication and sensor ranges. Sheng et al. [188] augmented a distributed bidding algorithm with two measures to accom- modate the limited-range communication for ecient, coordinated frontier-based exploration by multiple robots. These features handle the limited communication ranges of the robots. The 37 robots explore areas such that it leads to maximum information gain in minimum time. A near- ness measure is introduced so that the robots tend to stay close together within appropriate limits. An important consideration when two robots meet is how do they exchange as little of their knowledge as possible to synchronize their maps. Vazquez and Malcolm [189] develop a behavior-based approach to explore an unknown area with communication limits. The behaviors include achieving communication connections, main- taining them, or avoiding collisions. This method is benchmarked against Yamauchi's seminal multi-robot exploration paper. Simulations were carried out for dierent environment sizes, and dierent communication limits and their method was found to be faster than Yamauchi's method. Elmaliach et al. [190] study the area patrolling problem in closed areas where constraints are laid down on the frequency with which the target locations are visited, and an algorithm based on nding Hamiltonian cycle is introduced. Then they also introduce deadline constraints for handling contingency events and design algorithms to recover from such scenarios quickly. Howard et al. [191] deployed mobile sensing robots in a stationary conguration to maximize the area covered. The authors use potential eld-based methods to achieve the static sensor network conguration in the environment. Simulation experiments study the time to deploy a network as well as the node spacing that is generated. Velocity and acceleration limits are added to the potential eld dynamics model to prevent oscillations during convergence. It does not, however, prevent the system from reaching static equilibrium. Batalin and Sukhatme [192] seek to achieve good coverage of an unknown environment. To achieve this, they disperse the robots so that their coverage regions do not overlap. The infor- mative approach is to form local coalitions among robots within each other's range. Then the information is exchanged, and directions of motions decided that spread out the sensor coverage to the maximum. Their molecular approach, on the other hand, assigns a direction to the robots 38 away from others without any local information exchange or deliberation. Both approaches out- perform the basic approach based only on robots maximizing their covered area. However, they nd the molecular approach to perform better than the informative one but only very slightly. Marjovi et al. [193] consider a search and rescue mission in a big warehouse that caught re and is lled with smoke. Khepera III robots with innovative smoke sensors are deployed in the warehouse to detect the re sources while exploring the entire oor space. They employ frontier-based exploration technique such that the time taken to nish exploring the warehouse is minimized. The frontier is assigned to the robots based on their distances. They benchmark their method against the optimal solution to cover the region if it was known entirely. Senthilkumar and Bharadwaj [194] study the multi-robot exploration of an unknown envi- ronment using an online ant-inspired robot dynamics to completely cover the area based on the spanning tree coverage method. They also handle partially occupied cells and allow for narrow open spaces to be shared among multiple robots. Their method is benchmarked against their previous algorithm in which spanning trees are not allowed to intersect in more open spaces so that collisions are avoided. Their new algorithm proves to be faster. Zlot et al. [195] present their work on frontier-based exploration trying to maximize informa- tion gain. Their method is robust to communication failures and can handle dynamic addition or deletion of robots. The frontier is expanded based on greedy, random, or quadtree-based strat- egy. Physical experiments were carried out with a team of ActivMedia PioneerII-DX robots. They compare dierent strategies based on area covered per unit distance traveled and found exploration eciency to improve by a factor of 3:4 for a four-robot team. De Hoog et al. [196] consider robots that can be in two roles: explorers or relays. The explorers cover the complete area and return periodically to rendezvous points to pass on the data to relays. The relays then take the information to a central command center. They compare their algorithmic performance with frontier-based methods. In terms of area covered, non-limited 39 frontier methods perform better. In terms of responsiveness to command centers, communication- limited frontier methods perform better than role-based. But the role-based method provides a trade-o between complete area coverage and maintaining tight communication links between the robots and the command center. De Hoog et al. [197] work with dynamic environments where paths can become suddenly blocked, and communication links may break suddenly. The rendezvous points are selected for explorer and relay robots by rst using Hilditch's Algorithm for Skeletonization to compute a small set of points in the robot's chosen frontier and then selecting the one with best communi- cation range and traversal costs. The frontier methods were faster than the rendezvous methods, but their command center performance was poorer. The authors' advanced rendezvous point selection method was shown to outperform the simple method in oce-like, open, and cluttered environments. Ouyang et al. [198] design a decentralized multi-robot active sensing algorithm and apply it to real-world scenarios. The task for multiple robots is to gather information at dierent locations to predict the unknown, non-stationary phenomenon and estimate the spatial correlation structure of the phenomenon. The phenomena studied here include the plankton density in oceanic regions and trac speed data on urban road networks. These phenomena are modeled using a Dirichlet process mixture of Gaussian processes. Hassan et al. [199] use Voronoi partitioning while optimizing multiple objectives and verifying the resulting multi-robot cleaning using torque heatmaps. Jager et al. [200] develop a dynamic, decentralized area partitioning method while accounting for limited inter-robot communication. Carlsson [201] solves a stochastic vehicle routing problem by dividing a given region using probabilistic analysis of traveling salesman problem (TSP) tours such that all vehicles have a balanced workload. Higuera et al. [202] use subgradient search methods to divide an area among multiple robots based on their speed proles. Agarwal et al. [203] compute rectangle partitions 40 of holed rectilinear regions and distribute them among multiple unmanned reconnaissance aerial vehicles (URAVs) based on their heterogeneous capabilities. Yamauchi et al. [204] established the use of frontiers to guide maximal information gain based exploration. In this work, the idea was to detect the boundaries between explored and unexplored regions as frontier cells and greedily explore the closest frontier regions. Sheng et al. [205] consider limited-range communication and develop a distributed bidding algorithm where every robot bids for its claim to frontier cells and tries to remain close to each other to minimize communication volume. Burgard et al. [206] use multiple robots to explore an environment in minimum time by assigning utility values to frontier cells available to robots such that the robots explore varied locations. Ma et al. [207] use a genetic algorithm (GA) to compute the optimal assignment of the frontier cells to multiple robots based on the utility of these cells calculated using Burgard's technique. Wang et al. [208] develop a frontier-based grid exploration approach where they use the PSO method to prevent multiple robots heading towards the same cell. Zhou et al. [209] use the PSO method to guide the robots to explore remote frontiers, thus minimizing the exploration time. Jung et al. [210] develop a coverage method based on dynamics of autonomous underwater vehicles (AUVs) and sea current disturbances. While each AUV in a team uses the developed method to cover regions, they avoid colliding with each other leading to entire area coverage. 41 Chapter 3 Modelling and Verication of Contingency Resolution Strategies for Multi-robot Missions Using Temporal Logic In this chapter, a framework to model multi-robot missions using formal methods is presented and desired properties for such systems are veried. 3.1 Introduction Multi-robot missions may encounter unexpected situations that can halt or alter mission tasks. For example, consider the case of a large ship that has to be escorted by robotic boats and drones. Each robot has been assigned its nominal task schedules. However, the ship's path may experience a blockage; robots' batteries may die; communication links may stop working; a civilian vessel may be spotted in the protected region around the ship. These situations will interrupt the regularly scheduled tasks. Due to the presence of such contingencies, contingency resolution strategies must be designed to contain and minimize the adverse eects of contingencies without drastically aecting the primary mission objective such as minimizing the expected time to produce parts or reach a location. In the civilian vessel escorting example, one strategy could be to rst direct a drone to assess the situation and, if the intrusion is conrmed to jeopardize the mission signicantly, then 42 a robotic boat must approach the intruder and warn it to move away. However, there may be numerous such strategies having dierent impacts on the mission. Some may lower operational costs while others lower mission makespan. Appropriate methods are needed to select the most promising strategies. Since contingencies occur unexpectedly or even rarely, one cannot test the resolution strategies as rigorously as one can test the nominal mission execution strategies. One could, in principle, run multiple high-delity simulations to decide how eective these strategies are and which ones are better. The problem with running such simulations is that they are not easily scalable for larger problem sizes because they try to encompass several minute details and run on professional simulation software packages which are resource-intensive and time-consuming. Also, manually coding dierent possible mission executions and verifying them through computer simulations is very time-consuming and maybe unrealistic. This chapter focuses on leveraging the use of model checkers to the domain of multi-robot missions to assess the adequacy of contingency resolution strategies to minimize the adverse eects of contingencies on the mission execution. In robotics research, the use of formal methods has become quite popular to analyze mission execution protocols and contingency resolution strategies. Formal methods provide a complete framework for handling dynamic and uncertain systems in a mathematically rigorous, and logically consistent manner. However, such model checkers were traditionally designed for other applications such as embedded systems and computer hardware design. So when modeling multi-robot systems, generally some simplications have to be made. The challenge is that one must not oversimplify the mission model so much so that running high- delity simulations would have proved to be the better choice. However, it must be ensured that the mission model adheres to the semantics implemented by the model checkers. This chapter presents a framework for the nominal modeling of the mission using formal methods, the design of contingency resolution strategies for model checking and evaluation of these strategies with respect to desired mission objectives. 43 3.2 Problem Statement The objective is to develop a framework to apply model checking tools to incorporate contin- gency models in nominal mission models and evaluate the eectiveness of contingency resolution strategies. Given a mission model and alternative strategies to handle contingencies, one must evaluate the performance of these strategies using a model checking framework. In the next sec- tion, the various components required for designing such a framework are described and then this framework is applied to two case studies in the next two chapters. 3.3 Overview of Approach 3.3.1 Nominal Mission Modeling for Deterministic Transitions Mission modeling and verication methods require a nominal mission description as input. One must generate a formal transition model for the system based on such a description. Model checking refers to the automatic and exhaustive verication of whether a given specication is satised by the system. Therefore, both the mission model and specications are needed for model Figure 3.1: Characterization of multi-USV escorting mission scenario (Not drawn to scale) 44 checking. A mission example representative of multi-robot surveillance and patrolling missions is used for illustration purpose. Suppose there is a high-value target that has to be guided by a team of mobile robots through waypoints to reach the desired location ultimately. An example is shown in Figure 3.1, where a ship has just entered a port and must be escorted by a team of autonomous unmanned surface vessels to its designated location. However, the modeling presented here is independent of this specic mission and applies generally to multi-robot patrolling missions involving any mobile robot. Based on such a mission description, the nominal tasks comprising the mission are identied. Most missions consist of spatially distributed tasks that may be performed concurrently by mul- tiple agents. Agents may be human workers or robots. One example is the task of escorting the high-value target from one waypoint to the next performed by a team of mobile robots. The pre-conditions and post-conditions are identied for each task. If the pre-conditions for a specic task are satised then an agent, available and well-suited for carrying out the task, executes it which results in a change of the mission scenario as described by the task's post-conditions. Then a mission execution plan is chosen that is well-suited for the mission under consideration. It is a sequence in which the available tasks must be executed to meet the mission objective while adhering to pre-conditions and post-conditions of all tasks. For example, the mobile robots will meet up with the high-value target at a safe location, and then the convoy will proceed through each successive waypoint until it reaches its nal destination. Missions are generally of two types, deterministic and probabilistic. If a mission only allows an agent to transition from a current state to a unique new state when certain conditions are met at the current state, then the mission is modeled as a deterministic process. If there is at least one agent state where one or more probabilistic transition rules are applicable, then the mission is modeled as a probabilistic process. To perform nominal modeling and verication for deterministic missions, proceed as follows:- 45 • Identify the set of discrete states (let v j ai denote the j-th variable for the i-th agent a i ) available for each agent along with their domains and initial values. For example, the location of the high-value target may be a mission variable whose domain is given by the designated waypoints. The initial state for each such variable must also be characterized. For example, mobile robots begin their operations initially from some designated resting station. • Identify the environment variables (let v j e denote the j-th environment variable) that keep track of relevant factors in mission scenario: For example, some variables may keep track of whether the high-value target is encountering a dangerous situation. It may be assumed that these contingency variables are maintained and updated by a ground control station (GCS). The GCS receives sensory data that helps it to detect such unexpected contingencies and communicate their occurrence to mobile robots. • Encode transition rules (let t j denote the j-th transition rule) which specify constraints for agents to transition from one state to another in terms of tasks' pre-conditions and post- conditions which in turn are specied in terms of agents' states and environment variables. For example, a transition rule specifying the motion from one waypoint to another represents a task for the mobile robot to execute. Such transition rules may be encoded with ease in model checkers like NuSMV as: if pre-conditions, then post-conditions. Transition systems are adopted for modeling the missions. These are computer science tools used to describe transition rules for discrete-state systems. Transition systems are represented using Kripke structures. According to Kripke semantics, a Kripke frame is represented byhS;Ri whereS refers to the states that may be taken by the system. For missions, states are of two types, one representing the states which the agents assigned to the mission may reach, v j ai , and another representing the states that environment may reach, v j e . R SS represents the accessibility relations. The accessibility relations indicate which states can system access or transition to from 46 a given state. R comprises of transition relations t j which indicate which states a system may access or transition to from a given state. Let AP denote the set of atomic propositions relevant for the system under study. When atomic propositions that can be veried for the system are provided, the Kripke model for the system gets dened. Kripke model is denoted byhS;R;Li where L : S 2 AP ! f>;?g is a labelling function that returns true or false for a given state and atomic proposition(s) over it. If multiple atomic propositions are queried, this function can only return true if all atomic propositions are individually satised by the queried state. Finally if the initial states of the system denoted by I = [v j ai (0);v j e (0)] are identied, the Kripke structure is given byhS;I;R;Li. 3.3.2 Nominal Mission Modeling for Probabilistic Transitions Transition rules between states of a system are not always deterministic. For stochastic systems, probabilistic transition rules are specied [211]. Stochastic systems with discrete states are gen- erally modeled as Markov Decision Processes (MDPs). An MDP is a discrete-time stochastic framework denoted by a 4-tuple M =hS;I;A s ;P a i. Here S represents a nite set of states the system may transition to. I denotes the initial states the system begins from. A s lays down the nite set of actions that are applicable at states2S. P a (s;s 0 ) represents the probability that the system will transition to state s 0 in the next step if it is currently in state s and adopts action a. A nite path through an MDP is a sequence of alternating states and actions that may be denoted as follows: =s 0 a0 !s 1 a1 ! (3.1) PRISM allows one to dene reward structures that help in computing expected total rewards for the system. One can either dene state-based rewards r :S!Q or transition-based rewards r : SA! Q. Then the total reward for path consisting of N state transitions is given by 47 rew(r;) def = N P i=0 r(s i ;a i ). The expected total reward is computed over all nite paths that meet some user-dened conditions. Discrete-time Markov chains (DTMCs) are also feasible modeling tools in several cases. MDPs allow for states in which multiple transition rules are applicable and resolve such con icts based on the principles of nondeterminism. However, DTMCs dier from MDPs in that they allow only one probabilistic transition rule for every possible state of the system. NuSMV allows some simple handling of probabilistic behaviors, but it is limited in scope. The rst case study discussed in next chapter is mostly deterministic, but it includes a few simple probabilistic transition rules. For example, if an agent may execute two tasks from its state x, one with 2=3 probability where it moves to state y and another with 1=3 probability where it moves to state z. It would appear in NuSMV as follows. next(agent_state) := case agent_state = x : {y, y, z}; esac; NuSMV excepts the user to pick each mission variable in a sequence. Then for all possible cases that the variable may encounter, one must specify the corresponding change in its state. NuSMV also requires the user to repeat the variable names to match their probabilities. Coding such transition rules in NuSMV may be inconvenient for complex stochastic systems. The probabilistic model checkers are more convenient and eective. PRISM is a popular tool for this purpose. Unlike the exhaustive enumeration of NuSMV, PRISM requires the user to encode dierent conditions that the user considers relevant for the mission and then specify how all the mission variables will change if those conditions hold. For example, if an agent may execute two tasks from its state x, one with 75% probability where it moves to state y and another with 25% probability where it moves to state z. It would appear in PRISM as follows. 48 [] agent_state = x -> 0.75 : y + 0.25 : z; Real-world systems are often simulated by appropriately synchronizing these transition rules in PRISM. The extensive use of this tool for probabilistic model checking is demonstrated with the help of the second case study. 3.3.3 Modeling of Contingency Resolution Strategies These are the steps to incorporate contingency information in mission execution. Firstly, design the contingency resolution strategies which are invoked when a contingency is reported during mission execution. However, these strategies do not merely assign an agent to execute a specic action to resolve a contingency. These strategies may require multiple actions to be performed in a sequence. So for dierent cases, dierent sets of actions may have to be invoked. Below, dierent examples of contingencies are given that may be encountered during multi-robot missions such as those described earlier. • Suppose robots are required to maintain contact with GCS at all time. This may be neces- sary for robotic operations so that human remote operators may take control of the robots if they break down. Since the ground station must always maintain a communication link with the robots, a variable called \comm link[i]" is introduced. This Boolean variable denotes whether the communication link between GCS and i-th mobile robot is working or not. Then a strategy is laid down for resolving communication failures. If the communication link breaks down, then the robot is supposed to go back to its station for repair purpose and avoid any unsafe situation in the open waters. When the link between the station and the robot is restored, it rejoins the rest of the convoy. • It is common to nd missions interrupted because suddenly some event took place which halts the mission. The only way to resume the mission is for the agents to address the event rst. For example, when the high-value target is moving through a crowded region 49 and is being assisted by multiple mobile robots, it is possible for another vehicle to come dangerously close to the protected asset. To prevent collisions, the following strategy is designed. If another vehicle is reported in the protected zone, then a robot is instructed to rst go to the region to conrm whether the vehicle is still there. If the vehicle's presence is conrmed, then the robot must approach and warn the intruder to leave that region and return to resume its mission only after the intruder has acknowledged the message. • In busy seaports, a civilian vessel may come dangerously close to a ship. Following the pre- vious example, one can consider a slight modication to the contingency resolution strategy. Suppose each unmanned surface vehicle (USV) had an unmanned aerial vehicle (UAV) mounted on top of it. The UAV is used to conrm the presence of the vessel because it may roam easily in the port region compared to the USV. If UAV conrms the civilian vessel, then the USV will approach and warn the intruder. • The path ahead of the convoy may be blocked. Static obstruction, say debris may be found on the path which when reported, a robot must go to that location to conrm the obstruction and its nature. Upon conrmation, it must attempt clearing the path, and until the path is cleared, the mission is paused. Only when the path is cleared, the mission resumes. One can assume that if there is large-sized debris that the robot will be unable to clear, then it will message the central mission controller to send human-operated tugboats to clear the debris. Again, for the ship example, one may use the UAVs for contingency conrmation and the USVs for contingency resolution. Secondly, encode the contingency resolution strategies using appropriate transition rules. Model checking is crucial in detecting whether humans have introduced any inconsistencies when coding such transition rules. For example, if the communication link fori-th robot breaks, then it should head towards the base station coded as !comm_link[i]: base_station. 50 If multiple contingencies coincide, then an essential aspect of contingency planning is to decide how the respective strategies should synchronize among themselves. Synchronizing contingency actions in dierent ways may signicantly impact mission performance. This is clearly demon- strated in the second case study. 3.3.4 Model Checking and Verication 3.3.4.1 Deterministic Transition Systems The system properties to be veried by the user are encoded with the help of temporal logic. After the mission model and contingency resolution strategies are coded in the programming language of the model checking software and subsequently compiled, the user assesses those system properties using available model checking tools. For deterministic systems, one must verify whether the system eventually reaches a desirable state. There may be multiple such desirable states. For deterministic transitions, one may use LTL formulas to specify goal states. If states of a system are considered as nodes then the transition rules of the system dene a graph-like structure. CTL and LTL operators help in reasoning in this domain. While CTL operators explicitly help in reasoning about branching paths, LTL operators help in reasoning about temporal nature of system execution in a single path. CTL operator A is used to reason about all paths that a system might take whereas operator E tries to reason whether there exists a path reaching some desired state. The LTL operators assume a set of atomic propositionsA P are dened. LTL builds up on the logical connectives (or :_; and :^; negation ::) by proposing new temporal modal operators: X (neXt), U (Until), F (Finally), G (Globally). Ifp 1 ;p 2 2A P are atomic propositions then Xp 1 indicates that p 1 holds true in the next time step, p 1 U p 2 indicates that p 1 is true at least until p 2 becomes true, Fp 1 indicates that eventually p 1 holds true and Gp 1 indicates that p 1 always holds true. 51 One may also verify whether risky or inconsistent behaviors have been injected into mission execution due to human error. For example, in vehicle routing problems, one generally wishes to verify whether the vehicle reaches a desirable state and stays there. For this, the following type of LTL specication may be used: FG; where is a logical proposition about the mission. may be composed of conjunctions and disjunctions of atomic propositions. The LTL formula states that the logical condition will eventually be always satised. The G operator is used to ask the model checker to verify whether the vehicle achieves equilibrium at the desired location once it reaches that location safely or it has a tendency to wander o to another location due to the way mission specications were encoded. Before verifying such properties, one must explicitly ask the model checker to discard specic paths of model execution. This is done by making a fairness assumption which restricts NuSMV only to consider those mission executions which are generally expected in real-world. When verifying the correctness of a given mission protocol, one wants to discard trivial counterexamples, say those produced due to livelock situations. This is because the contingencies may put the system in a never-ending loop. For example, suppose the convoy is interrupted by a contingency event, and the mobile robots are assigned to resolve it. Once the contingency has been resolved, and the convoy is ready to proceed with the mission again, another contingency event occurs. This may keep happening again and again. This is a livelock situation. 3.3.4.2 Probabilistic Transition Systems Deterministic missions require exhaustive enumeration of states possibly reached by the system to verify LTL properties. But this may not be computationally possible due to physical memory limits. So it may be computationally more ecient to model missions as probabilistic processes. Many missions happen to be probabilistic. But then one is only able to provide probabilistic guarantees by computing the probability with which a system will reach a specic state in a par- ticular duration or expected time elapsed before system encounters a goal state. For probabilistic 52 systems, probabilistic operator P is added to the previous LTL operators, and this temporal logic is referred to as Probabilistic Computation Tree Logic (PCTL). These PCTL formulas are most commonly used to query about probabilistic reachability conditions which may refer to querying the probability that the system eventually satises a set of atomic propositions in a conjunctive or disjunctive way or even a mixture of the two. PCTL verication results produced by PRISM are used to evaluate how the dierent contingency resolution strategies perform and how mission performance varies by changing mission settings. For probabilistic model checking, numerical methods like value iteration are used. Numerical techniques create symbolic representations using binary decision diagrams (BDDs) and multiter- minal BDDs (MTBDDs) and operate iteratively in conjunction with a sequential stopping rule. Systems exhibiting nondeterminism are modeled as MDPs, and one can only query the minimum or maximum values of probabilities and expected rewards. One is generally interested in querying about probabilistic reachability conditions such as: Pmin=? [F t \cond" ], asking what is the minimum probability that within rstt state transitions the system satises the logical condition \cond" which may be a union or intersection of atomic propositions. If a reward structure T R assigns real-world time duration during system's state transitions then PCTL expression like T Rmin=? [ F \cond" ] queries the expected time in which system satises the specied condition. PRISM returns a nite output for this query if and only if Pmax 1 [ F \cond" ] which means that the condition \cond" is eventually satised by the system almost surely. However, Pmax 1 [ F \cond" ] maybe true even though a stronger condition like AF \cond" fails. As of PRISM 4.4, one can only query the actual value of Pmax because bound checking is not allowed. The bound checking expression was only used for clarity. MDPs are only solved using numerical methods which place a signicant restriction on the problem sizes that may be investigated. Missions are modeled as DTMCs so as to investigate problems of larger sizes because DTMCs are computationally more ecient than MDPs. In addition to numerical methods, even statistical 53 methods may be applied to DTMCs for performing model checking. They involve Monte Carlo simulation and sampling for hypothesis testing and are much faster. Statistical methods can handle larger problem sizes as compared to the numerical methods. 3.4 Task Network Generation Temporal logic-based modeling is also used to generate task precedence graphs for multi-robot missions. For example, missions involving multiple mobile robots usually comprise precedence- constrained survey-like or travel-to-location tasks [212]. Often it is assumed that the mission supervisor provides such precedence graphs. However, temporal logic may be used to extract such precedence graphs from mission specications itself. These task-precedence graphs are then provided as input to task allocation and scheduling algorithm. Two types of LTL constraints generally suce for all cases. Firstly, a system may have Sequencing with Avoidance and without Repetition (SAR) constraints. Suppose task 2 cannot be started until 1 has been nished and both tasks have to be done only once throughout the mission. The LTL expression for this SAR constraint is: F ( 1 ^ F 2 )^ G( 1 =) XG: 1 )^ G( 2 =) XG: 2 ) (3.2) Each such SAR constraint creates a directed edge originating at a node representing task 1 and ending in node representing task 2 . Secondly, the system may have a simple reachability constraint like F 1 , thus adding a node to the task network. Using these two types of LTL constraints that may be extracted from the mission description, one can construct task precedence graphs which serve as an input to the previously developed task allocation method. The resulting task network will be a directed acyclic graph with the possibility of free nodes. So it may not be a connected graph even in the undirected sense. 54 3.5 Summary This chapter presents a framework that can model nominal missions with deterministic and prob- abilistic transitions. An approach is also provided for modeling the contingency resolution strate- gies with specic examples of contingencies. Existing model checkers are used to evaluate these strategies. 55 Chapter 4 Multi-USV Case Study In this chapter, a case study involving a mission in port environments is presented to escort a ship to its docking location using multiple unmanned surface vehicles. 4.1 Introduction NASA's Mission Control Centre is an excellent example of how complex missions are being central- ized and automated. Today, the crew on board the international space missions mostly performs research activities while the space ight and other safety conditions are maintained through a ground station via remote control. This concept of automating complex missions involving robots and AI software through a base station supervised by humans is a major focus of current automa- tion goals. It is desirable for humans to be able to supervise and control the entire mission through robots while sitting in a base station far away from the mission activities. However, automation technologies are still lacking in some areas so that humans may also be needed to collaborate with robots. A mission involving multiple unmanned surface vehicles (USVs) in a busy port is analyzed where the USVs must escort a ship to its designated docking location. The autonomous vessels used during this mission are assumed to be under the supervision of a base station just like with space missions so that in case of a catastrophic failure or mission disruption, human supervisors 56 can take over. Three types of unexpected contingency events are introduced that the system must be robust against. Mission execution protocols are designed taking into account the desired mission objectives. But these protocols must be checked to be logically consistent and fail-proof, so their correctness is veried using NuSMV model checker [213]. 4.2 Problem Statement Given an English description of the ship escorting mission, the framework presented in the last chapter must be able to encode it in a model checker and evaluate the performance of the con- tingency resolution strategies. One must rst model a ship escorting mission with the help of USVs mounted with UAVs. These robotic boats and drones must operate autonomously to assist the ship once it has entered the port until it has docked successfully. The dierent contingencies or accident scenarios that can occur must be modeled and strategies must be designed to use the robots to contain these situations without damaging the convoy mission. The framework developed in the previous chapter is used to evaluate the strategies to resolve dierent types of contingencies. Figure 4.1: Characterization of multi-USV escorting mission scenario (Not drawn to scale) 57 4.3 Mission Modeling In this section, a multi-USV mission scenario is analyzed as shown in Figure 4.1 where a ship which has just entered a port must be escorted by a team of USVs to its docking location and assisted till it has been properly docked. The USVs are initially located in the USV station and must nally return to it after mission completion. Though the USVs are autonomous, they are supervised from a ground control station labeled in Figure 4.1 as GCS. The GCS receives sensory data that helps it to detect contingencies. The GCS is also required to maintain contact with USVs at all time. This is necessary for robotic operations for various reasons, the most important being the ability for human remote operators to take control of the USVs if the USVs fail. The Kripke structure for each of the transition system involved in this mission is described one by one along with its corresponding NuSMV codes. For this purpose, the USV states must rst be identied. Each USV can be located at the docking station (l s ) or one of the way-points generated for the ship motion (l 0 ;l 1 ; ;l w ). The straight line segment traversed by the ship from one way-point to another, sayl i1 tol i is denoted byp i . It is assumed that p i is the path generated by ship to move linearly from location l i1 to location l i . The region around path p i must be clear of other vessels. In busy ports, commercial or civilian vessels may come dangerously close to a ship and thus to prevent collisions it is important to keep the area around a ship's path clear. The region around path-segment p i is denoted by r i . When a civilian vessel intrudes into the protected region of the ship, these situations are referred to as contingencies of type A. Thus, if a contingency of type A arises in region r i , then a UAV is instructed to survey the region r i to conrm whether the civilian vessel is still there. If the UAV conrms that the civilian vessel is still there, then its corresponding USV will approach the intruder's vessel and warn it to leave. The USV must keep tracking the moving intruder vessel till the vessel has cleared the region. In NuSMV, this 58 contingency state of the USV is modeled by assigning it the task of monitoring entire region r i . Thus, u i =r i denotes the contingency state of type A. Contingencies of type B refer to the blockage of the path in front of the ship, thus making it dangerous, or at least not feasible, to go ahead. Now if contingency of type B occurs, then some static obstruction, say debris, have been found in the path of the ship which prevents the ship from traversing that segment, so a UAV is commanded to survey the path-segment to nd if there is a static obstruction. If there is an obstruction, then the associated USV of the UAV will be directed to attempt clearing the path, and until the path is cleared, the mission is paused. Only when the path is cleared, the mission can resume again. It is assumed that if there is large-sized debris, then the USV will be unable to clear. So it will message the central mission controller to send human-operated tugboats to clear the debris. This interaction is not explicitly modeled, and it is assumed that the obstruction is eventually cleared due to the intervention by the USV, and only then the mission resumes. So the additional states that can be accessed by the USVs are p 1 ; ;p w and r 1 ; ;r w . In NuSMV, this is coded as follows: u1_state : {ls,l0,l1,l2,p1,p2,r1,r2}; The states that ship s 0 may take is given by the computed way-points: l 0 ;l 1 ; ;l w . The location l 0 is assumed to be the starting location where the pilot assigned to the ship begins guiding it to enter the busy port. The central mission controller, which is generally a ground station on the port premises, computes the way-points needed to be traversed by the ship to reach its docking location, and it is denoted byl w . These way-points are computed as soon as the ship enters the port at location l 0 . Once the ship has reached its docking location, then it must perform complex and slow maneuvers to dock itself properly. During this docking action after reaching the way-point l w , the ship is assigned to the next state, which will be the execution of the task of getting \docked" properly. In NuSMV, this reads as follows: 59 u1_state : {ls,l0,l1,l2,docked}; The initial state of all the USVs, as well as the UAVs, is the port station where they rest and are periodically serviced for maintenance purposes. The initial state of the ship is the rst waypoint. The occurrence of contingencies during the mission is simulated by two variables for each segment. The variable \contingency type A[i]" holds non-negative integer values representing the number of other vessels intruding into ship's space r i . For simulation, an upper limit must be placed on the number of vessels that can appear in this region. Varying this limit signicantly aects the simulation results. For each intruder's vessel, only one USV is assigned to handle it. Similarly the Boolean contingency variable \contingency type B[i]" denotes whether the path- segment p i is obstructed or not. This ag may be raised even if the segment is deemed to have low depth or high current. If path-segment is obstructed, then at least two USVs are assigned to handle it. Again such constraints on the number of USVs are needed to handle the contingency tasks. They are user-dened and can be changed appropriately. If I represents the number of robots that must be assigned for a contingency task of type A, then I = 1. Also, assign II = 2, which has a similar meaning. In this work, I is xed but other values may be allowed for II . In the former case, it is assumed that the USV's job is to warn intruder vessels by turning on a beacon alarm and alerting the vessel driver to leave the area. This can be eciently done by a single USV. In the latter case, a static obstruction may have to be cleared. For this purpose, multiple USVs are employed in a coordinated eort to push away the debris. One can choose II = 1 if larger debris will not be encountered, or if human assistance will be provided when large obstructions are noticed. Similarly, one can have higher values of II to be safe, but employing USVs is expensive, so there must be a trade-o. In the case of real-world operations, these variables are maintained and updated by the central ground station, which regularly communicates this information to the USVs. Though the central 60 ground station must listen to the environmental sensory data to update these values, in th present simulation, their values are randomly sampled from a probability distribution which is assumed to mirror the sensory data recorded over time previously. Therefore, the initial values for the contingency variables are generated by sampling randomly from a probability distribution. Since the ground station must always maintain a communication link with the USVs, another state variable called \comm link[i]" is introduced. This is a Boolean variable maintained for USV u i , which denotes whether the communication link between the ground control station and the USV is working or not. Initially, this variable is set to true for all USVs because initially all USVs are at their station and assumed to be properly functioning before the mission commences. For the above mission scenario, if inputs are given in the form of the number of USVs assigned to the escorting mission, the number of way-points that must be traversed to complete the mission, and trac/environment modeling for the region around the ship's pre-planned path, then Matlab is used to generate a SMV le for the problem scenario. For example, if there are three USVs, three way-points, and only one civilian vessel can intrude upon the region around the ship's pre-planned path, then the generated SMV le's initial code looks as follows MODULE main VAR s0_state: {l0,l1,l2,docked}; u1_state: {ls,l0,l1,l2,p1,p2,r1,r2}; u2_state: {ls,l0,l1,l2,p1,p2,r1,r2}; u3_state: {ls,l0,l1,l2,p1,p2,r1,r2}; a1_state: {ls,l0,l1,l2,p1,p2,r1,r2}; a2_state: {ls,l0,l1,l2,p1,p2,r1,r2}; a3_state: {ls,l0,l1,l2,p1,p2,r1,r2}; comm_link: array 1..3 of boolean; contingency_type_A: array 1..2 of 0..1; 61 contingency_type_B: array 1..2 of boolean; DEFINE num_USVs_for_2nd_contingency:=2; ASSIGN init(s0_state):=l0; init(u1_state):=ls; init(u2_state):=ls; init(u3_state):=ls; init(a1_state):=ls; init(a2_state):=ls; init(a3_state):=ls; init(comm_link[1]):=TRUE; init(comm_link[2]):=TRUE; init(comm_link[3]):=TRUE; The transition relationships are described for these states as follows. The ship s 0 which will be assisted in docking can travel from way-point l i1 to l i only when certain conditions are met. All USVs along with their UAVs should have approached the ship's current location. The communication links of all these USVs with the central mission control station should be in proper working condition. Also, contingency tasks of type A and B should not be aecting the mission. These entire constraints are coded as follows: u1_state=l0 & u2_state=l0 & u3_state=l0 & a1_state=l0 & a2_state=l0 & a3_state=l0 & comm_link[1] & comm_link[2] & comm_link[3] & s0_state=l0 & contingency_type_A[1]=0 & !contingency_type_B[1] : l1; Also if the ship is at its last way-point, then it can only start getting docked if all the USVs along with their UAVs are around the ship to assist it while docking and their communication 62 Figure 4.2: Kripke modeling for the ship links are working properly. If none of these conditions are met, then the ship remains at its current location. u1_state=l2 & u2_state=l2 & u3_state=l2 & a1_state=l2 & a2_state=l2 & a3_state=l2 & comm_link[1] & comm_link[2] & comm_link[3] & s0_state=l2 : docked; Figure 4.2 shows the Kripke graph for the ship based on the above transition rules. Next, the transition rules for the USVs are described. Figure 4.3 shows the partial Kripke graph for a USV based on the following transition rules. A port station is a location inside the port where USVs rest when they are idle and not participating in any mission. If they suered a communication failure, then they must report to the port station to get the communication equipment repaired or wait safely till the communication link is back on. If the USV is at the port station, the communication link is working, and the ship is assigned to go to any of the pre-planned way-points, then the USV must also approach the same way-point. This is coded up as follows: u1_state=ls & comm_link[1] & (next(s0_state)=l0 | next(s0_state)=l1 | next(s0_state)=l2): next(s0_state); This transition rule helps take care of both the cases that a USV may nd itself in when located at the station. This can be readily veried. It is assumed that the mission begins when the ship has been guided by the pilot to the port location l 0 . So if the USV is at the station before the starting of mission execution, then the ship will remain stuck at port location l 0 until the USV also arrives atl 0 ; thus next(s0_state)=l0 holds. 63 Figure 4.3: Kripke modeling for a USV in the absence of contingencies of type A and B Then the USV heads towards the location next(s0_state) which is clearlyl 0 . It is assumed that a low-level controller implemented for the USVs will guide them to the ship's location l 0 such that all USVs are in proper formation around the ship and well-separated waiting for a new task to be assigned to them. Another case may also occur so that the USV suers a communication failure prompting it to return its base station l s . In this case, it is desirable for the USV to remain at l s till the communication link has been restored. Once restored, then the USV must meet the ship at the way-point which the ship is headed towards. If the ship along with all USVs and their corresponding UAVs are at a way-point l i1 and none of the three contingencies are impacting the mission, then the USV heads towards the next way-point l i . For i = 1, u1_state=l0 & u2_state=l0 & u3_state=l0 & a1_state=l0 & a2_state=l0 & a3_state=l0 & s0_state=l0 & contingency_type_A[1]=0 & !contingency_type_B[1] & comm_link[1]: l1; If there are civilian vessels inside the region r i but sucient number of USVs have not been sent to warn o the vessels, and the UAV a 1 corresponding to the USV u 1 is already deployed in the region to warn o the vessel, then the USV must go to the region r i to warn o the intruder 64 civilian vessel. In all such USV operations, the communication link between the USV and the ground station must be working properly for any operation to proceed. u1_state=l0 & contingency_type_A[1]>count(u3_state=r1,u2_state=r1) & a1_state=r1 & comm_link[1]: r1; If there are not II USVs surveying and clearing the debris in the path-segment p i when path blockage has been reported, then such contingencies of typeB are also handled similarly, as shown below. u1_state=l0 & toint(contingency_type_B[1])*num_USVs_for_2nd_contingency > count(u3_state=p1,u2_state=p1) & a1_state=p1 & comm_link[1]: p1; If the USV is in region r i because it was sent to warn o intruder civilian vessels and the ship is at way-point l i1 waiting for the contingency to be resolved, then once the vessels have acknowledged the warning signals of the USVs and moved away, the USV must go back to the ship's location. Similar logic holds for contingencies of type B. Figure 4.4: Strategy for handling communication failure 65 Figure 4.5: Kripke modeling for a USV with three way-points only u1_state=r1 & s0_state=l0 & contingency_type_A[1]=0 & comm_link[1]: l0; u1_state=p1 & s0_state=l0 & !contingency_type_B[1] & comm_link[1]: l0; Finally, when the USVs have successfully helped the ship in docking to its assigned location, then they head back to their base station l s . u1_state=l2 & u2_state=l2 & u3_state=l2 & a1_state=l2 & a2_state=l2 & a3_state=l2 & s0_state=docked & comm_link[1]: ls; If communication signal fails then the USV is headed towards the base station. !comm_link[1] : ls; Figure 4.5 shows the full Kripke graph for a USV based on the above transition rules. Next, transition rules are given for the UAVs mounted atop the USVs. Suppose the USV u i and UAV a i are at location l j1 , then if there are more intruder civilian vessels to warn o than the number of UAVs currently assigned to verify and detect their presence, then the UAVa i must go to the regionr j to decide whether to trigger USV response. This is desired because it is easier to manoeuvre a UAV in a busy port than a USV, but UAVs will not be able to actually resolve the contingency. For that purpose, USVs must eventually be used. For i = 1 and j = 1, 66 u1_state=l0 & a1_state=l0 & contingency_type_A[1]>count(a3_state=r1,a2_state=r1) & comm_link[1]: r1; Contingencies of type B are similarly handled as well. u1_state=l0 & a1_state=l0 & toint(contingency_type_B[1])* num_USVs_for_2nd_contingency > count(a3_state=p1,a2_state=p1) & comm_link[1]: p1; If the UAV is already in the region r j and there are less UAVs than the number of intruder civilian vessels estimated to be present by the central ground station, then the UAV must wait for more UAVs to enter the region before heading to survey the location where the intruder civilian vessels are believed to be present. a1_state=r1 & contingency_type_A[1]>count(a3_state=r1,a2_state=r1) & comm_link[1]: r1; Similar transition rule follows for contingencies of type B. a1_state=p1 & toint(contingency_type_B[1])*num_USVs_for_2nd_contingency > count(a3_state=p1,a2_state=p1) & comm_link[1]: p1; When such cases are encountered by the vessels for which no transition rules have been en- coded, then it is expected that the ship and the USVs continue performing the task they are doing or remain idle at the assigned task's terminal state. However, UAVs are supposed to be mounted on top of USVs, so they are expected to track USV's next moves in the above cases. These rules are shown below for the ship, USVs, and UAVs respectively. TRUE : s0_state; TRUE : u1_state; TRUE : next(u1_state); 67 Below, the transition rules for the contingency variables are provided. In practice, these are maintained and updated by the ground control station, so these rules simulate the ground control station. If as many USVs as the intruder civilian vessels are in a region surveying and warning them o, then all contingencies of typeA are designated the status of being resolved. This is expressed in NuSMV as follows. count(u1_state=r1,u2_state=r1,u3_state=r1)>=contingency_type_A[1]: 0; Each path-segment has an associated range [i min ;i max ] of intruder civilian vessel that may enter the region around the ship's path. It is assumed that i min = 0 so that for every segment, there is a possibility of there being no civilian intruder vessel. At location l 0 , the ship has just entered the port area, so it is expected that this area will be less crowded and hence have lower values of i max . As the ship's motion proceeds, it will encounter higher trac regions. So the middle segments have a higheri max . And during the end of ship's motion, it will be near the docking regions which will have lower civilian vessel population and hence lower values of i max . If UAVs arrive in the region where intruder civilian vessels were reported, then either the contingencies will get resolved or not. Contingency resolution here only means that when the UAV surveys the region for the intruder civilian vessel, then that vessel may have moved away or it was not there in the rst place and the sensor data was wrong or misinterpreted by ground station. Let the probability with which this can happen for every civilian vessel be p res . It is assumed that this is 50%. So if i u civilian vessels were reported in the region, then after UAVs have reached the region, the number of civilian vessels still remaining in the region may be 0; ;i u . The probability ofi 0 u civilian vessels (0i 0 u i u ) still intruding the region is clearly iu i 0 u p iu res . So if i max = 3 and let i u = 2, then the transition rule would be: 68 Figure 4.6: Resolution strategy for contingencies of type A count(a1_state=r1,a2_state=r1,a3_state=r1) >=contingency_type_A[1] & contingency_type_A[1]=2: {0,1,1,2}; Above, the following holds: contingency_type_A[1]=i u . This is in relation to the path-segment r 1 where two civilian vessels have been reported. Thus, a similar rule must be coded for all possible values of i u . The number of civilian vessels that may be reported is considered to be uniformly random. So if a segment can have a maximum of say i max = 3 vessels, then a nal transition rule may be added as follows: TRUE : 0..3; The transition rules for contingencies of type B are handled in the same way. For example, next(contingency_type_B[1]) := case count(u1_state=p1,u2_state=p1,u3_state=p1)>= num_USVs_for_2nd_contingency: FALSE; TRUE : {TRUE,FALSE}; 69 esac; No separate rule is needed for UAVs because this contingency occurs singly, and is similar to the case of contingency of type A if it is assumed that i max =i u = 1. It gets handled automatically by the nal rule: TRUE : {TRUE,FALSE}; Also, one could let the communication link be randomly maintained by the NuSMV solver or associate a probability of p comm . Say, the probability of the communication link going down is 1=3, then the transition rules for the communication link will be as follows: next(comm_link) := case TRUE : {TRUE,TRUE,FALSE}; esac; The Kripke structure for the ship, USVs, UAVs, and the ground control station (which is being simulated through the contingency and communication link variables) is incomplete without explicitly mentioning the labeling functions. For the vessels, this is obvious because the labeling functions are the assertions regarding whether a vessel is at some location or doing some task. For example, in the case of the ship, s0_state=l0 is a labeling function which asks whether the ship has entered the port region or not. These labeling functions are crucial for triggering state transitions. For example, there is no point in commanding the USVs to head out of their station until the ship has entered the port region. Next, the labeling functions for the ground control station are claried. The ground control station is indirectly being modeled through the modeling of contingencies and communication links. Thus, whether there are enough USVs in a region to resolve a contingency or whether the communication link is properly working are examples of labeling functions used to trigger and control the base station's mission progress. 70 4.4 Model Checking and Verication As explained in Section 3.3.4.1, it must be veried whether the ship docks successfully and all robots return to their station in the presence of all three types of contingencies. So LTL speci- cation to be veried is as follows: 1 := FG (s 0 = docked)^ n i=1 FG (u i =l s )^ n i=1 FG (a i =l s ) (4.1) The above formula requires the ship to be docked and both the USVs and their corresponding UAVs rested back at their base station. A fairness condition is also added such that the ship has safely reached the docking location at least once, and this is given by F (s 0 = docked). This is done to prevent livelocks due to contingencies. Some practical upper limits could also be put on the number of times each contingency could occur because NuSMV allows bounded past operators. There may also be sophisticated ways to handle such livelocks in the model itself. One simple method is given for handling the same. This fairness condition may prevent the detection of bugs in the nominal mission modeling itself because the fairness condition may force NuSMV to ignore them. So, the correctness of the nominal mission model must be veried in the absence of both contingencies and the fairness condition to detect bugs that may have been introduced by human coders. For example, a transition rule is coded for each USV as follows: u1_state=Ls & comm_link[1] & (next(s0_state)=L0 | next(s0_state)=L1 | next(s0_state)=L2) : next(s0_state); This rule ensures that the USVs will initially join the ship at l 0 or if a USV has returned to its station due to communication failure then it rejoins the ship once its link is working again. So, the USV must track the ship's next state. 71 However, the ship's next state must be a waypoint. This is enforced by the disjunction of three atomic propositions, as shown above. This is done because if the ship's next state is docked, meaning the ship is at its last waypoint and trying to dock itself, then this rule will prompt the USVs to take the docked state. But docked is not a state that the USVs can take. Therefore, if the human coder had forgotten to encode this last condition, then NuSMV will throw up an error. Thus, NuSMV allows bug correction. However, if the USV is sent to location l w in order to assist the ship while it is docking to its designated location, one could code it up as follows: u1_state=Ls & comm_link[1] & next(s0_state)=docked : L2; Here, we simply note thatw = 2 because we consider only three way-points in the current example. Ii is also important to verify that the USV's communication links are working properly before it is commanded to join the other USVs at the docking location. For the contingency-incorporated model, it is rst veried that the mission model does not have deadlocks by checking for transition relation totality using the NuSMV command check_fsm. It is conrmed that no deadlocks exist so Equation 4.1 is veried using the NuSMV command check_ltlspec. NuSMV conrms that the property holds. A model trace is generated to verify this by checking the correctness of:. This returns a counterexample in the form of a mission trace. The mission trace returned by NuSMV is trivial as it only shows the convoy moving through each way-point without the occurrence of any contingency event until the vessels and drones are rested in their desired positions. A simple kind of inconsistency that also gets detected is as follows. Suppose II = 3 and only three USVs are available for the mission and a civilian vessel enters the ship's protected space, then no USV is available for clearing the ship's path if it gets blocked. 72 Also, the ship must visit the way-points in a sequence. Thus there are many SAR constraints implicitly encoded in the mission model which may be veried explicitly in NuSMV. The previous LTL expression needs to be modied as follows: 2 := FG (s 0 = docked)^ n i=1 FG (u i =l s ) ^ n i=1 FG (a i =l s )^ F((s 0 =l 0 )^ F((s 0 =l 1 )^ F )) ^ w1 i=0 (s 0 6=l i+1 Us 0 =l i ) ^ w1 i=0 G(s 0 =l i+1 =) XGs 0 6=l i ) (4.2) This also passes the verication test. More interesting results are generated by starting the system from dierent initial states and designing more complex LTL formulas. Let USVs u 1 and u 2 along with their UAVs initially be at way-point l 2 and USVu 3 is atl 0 . Also, assume that the communication link for USV u 2 is not working properly. Other information remains the same as previously. Then NuSMV produces a counterexample when verifying: 1 which is interpreted as follows: USV u 2 returns to the base and waits for the communication link to work again. In the next time-step, its communication link is working and it goes to l 0 while USV u 1 returns to the base. Then USV u 1 joins u 2 and u 3 at l 0 . The mission proceeds nominally thereafter. Another case is obtained by adding a contingency condition to the previous initial conditions that there is a civilian vessel intruding into region R 2 . The LTL specication: 3 is given below: 3 := 1 ^ ((C II = 1) U_ n i=1 (u i =R 2 )) (4.3) C II is a shorthand for the variable \contingency type A[2]". NuSMV counterexample is as follows: The convoy proceeds up to l 1 as usual. Then blockage for path P 2 is reported while a civilian vessel is still inR 2 . So,a 1 goes tol(R 2 ) anda 2 ;a 3 go tol(P 2 ). The contingencies are still present 73 Table 4.1: Runtime statistics collected for dierent problem sizes Number of USVs Number of way-points Wall-clock time RAM 3 3 7.38 sec 30.182 MB 3 4 2.54 min 81.639 MB 3 5 5.09 min 214.021 MB 4 3 34.01 min 350.021 MB 4 4 6.04 hr 3.187 GB 4 5 16.52 hr 8.090 GB 5 3 30.70 hr 8.885 GB so the corresponding USVs head towards l(R 2 ) and l(P 2 ). Then the contingencies are resolved and the vessels and drones return to l 1 after which mission proceeds nominally. Bounded model checking was also performed using either of the two Boolean satisability problem (SAT) solvers: Zcha and MiniSat. In both cases, a counterexample was generated at the eighth level as follows: The convoy proceeds up to l 1 as usual, and then a 1 goes to l(R 2 ) because an intruder is sighted. Then communication links for the USVsu 2 andu 3 fail simultaneously. So they return to the base while u 1 reaches l(R 2 ) to warn the civilian vessel. Then all contingencies are resolved. Finally, the convoy reassembles at l 1 and mission proceeds nominally. NuSMV also reports run-time statistics. For three USVs and three way-points, the reported user time elapsed is 7:4 seconds, and the system time is 0:015 seconds. Thus, the total CPU execution time consumed is 7:415 sec. The wall-clock time for this process is 7:38 seconds. The RAM allocated for this process is 30; 906 KB 30:182 MB. The operating system is Ubuntu MATE 18.04.1 and workstation is Dell Precision Tower 3620 including eight Intel Xeon E3-1245V5 CPUs having 3.5GHz speed (but this process seems to be single-threaded) and a total of 32GB RAM. MATLAB was used to generate SMV les for dierent numbers of USVs and way-points. 74 From Table 4.1, it is observed that the running time and allocated memory rise exponentially with an increase in problem size and USVs impact run-time statistics much more drastically than way-points. 4.5 Summary A formal model of a ship escorting mission in a busy port is presented. Since running high- delity simulation experiments is computationally prohibitive, it is demonstrated how verication techniques like model checking can be used to model and verify the performance of such missions. The correctness of contingency resolution strategies is veried using linear temporal logic formulas in NuSMV software. 75 Chapter 5 Multi-robot Assembly Case Study In this chapter 1 , a case study involving assembly cells is presented and it is described how the robots and humans interact to produce assemblies eciently. 5.1 Introduction In this chapter, a manufacturing assembly line is analyzed which must carry out complex assembly tasks. The modeling approach presented in this chapter is inspired from the following robotic assembly example. Figure 5.1 shows a three-manipulator assembly setup that has been assigned to assemble a mock-up of a satellite [215]. The parts required for the assembly process may arrive at the assembly station through a conveyor system from where they are picked up by a mobile manipulator (Figure 5.2) [216{219]. These parts are then assembled to build the satellite mock-up whose assembled CAD view is shown in Figure 5.3. Figure 5.4 shows the exploded view of the CAD model, and the YouTube video [215] shows the steps for assembling this model from scratch. For large-scale, real operations, each manipulator would be assigned its station and consider multiple such cells working cooperatively to produce complex end-products. Thus, stations would be producing sub-assemblies that would be processed in each cell. Moreover, the nished products from each cell will feed into each other to accomplish end-product completion. 1 The work in this chapter is derived from the work published in [214] 76 It is generally challenging to automate assembly tasks of high complexity fully. For this purpose, human workers must collaborate with robots. Thus, human involvement is modeled, not in a far-o base station, but at the mission site itself along with the robots. The entire assembly line may still be supervised and monitored from a base station located away from the actual operation site. This mission is modeled and analyzed in the PRISM model checker [220]. 5.2 Problem Statement Given an English description of an assembly line process, it must be encoded formally in a model checker and then used to evaluate dierent contingency resolution strategies for preventing assem- bly line stoppage or failure. The assembly cells comprising of assembly stations must be modeled formally so that they can be encoded in a model checker. Human workers and robots must be incorporated in this formal model to automate assembly production. The eectiveness of humans to assist robots in recovering from faulty assemblies must be analyzed via dierent contingency resolution strategies. An assembly line can be set up and then the divide-and-conquer approach can be used to analyze it in smaller units. Those results can then be aggregated into a whole because an entire assembly line cannot be eciently modeled in model checking tools. 5.3 Overview of Approach High-level operations planning comprises of operation modeling, verication of the nominal op- erations plan, and assessment of dierent contingency resolution strategies. Initially, a nominal operation description is given using which the nominal tasks comprising the operation are iden- tied. Agents assigned to the tasks may be human workers or robots. The pre-conditions and post-conditions are identied for each task. The set of discrete states that can be assigned to each agent along with their domains and initial values is also identied. Other environment variables can also be identied to track the operation progress. Transition rules are encoded that specify 77 Figure 5.1: Three manipulators assigned for cooperative assembly operation Figure 5.2: Mobile manipulator constraints for agents to transition from one state to another in terms of tasks' pre-conditions and post-conditions. A nominal operations execution plan provides a sequence in which the available 78 Figure 5.3: CAD model for the assembled satellite mock-up tasks must be executed to meet the operation objective while adhering to pre-conditions and post-conditions of all tasks. It is assumed that the transition rules between states of a system are probabilistic. Markov decision processes are adopted for modeling these operations. When there is only one probabilistic transition rule for every possible state of the system, then the process is modeled as a discrete- time Markov chain (DTMC) in PRISM which is commonly used for probabilistic model checking. Model checking for DTMCs is done via numerical or statistical methods in PRISM. Statistical methods allow handling of larger problem sizes. The contingency resolution strategies are designed that get invoked when a contingency is reported during operation execution. These strategies may require multiple actions to be per- formed in a sequence which is simulated by appropriately synchronizing specic transition rules in PRISM. For dierent cases, dierent sets of actions may have to be invoked. For probabilistic systems, the formulated specications are veried using Probabilistic Computation Tree Logic 79 Figure 5.4: Exploded view of the CAD model for the satellite's mock-up (PCTL). Probabilistic reachability conditions may be queried, which means querying the prob- ability that the system eventually satises a set of atomic propositions. If a reward structure is dened, the expected value of dierent variables may also be computed. The approach to analyzing multiple assembly cells working cooperatively is rst to analyze specic groups of assembly stations (referred to as a cell) using formal methods and then perform a Monte Carlo analysis of the entire system working together. This is because large systems cannot be entirely modeled in model checkers due to computational memory requirements. However, if the more extensive system is decomposed into smaller subsystems, then it is possible to encode them in a model checker and analyze them eciently. Thereafter, performing an aggregate analysis for the whole system using Monte Carlo simulations would not be so memory-intensive because model checking results could be used. It may be possible to analyze the entire system by only using Monte Carlo simulations without performing any decomposition and subsequent model checking, but that appraoch would fail to leverage the rich eld of temporal logic model verication. Since the probabilistic model checking community has made signicant progress in developing model 80 checkers to solve Markov models under temporal logic constraints, it has become a popular tool for use in robotics research as part of the broader application of formal methods to robotics. 5.4 Modeling of Assembly Cell 5.4.1 Nominal Operation Description A simple assembly cell consisting of N assembly stations S 1 ;S 2 ; ;S N is described below. A visual schematic for N = 3 is presented in Figure 5.5. The variables used in the Figure are explained in Section 5.4.2. This model allows human intervention at each station when the desired production process fails. Each station is also provided with one dexterous mobile manipulator which performs all assembly-related tasks. The robot designated at stationS i is denoted byM i . A conveyor system is modeled to bring parts from the storage area. The parts are then picked from the conveyor belt and transported to the designated assembly station using a simple pick-and- place mobile robot. Each station S i is provided with such a robot denoted by P i . This conveyor belt system is referred to asB 1 . Figure 5.5: Schematic representation of multi-robot assembly cell 81 There is another conveyor belt system B 2 operating across the stations. This is because it is assumed that serial production has been set up across the stations. The job of this conveyor system is merely to transfer parts from the preceding station to the next station for further processing. So, once the assembly robots have nished their tasks and are ready to transfer the part to the next station, they place the assembled unit on the conveyor system which then transfers the parts to the next station. Thus, the assembly process starts at the rst station, and the fully assembled part is produced at the last station. Then another pick-and-place robot is designated to pick the part and transfer it to the desired location. Thus, an assembly cell consisting of N stations deploys 2N + 1 robots. One could alternatively also model the cell to deploy fewer robots by combining the responsibilities of various robots into a single robot's tasks. 5.4.2 State Variables The system model naturally translates into a set of variables. As shown in Figure 5.5, the operation begins with two sets of parts being delivered at the rst assembly stationS 1 . The number of rst set of parts delivered to station S 1 is denoted by n(1; 1) and the number of second set of parts by n(1; 2). The convention used is to use the rst index to denote the assembly station and the second one to denote a variable index. Then the assemblies produced at station S 1 is denoted by n(1; 3). For all other assembly stationsS i such thati> 1, one set of parts arrive via the conveyor system B 1 and a set of partially completed assemblies arrive from their preceding stations S i1 . The number of parts delivered by robot P i is denoted byn(i; 1) and the number of parts arriving from previous stations is denoted byn(i; 2). Thus, a new set of parts is delivered to stations, and these are tted properly in the assemblies arriving from the preceding stations. Such parts will be called intermediate units (denoted by n(i; 1)) and the partial assemblies will be referred to as sub-assemblies (denoted byn(i; 2)). The total assemblies produced by each station at any time is tracked using the variable n(i; 3). For a given cell, one is generally interested in the total number of nal assemblies produced by the last station, n(N; 3). 82 All variables naturally have a nite range associated with it due to space constraints. For simulation purposes, even smaller ranges may be imposed due to solver constraints. All the variables declared above are non-negative integers. The upper bounds for these variables are also impacted based on whether numerical or statistical model checking methods are used. If the numerical method is used, then the model size must be kept small. Therefore, high upper bounds cannot be placed for these variables. On the other hand, statistical methods provide approximate results but accommodate larger model sizes, thereby allowing for comparatively higher upper bounds. All variables are initially expected to be zero. However, manufacturing units tend to keep multiple intermediate units at the station before the conveyor operation, and assembly process begins. This buering of parts is done to avoid any bottlenecks in further operations due to a shortage of intermediate units required to undergo assembly. This concept is referred to as initial part buer. Thus, all the variablesn(i; 1) andn(1; 2) may be greater than zero, sayN I . All other variables are zero initially. For example, below, the range of a variable n(2; 3) and its initial state are declared. Thus, at any instant, there must not be more than ve assemblies produced by station S 2 . The variable n(2; 3) is shortened to n23 in PRISM. This convention is followed throughout this chapter. const int max_assemblies = 5 ; n23 : [0..max_assemblies] init 0 ; Next, the modeling of assembly robots M i is described. The description of robot M 1 diers from that of the rest because its model is a simplied version of that of others. So, rst the model for robotM i is described whereNi 2. Figure 5.6 provides a visualization aid for this model. The sub-assemblies rst arrive from station S i1 to stationS i . Here, the sub-assemblies rst undergo a pre-detection process by robot M i to ensure whether the arriving sub-assemblies are as expected and ready for further processing. If pre-detection 83 Figure 5.6: Probabilistic transition model for assembly robot succeeds with probabilityp 1 , then the sub-assembly is ready for the assembly process. Otherwise, the sub-assembly must undergo a pre-repair process. The number of parts queued for pre-repair is denoted by x(i; 1) whereas the number of parts queued up for assembly process is denoted by x(i; 2). Each stationS i has two storage location referred to as contingency buers:C(i; 1) andC(i; 2). Here, parts which fail during repairing operations are sent for repair by a human helper. Each sta- tionS i is assigned its own human helperH i . So if the pre-repair process succeeds with probability p 2 , then the sub-assembly is queued up for the assembly process. However, if during the pre-repair process, robot M i fails to resolve the problems in the sub-assemblies, then those sub-assemblies are queued up in contingency buerC(i; 1). The number of parts queued up in contingency buer C(i; 1) is denoted by c(i; 1). If the assembly process succeeds with probability p 3 , then the assembled part is queued up for a post-detection operation where the robot again conrms using image processing algorithms whether the desired result has been achieved. Otherwise, if the assembly process fails (this is likely because some assembly procedures may be very complex to carry out and may test robot capabilities to its utmost limit), then the robot merely queues up the part for attempting its assembly later again. The number of parts queued up for post-detection is denoted by x(i; 3). 84 If post-detection succeeds with probability p 4 , then the sub-assembly is ready for being trans- ferred to the next station,S i+1 . Otherwise, the sub-assembly must undergo a post-repair process. The number of parts queued for post-repair is denoted by x(i; 4). If the post-repair process succeeds with probability p 5 , then the sub-assembly is ready for being transferred to the next station, S i+1 . However, if during the post-repair process, robot M i fails to resolve the problems in the sub-assemblies, then those sub-assemblies are queued up in contingency buerC(i; 2). The number of parts queued up in contingency buer C(i; 2) is denoted by c(i; 2). There is a human helper assigned to stationS i denoted byH i who is responsible for both the contingency buers and attempts to resolve each sub-assembly sent to it. If human succeeds with probability p 6 each time he/she attempts contingency resolution at contingency buer C(i; 1), then these resolved sub-assemblies are queued up for assembly by robot M i . For sub-assemblies held atC(i; 2), the human may succeed with the same probability, and resolved sub-assemblies are declared as nished sub-assemblies for that station and ready for transfer via conveyor belt systemB 2 to the next assembly station,S i+1 . Model for robot M 1 is a simplied version of the above as it does not have to perform the pre-detection and pre-repair operations. This is because the two sets of parts are coming from the storage area and not from a station within the cell. So it is assumed that parts arriving from the storage area do not need to be veried. They will already be veried by the conveyor belt transport system B 1 but automated verication by conveyor system is not modeled here. Consequently, there is also no contingency buerC(1; 1). Robot M 1 directly performs its assembly operation followed by post-detection and post-repair operations. Based on the above modeling, one can check that the PRISM model contains 9N 2 variables and N + 1 modules. All the intermediate variables x(i;) referred to above have size constraints as they represent the process of storing parts. The lower bound for each variable is zero while the upper bound is user-dened. It is assumed that the upper bound on all variables is N max except 85 for n(N; 3) whose upper bound is denoted by N . The rationale for doing so is to be able to compute the expected production time for assembling as many parts as computationally feasible. So N can be made large despite keeping N max smaller. 86 Figure 5.7: Schematic representation of assembly processes in stationS i fori> 1 for strategys 1 . The rectangles represent storage containers, and the rounded rectangles represent processes involved during assembly. 87 5.4.3 Transition Rules The above description of the assembly model naturally results in probabilistic transition rules which can be coded in PRISM. The pre-detection process of robot M 2 in stationS 2 is expressed as the following transition rule in PRISM. const double p1 = 0.8; const int max_units = 5 ; [tick2] x21<max_units & x22<max_units & n22>0 -> (1-p1):(x21 =x21+1) & (n22 =n22-1) + p1:(x22 =x22+1)&(n22 =n22-1); According to the above rule, when a sub-assembly transferred from preceding station is pre- detected and succeeds it will increase the variable x(2; 2) by one and decrease n(2; 2) by one. Otherwise, if pre-detection ags the sub-assembly as having errors, then x(2; 1) is increased by one andn(2; 2) decreased by one. This rule must only be invoked if the variable bound constraints allow to do so. tick2 is a label for this transition rule. Such labels are used for synchronizing one rule with other rules. This will be elaborated in the next section. Similar rules as above exist for pre-repair, assembly, post-detect, and post-repair processes. Human helper H 2 must operate both the contingency buers and keep clearing the faulty sub-assemblies arriving in them. Consequently, the following rules hold for contingency buer C(2; 1): const double p6 = 0.95; const int max_units = 5 ; [tick7] x22<max_units & c21>0 -> p6 : (x22 =x22+1)&(c21 =c21-1); [tick7] !(x22<max_units & c21>0) -> true; The rst rule states that if there is a faulty sub-assembly in the contingency buer, and there is space for at least one more sub-assembly to be queued for assembly, then human helper H 2 attempts resolving the sub-assembly and may succeed with probability 95%. The second rule 88 only ensures that if the size constraints do not allow contingency resolution, there is a rule that explicitly states this with both rules sharing the same label. Accounting for all possibilities explicitly using transition rules with the same labels is required for synchronizing multiple rules properly. This will be elaborated in the next section. 5.4.4 Nominal Operation Strategy Next, the nominal protocol is presented through which assembly operations proceed. The entire assembly process described in the previous section keeps repeating in cycles. Each cycle comprises of eight steps involving specic operations like pre-detection and post-repair. Each station is implemented as a separate PRISM module. In the rst step of each cycle, both the conveyor systems are implemented concurrently. The conveyor system B 1 is supposed to bring parts from the storage and stop for the robots P i to pick the parts and take them to their respective stations. For eciency of the operations, it is assumed that the conveyor system would have been automated such that the belt stops such that the parts desired by robot P i are optimally located. For each station, it must be checked that the constraint n(i; 1) < N max for delivering intermediate units. Additionally, for the rst station, which also receives a second set of intermediate units, one must also checkn(1; 2)<N max . Therefore, N + 1 constraints must be checked for the rst conveyor system operation. The conveyor system B 2 transfers nished sub-assemblies from one station to the next. For N stations, N 1 such transfers can take place if size constraints allow so. For each station S i excluding the rst, one must only check two size constraints: n(i 1; 3)> 0 and n(i; 2)<N max . The two size constraints can be combined into one using the conjunction operator. Thus, N 1 constraints must be checked for the second conveyor system operation. All these constraints are independent of each other. So the total of 2N constraints lead to 2 2N cases, and transition rules are encoded for each of these cases for each station. Thus, there will be N 2 2N transition rules relating to the rst step, and hence, they must all be synchronized 89 together. For this purpose, an additional clock module is used which cycles through eight steps. According to the nominal strategy, each cycle comprises of eight steps. Hence the clock has eight steps. A label tick1 is created for its rst step and label allN 2 2N rules also as tick1. Another simplifying assumption is made that both the conveyor systems are working deterministically. If dierent operations in these conveyor systems were probabilistic and had dierent probability values, then they could not be synchronized. This is a reasonable assumption because conveyor systems are very ecient. Most problems generally arise in the robotic aspect of the operations. In the steps second through sixth, the assembly robot performs its pre-detection, pre-repair, assembly, post-detection, post-repair operations consecutively. Then in the seventh and eight steps, the human helper seeks to resolve faulty sub-assemblies in contingency buersC(i; 1) and C(i; 2) respectively. No human activity is allowed inside the station while the robot is performing any of its tasks because of safety purpose. The goal is to compute the expected production time for N parts. Therefore, a reward structure labeled as time_duration must be dened. Since conveyor operations are fast, and robotP i may take some time to transfer intermediate units from the conveyor belt to the stations, it is assumed that it takes around 10 seconds. Time is recorded in minutes. So the time taken in the rst step is rounded to 0:2 minute. Since computer vision algorithms have become very fast, signicant time spent during the detection operations is while manipulating the camera to capture images from dierent angles or scan point clouds. So detection routines are also assumed to take 0:2 minute. The repair and assembly processes take two minutes. The human helper takes ve minutes. 5.4.5 Contingency Resolution Strategies The strategy described in the previous section is referred to as strategy s 1 . A nondeterministic version of this protocol referred to as strategy A is also designed, which will be implemented as an MDP in PRISM. The rst step remains the same. In the second step, the assembly robots 90 perform one of their detection actions taking a maximum of 0:2 minute. In the third step, the assembly robots perform one of their assembly/repair actions, taking a maximum of two minutes. In the fourth step, human helper repairs parts in one of the contingency buers requiring ve minutes. Four more strategies are provided to resolve the contingencies which are variations of strategy s 1 . Instead of keeping a human helper at each station, there could be only one human assigned to the entire cell. This is referred to as strategy s 2 . Each cycle of strategy s 2 is implemented as a concatenation of N cycles of strategy s 1 . Each of the N cycles are referred to as sub-cycles. In each sub-cycle except for the last one, there are eight steps. In the last sub-cycle, there are only seven steps. Thus, there are a total of 8N 1 steps in each cycle of strategy s 2 . The rst six steps of each sub-cycle are same as strategy s 1 . In the seventh step of each sub-cycle, the human seeks to resolve faulty sub-assemblies in con- tingency buerC(i; 1), and in the eighth step of each sub-cycle, the human seeks to resolve faulty sub-assemblies in contingency buer C(i; 2). Since the rst station does not have contingency buerC(1; 1), the contingencies for contingency buerC(2; 1) are resolved in the seventh step. Thus, for i-th sub-cycle such that i<N, the human helper resolves faulty sub-assemblies at contingency buerC(i+1; 1) in the 7i-th step, and at contingency buerC(i; 2) in the 8i-th step. For the last cycle, faulty sub-assemblies at contingency buerC(N; 2) are resolved in its seventh step, and there is no eighth step, hence 8N 1 total steps. Strategy s 3 is similar to s 1 but with a slightly dierent protocol of resolving contingencies. Suppose, strategy s 1 had only seven steps. The human helper at each station resolves faulty sub-assemblies in the rst buer in the seventh step for odd cycles, and in the second buer in the seventh step for even cycles. Thus, a cycle for strategy s 3 is implemented as a concatenation of two cycles of strategy s 1 each curtailed to only seven steps. Thus, humanH i such thati> 1 resolves faulty sub-assemblies at contingency buerC(i; 1) in its seventh step, and at contingency buerC(i; 2) in its fourteenth 91 step. HumanH 1 resolves faulty sub-assemblies at contingency buerC(i; 1) in both the seventh and fourteenth step. Contingencies of a dierent nature may also arise during assembly processes such that the human analyses the faulty sub-assembly and gures out that if he/she makes a minor x and sends the part back to the assembly robot for another pass through the sub-assembly, then the robot is highly likely to succeed. This procedure also saves the human's time because rather than fully correcting the error with the sub-assemblies, he/she is only making a minor x and using the robot for the rest of the work. For example, a human decides that if a washer is slightly tweaked, then the assembly robot will succeed with the rest of the assembly. Also, even if the human worker feels that the faulty sub-assembly might take signicant time to repair, he/she decides to send the part back to the automated assembly line hoping the robot to repair it successfully. Since this model of the human worker is aimed at saving time, it is assumed that he/she takes only two minutes for the contingency resolution operation. So it is assumed that the human worker succeeds in resolving contingencies with p 61 = 80% probability, fails and puts the part back in the contingency buer with p 62 = 5% probability, and sends the part for repair by the robot with p 63 = 15% probability. So, the contingency resolution rule mentioned in Section 5.4.3 gets transformed as follows. [tick7] x21<max_units & x22<max_units & c21>0 -> p61 : (x22 =x22+1)&(c21 =c21-1) + p62 : true + p63 : (x21 =x21+1)&(c21 =c21-1) ; The probability values may be varied by the user for dierent settings. This helps us simulate the human worker who will actively use the assembly robot multiple times to speed up his/her own repairing task. Applying this model of the human worker on strategy s 2 produces strategy s 4 , and on strategy s 1 produces strategy s 5 . 92 Table 5.1: Eect of initial part buer on production time Initial Part Buer Strategy 1 Production Time (in min) Strategy A Production Time (in min) 0 45.17 43.10 1 45.07 42.90 2 44.93 42.70 3 44.75 42.50 4 44.55 42.30 Table 5.2: Eect of contingency resolution probability on production time Contingency Resolution Probability Strategy 1 Production Time (in min) Strategy A Production Time (in min) 0.50 48.67 46.58 0.60 47.52 45.43 0.70 46.64 44.56 0.80 45.96 43.88 0.85 45.67 43.59 0.90 45.41 43.33 0.95 45.17 43.10 1.00 44.96 42.89 5.5 Model Checking and Verication 5.5.1 Run-time Analysis The computational performance of dierent contingency resolution strategies under dierent op- eration settings is studied to gain better insights into the complexity of the process. The operating system is Ubuntu MATE 18.04.1 and workstation is Dell Precision Tower 3620 including eight Intel Xeon E3-1245V5 CPUs having 3.5GHz speed and a total of 32GB RAM. For a 5-station cell running strategys 1 , withN max = 2 andN = 3, probabilities of 90% for all operations, and zero initial buer, it took 82:09 seconds to compute the reachable states using breadth-rst search in 529 iterations. The resulting DTMC had about 116 quadrillion states and 483 quadrillion tran- sitions. The underlying transition matrix consisted of 0:5 million nodes. The expected time for the cell to produce three parts is then computed using PRISM's hybrid computation engine with the following expression given below: 93 Table 5.3: Computational performance of s 1 with varying parameters # stations N max N Time for model building (in sec) Time for model checking (in sec) 2 2 3 0.63 0.05 2 2 2 0.57 0.04 3 2 2 7.77 0.26 3 2 3 7.43 0.38 3 1 3 0.16 0.41 3 1 5 0.25 0.94 3 3 3 59.30 0.42 4 2 3 38.86 4.66 4 1 3 0.54 4.13 4 3 3 367.11 3.05 5 1 2 1.09 30.13 5 1 3 1.19 46.40 5 2 2 99.65 37.10 R{"time_duration"}=? [ F n53=3 ] where time_duration is the label given to the reward structure dened in Section 5.4.4. The sampling method is used because the numerical method cannot handle such a large problem size. The expected production time is computed to be 31:81 minutes based on 95% condence level and condence interval width of 30 seconds. It took 828 sampling iterations and 59:24 seconds. The average path length explored was 66. If instead, there was a 2-station cell, then the DTMC could be solved numerically in 6:18 seconds with 88 Jacobi iterations requiring 64:6MB RAM for the expected production time of 14:19 minutes. Table 5.3 records the computational time for strategy s 1 for the same cell setting as above by varying the three parameters that aect the problem size: N;N max ;N . Tables 5.4 and 5.5 do the same for strategies s 2 and s 3 respectively. As expected, strategy s 1 can accommodate larger problems sizes more eciently than strategy s 2 which in turn can accommodate larger problem sizes more eciently than strategy s 3 . Changing N max primarily increases the time for model building whereas N primarily increases the time for model checking. Strategy s 4 and s 5 have similar computational performance as s 2 and s 1 respectively. 94 5.5.2 Assembly Cell Analysis The results for a single assembly station are analyzed rst. All variables are given an upper bound of four, and the expected time to produce ten assemblies is to be computed. And the initial part buer is zero for each station. An assembly cell is perfect if all success probabilities are one. The expected production time for a perfect cell is the lowest bound achievable for the given transition system. These values are 26 and 26:12 minutes and strategiess 1 andA respectively. It is assumed that a set of realistic values are used for the probabilities. It is assumed that the human worker succeeds with a probability of 95% while all robot's actions succeed with 80% probability. Table 5.4: Computational performance of s 2 with varying parameters # stations N max N Time for model building (in sec) Time for model checking (in sec) 2 2 3 1.50 0.07 2 1 3 0.05 0.05 2 3 3 9.04 0.05 2 2 4 1.78 0.09 3 1 2 0.46 0.37 3 2 2 47.51 0.50 3 2 3 47.04 0.78 3 2 5 67.54 2.19 Figure 5.8: Eect of dierent strategies on expected production time 95 Table 5.5: Computational performance of s 3 with varying parameters # stations N max N Time for model building (in sec) Time for model checking (in sec) 2 2 3 1.45 0.08 2 3 3 7.43 0.07 2 3 4 8.87 0.08 2 4 5 34.89 0.13 3 2 3 18.25 0.55 3 3 4 219.01 0.90 4 2 3 133.89 6.34 5 1 3 2.91 119.00 Table 5.6: Eect of human characteristics on production time for strategy s 1 Contingency Resolution Probability Contingency Resolution Duration Expected Production Time (in min) 0.99 5 45.01 7 46.72 10 49.31 0.80 5 45.96 7 48.05 10 51.18 Table 5.7: Eect of human characteristics on production time for strategy A Contingency Resolution Probability Contingency Resolution Duration Expected Production Time (in min) 0.99 5 42.93 7 44.50 10 46.82 0.80 5 43.88 7 45.79 10 48.61 Table 5.1 shows that increasing the initial part buer from zero to four decreases expected production time by more than 30 seconds for both strategies s 1 and A which is very signicant for factory throughput in the long run. The nondeterministic station comprising of cycles of four steps shows a decrease of 0:2 minutes per row because the addition of one item in the initial buer helps skip one conveyor operation. This eect is not so vividly noticeable in the DTMC version. 96 Increasing the contingency buer size does not have any notable eect. But increasing human resolution probability from 50% to 100% reduces time signicantly for both strategies (Table 5.2). Probabilistic model checking may also be used as a recruitment tool. Suppose a fast worker could nish resolving contingencies in ve minutes, and an expert worker could succeed with a probability of 99%. Let a slow worker take ten minutes and a less skilled worker succeed with probability 80%. Then, the expected production times for all four combinations of the worker is recorded in Table 5.6 for strategy s 1 and in Table 5.7 for strategy A. For both strategies, a slow worker is a poor choice. Even if the slow worker does not take twice as much time, but only seven minutes even then a slow worker is not preferable. Thus, at 80% success rate for robot's actions, there does not appear to be any trade-o between being slow and being less expert in this mission scenario. But these observations would vary for dierent robot's success probabilities. But this tool may be applied similarly to study those cases as well as more complicated assembly cell models where there are more human factors involved. Since the problem size above is small, numerical model checking methods are used in PRISM to generate the above results. Hence, multiple solution runs need not be executed because the numerical methods converge to the same number in every run. The problem with analyzing multi-station assembly cells as MDPs is state space explosion which makes model building, as well as model checking, very memory-intensive and time-consuming, if not impossible. However, the DTMC protocol of strategy s 1 may be extended to multi-station assembly cells. A 2-station cell can be simply obtained by ignoring operations of station A 3 . For such a cell, suppose all variables are given an upper bound of two, and the expected time to produce three assemblies is to be computed. And the initial part buer is one for each station. The expected production time for a perfect cell is 10:2 minutes. Now let human succeed with 95% probability, robot assembly with 90% and other operations with 80%. The expected production time of strategy s 1 is compared with four other strategies in Figure 5.8 and the time is shown as the percentage increase with respect to that of the perfect 97 cell. The simulation is run 100 times to report statistics at 95% condence level, and width of 0:5 minute. On an average, strategys 5 performs the best but only marginally better than strategys 4 . Strategy s 2 shows signicant improvement over strategy s 1 on account of using only one human to balance the task of two humans eciently. However, when the two humans organize themselves better as in strategy s 3 , then they outperform strategy s 2 by a small margin. These strategies are compared only for 2-station cells because model checking for strategies s 2 ;s 3 ;s 4 becomes very time-consuming for 3-station cells. This is because these strategies required coding around 22 steps per cycle for three stations, and the modeling of the conveyor system triggers scaling issues. The conveyor system must take into account 2 6 possibilities arising due to storage-station interactions which for the three assembly stations leads to 3 2 6 transition rules. PRISM run-time statistics illustrate these issues clearly. Model construction for two stations with strategy 2 took 1:9 seconds, having around 4:3 million states and 6:5 million transitions. Statistical model checking took 0:14 seconds. But model construction for three stations with strategy s 2 took 21:4 seconds, and verication took 2:1 seconds. The 3-station assembly cell operating with strategy s 1 is considered next, and same variable upper bounds and initial part buer are used as before. If all success probabilities within a station are at 90%, it is referred to as a low station. High stations have success probabilities throughout at 99%. The conveyor system is assumed to be operating perfectly. A perfect cell would take 12:8 minutes. Figure 5.9a, shows the expected production time as the percentage increase with respect to that of the perfect cell for all 2 3 combinations of high and low stations in the cell. For example, LHL on x-axis denotes that the rst and third stations are low, and the second one is high. The rst station being low is not as critical as the next two stations being low. A similar trend emerges for strategy s 5 (letting human helper 3-way split to be the same as 80 15 5%) in Figure 5.9b. At these high probabilities for the 3-station assembly cell, both strategies start performing similarly. 98 (a) Strategy s 1 (b) Strategy s 5 Figure 5.9: Eect of dierent combinations of stations on expected production time. 5.5.3 Aggregate Analysis of Large-scale Assembly Operations This section focuses on decomposing large-scale assembly operations into smaller and relatively independent assembly cells, and then analyzing each cell using the probabilistic model checking 99 Figure 5.10: Scenario for multiple assembly cells interacting with each other tool, PRISM. Finally, the results of model checking are aggregated at the highest assembly op- eration level to gain better insights into the system. A production line is set up across multiple assembly cells, as shown in Figure 5.10. The gure shows ve cells labeled as A;B;C;D;E. It is assumed that the probability values representing operational eciency are the same for each station within a cell. The specications for each cell are given below:- CellA comprises of two stations and executes strategy s 1 . N max = 3; N = 4; N I = 2; p 1 = 90%; p 2 = 90%; p 3 = 90%; p 4 = 90%; p 5 = 90%; p 6 = 95%. CellB comprises of three stations and executes strategys 5 . N max = 2; N = 5; N I = 0; p 1 = 80%; p 2 = 85%; p 3 = 90%; p 4 = 95%; p 5 = 80%; p 61 = 80%; p 62 = 5%; p 63 = 15%. CellC comprises of two stations and executes strategy s 5 . N max = 3; N = 3; N I = 3; p 1 = 95%; p 2 = 90%; p 3 = 90%; p 4 = 90%; p 5 = 95%; p 61 = 80%; p 62 = 5%; p 63 = 15%. CellD comprises of three stations and executes strategys 2 . N max = 2; N = 3; N I = 0; p 1 = 85%; p 2 = 80%; p 3 = 90%; p 4 = 80%; p 5 = 85%; p 6 = 90%. CellE comprises of ve stations and executes strategy s 3 . N max = 3; N = 4; N I = 1; p 1 = 85%; p 2 = 80%; p 3 = 90%; p 4 = 87%; p 5 = 86%; p 6 = 92%. The parts produced at cell A and cell B feed into cell C. Also, parts produced at cell C and cell D feed into cell E. So discrepancy between the operations at cell A and cell B may create 100 bottleneck situations for cell C, and similarly, cells C andD may create bottleneck situations for cellE. For example, cellA may produce parts at the rate of 3 minutes per part, and cell B at the rate of 5 minutes per part. So, cell C must wait for 2 minutes to let the part from cell B arrive assuming that all cells start their operations at the same time because cell C needs parts from both the cells to proceed. PRISM places substantial constraints on the possible problem sizes that can be solved. There- fore, it is not computationally feasible to try to model the entire production line in PRISM. Each cell can be individually analyzed according to the modeling approach using PRISM software de- scribed in Section 5.4. The PCTL specications may be used to compute the expected production time forN parts at each cell, sayT . Thus, the time taken to produce a single part by each cell may be estimated to be T 1 = T =N . One could instead have queried PRISM to compute the expected time to produce one part which would be faster, but a better estimate can be found by averaging over the time to produce multiple parts because this is representative of a longer run of assembly operations. UsingT 1 data for each cell, one may reason about the potential bottlenecks in a production line similar to Figure 5.10. The cell settings may be modied to minimize such bottlenecks. Rather than the user writing a new PRISM script for each dierent cell settings which would be very time-consuming and prone to errors, an automated way is used to generate PRISM scripts in MATLAB for the specic parameter values of a cell. Thus, it becomes feasible to consider dierent possible cell settings and choose the ones which provide the best trade-o. PRISM codes are generated for cells A;B;C;D;E automatically in MATLAB based on the input cell parameters. One can then query the expected time for the cells to produce N parts from which the expected time to produce one part can be approximated. The expected time T 1 for each cell to produce one part is given below. Cell A requires 4:475 minutes, and cell B takes 5:894 minutes per part. These are expected values from statistical model checking in PRISM over 100 runs. Cell B is employing its optimal 101 strategy. However, cell A employs two humans and strategy s 3 . So another strategy could be found where fewer humans are employed even if this results in worsening of cell A's production time as long as it does not overshoot 5:894 minutes. This is because cell C would not be able to proceed unless parts arrive from both cells A and B. If cell A switches to strategy s 2 , then it employs only one human and brings down the operational costs signicantly and takes 4:482 minutes per part. Similarly, cells C and D must feed parts to cell E, but cell C takes 4:406 minutes and cell D takes 8:115 minutes per part. Cell C is employing its optimal strategy and requires two humans. However, even if one switches to strategies requiring fewer humans, the production time does not increase. In such cases, there may be a user preference to achieve decreased production times as it also leads to lower operational costs. However, the bottleneck issue remains unresolved because there is still going to be a signicant gap between the time taken by cells C and D to produce their assemblies. So cell D's production time may be brought down without bringing any more humans. Cell D is employing strategy s 2 and requires only one human but if one switches to strategy s 4 it still requires one human and it brings down the production time to 7:673 minutes per part. The probability values associated with the human worker for cell D's strategy s 4 is set to p 61 = 80%;p 62 = 5%;p 63 = 15%. The few seconds saved as a result of this adjustment and the signicant decrease in the operational costs due to reducing human workers translates to huge prots for the manufacturing operations in the long run. This analysis is fully automated in MATLAB, and the user does not need to interact with the model checker directly. Thus, the user only needs to specify his/her competing interests. For example, it was specied that the human costs as well as reduce bottlenecks must be brought down as much as possible. Simply optimizing the performance for each cell individually would not suce. Here searching was restricted to strategies. The probability of human resolution for cell D was also chosen. This implies choosing a human worker with that specic skill level to work in the cell. However, one could search over other cell parameters like the initial part buer, storage limits, and also 102 the other probability values, if that is allowed for the operation. That is again straightforward multi-parameter optimization problem and can be similarly encoded in MATLAB. CellE produces parts at the rate of 9:116 minutes per part. Based on these production rates, one may perform queue-theoretic analysis to study the generated throughput. However, that is beyond the scope of the present work. Thus, with this approach, it is demonstrated with an example that analysis for fteen assembly stations can be analyzed, whereas using PRISM, it is dicult to perform analysis for more than ve stations eciently. Similarly, large-scale assembly operations could be analyzed by decomposing it into smaller cells and reasoning about cell settings. 5.6 Summary A simplied model of production lines and assembly cells is presented that mimics the real-world factory layout. Then this model is encoded in PRISM and PRISM's analytical tools are used to study the eect of dierent mission settings on the mission performance. The performance of dierent contingency resolution strategies is compared, thereby indicating the usage of model checking to plan for contingencies. The subsystems are analyzed individually, and their results are aggregated to evaluate the performance of the proposed operations plan for the entire system, thereby helping to design more optimized systems. It is demonstrated how the cell settings could be modied based on feedback from the model verication results to optimize production performance. 103 Chapter 6 Incorporation of Contingency Tasks in Nominal Task Allocation In this chapter 1 , a proactive approach for incorporating potential contingency tasks in task- schedules of multiple robots assigned to a complex mission is presented. 6.1 Introduction The deployment of robot coalitions requires optimal task assignment. Such problems are generally formulated as multi-robot task allocation (MRTA) problem [222,223]. In this chapter, the problem scenarios considered involve single-task robots, multi-robot tasks, and a time-extended assignment problem (ST-MR-TA). However, it is solved using an iterative application of MRTA's instanta- neous assignment formulation (ST-MR-IA). Since this can be formulated as a set partitioning problem which is strongly NPhard, one naturally chooses to apply heuristics-based computa- tionally ecient approximation algorithms. Heuristics approaches generally identify feasible tasks based on precedence constraints, enumerate feasible coalitions based on resource constraints, and then apply greedy heuristics for scheduling coalitions to tasks. The heuristics are designed to 1 The work in this chapter is derived from the work published in [212,221] 104 Figure 6.1: Example involving USVs where a contingency task impacts the mission minimize the expected mission completion time or the maximum ending time of all the tasks. This is often referred to as makespan (or bottleneck) optimization. In this chapter, there are additional features associated with the execution of mission tasks. A mission task can be interrupted, and its assigned robots can be reassigned to another mission task. A task may cause a bottleneck by keeping its assigned robots idle, which leads to poor resource utilization. This feature of interrupting tasks is incorporated to achieve lower mission completion times. Next, task execution may start even when a fraction of the team assigned to the task has arrived at the task location. This is because some robots in the team may take a larger duration to arrive at the task location. If robots arriving early remain idle while waiting for the other robots to arrive, the assigned coalition is not being properly utilized in nishing the task. When the minimum number of robots required by the task for execution have arrived at the task location, they start working and coordinating with other robots after they arrive to complete the task. Cost-eectively deploying multi-robot teams requires mission planning to be automated. This involves the identication of tasks required for completing the mission while accounting for inter- task dependencies and then allocating robots to these mission tasks. The planned execution of mission tasks may get interrupted by the occurrence of unexpected events. The resumption of nominal mission work ow may require the execution of additional tasks. These new, additional 105 tasks are referred to as contingency tasks. Throughout the chapter, it is assumed that these unexpected events adversely aect the mission work ow because they require additional work for resolution and create unnecessary diversions. But they are not catastrophic because they do not cause mission failures and can be resolved through the execution of contingency tasks. Multi- robot teams should eciently handle such contingency tasks to ensure the safety and security of their corresponding logistics operations. Below, three concrete examples to clarify the notion of contingency tasks are presented: ? Figure 6.1 shows an example of a mission where multiple USVs must escort a ship to its designated berth location in a congested port. Routine mission tasks include: surveying the area and leading the ship to the dock. Suppose a monitoring system ags a foreign object that has been detected near the planned path of the ship, but this may be a case of false detection with a probability of 10%. For example, a small civilian vessel has appeared sud- denly, and if the ship continues proceeding, it may generate waves that may be catastrophic for the occupants of the civilian vessel. This prompts the execution of a contingency task with 90% probability requiring the closest USV to go near the object/vessel and ensure that it does not collide with the ship by issuing warning signals or contacting port authorities for intervention. The escorting task can proceed only after this contingency task is completed. ? Consider an automobile experiencing heavy trac congestion so that it may run out of gas before ending at its desired goal location if the congestion persists for a long time. If that happens, the vehicle must reach a nearby lling station and refuel. The refueling operation is a contingency task. If the probability of trac congestion being resolved is 90%, then the probability of refueling task impacting the mission is 10%. ? Returning back to the example of the escorting ship. Suppose that debris has been spotted in or around a narrow channel. Due to water current, the debris may ow into the nearby channel with 50% probability and render the channel nonoperational for navigation purposes. 106 This gives rise to a contingency task with 50% probability requiring a nearby USV to go to the channel and ag the location of debris for immediate removal by port authorities. The ship and the escorting USVs need not arrive close to the channel for debris clearing to proceed. Only the assigned USV approaches the debris and coordinates with the central mission controller to clear the blockage for the convoy to proceed while the convoy itself may be stalled at a safer distance. One may point out that these contingency tasks may be handled at the trajectory planning level if and when the need arises. Most literature dealing with contingency resolution and failure recovery is devoted to developing eective online strategies to the specic failures that may arise during their multi-robot missions. Then a distributed framework is implemented that results in the application of these specic failure strategies based on the just-in-time principle during the real-time mission execution. Most task allocation work does not consider contingency factors during the allocation and scheduling phases. Contingencies are handled on-the- y. Thus, most works in contingency management in robotic task allocation paradigms do not involve centralized approaches. Most of the related papers focus on devising specic methods for dierent failure modes, applying these to the robotic systems as and when required and analyzing the success rate. However, there are benets in considering a centralized framework to handle these contingency tasks while accounting for their uncertainty. It is assumed that the tasks required to resolve failures and unexpected situations during mission execution are known in advance. The contingency tasks considered here are caused due to the nature of tasks that are being executed rather than being associated with the robots accomplishing these tasks. This means that if robots are unable to nish a task, then it is not due to shortcomings on the part of robots or robot failures. It happens only because the task cannot be nished, and other tasks (contingency tasks) must be nished rst to nish the original tasks. The crux of such centralized operations is to allow 107 human cooperation and intervention because centralized mission control architectures comprise of a centralized mission controller (CMC) where mission execution is monitored by human operators who can interrupt the mission execution and remotely control the robots whenever such a need arises. It is undesirable to only implement a low-level (trajectory planning level) reactive contingency management module because it renders the role of CMC useless in contributing to preparedness towards probable events of mission failures (contingencies). When human operators are in the loop, the system needs to be designed such that there are strong prior beliefs regarding which mission tasks will most likely be interrupted, and these can be updated during mission execution. Humans have a better chance of contributing to contingency preparedness with the aid of such systems. As seen in the multi-USV ship escorting example, one can strongly predict which con- tingencies may occur and how to prepare the robot teams for them. As more and more activities in ports are automated, it would become easier to predict situations encountered during mission execution. Therefore, it becomes more ecient to address the issue of contingency tasks at the mission planning level itself. This may be augmented with the implementation of the low-level reactive contingency management module for better success guarantees. To incorporate a contingency task in a mission plan, one only requires to add new tasks to the mission plan. The central ground station monitoring situational awareness for the mission detects these potential contingency tasks where the word potential signies the associated uncertainty and a non-zero probability of the mission getting completed without interruption by these contingency tasks. Such potential contingency tasks may be handled using these two simple approaches. A naive approach to incorporating the contingency task in the mission plan is to update the mission plan by inserting all of these new tasks. However, there is an uncertainty associated with these contingency tasks impacting the mission. So there is a non-zero probability associated with some of these potential contingency tasks to not impact the mission execution at all. This must be taken into account while making decisions regarding the contingency tasks. As outlined above, 108 a conservative approach would incorporate all the reported potential contingency tasks without taking into any consideration the probabilities of the contingency tasks to impact the mission adversely. A reactive approach is based on the real-time detection of a contingency task depending on the robot's current situational awareness. For example, when a task is ready for execution by the assigned coalition, the mission planner checks whether its associated contingency task needs execution. Thus, this approach only resorts to the execution of those contingency tasks which are triggered just before the execution of the associated mission task. Both of these approaches are seen to produce suboptimal results with respect to the minimization of the expected time of mission completion. This is because both approaches have obvious shortcomings. The conservative approach may execute mission tasks that have no impact on the mission work ow whereas the reactive approach may lead to robots of a coalition waiting in the idle mode while a single robot is nishing the contingency task. It would have been better to handle the contingency task earlier. Based on the above discussion, a proactive approach is presented in this chapter to incorporate the potential contingency tasks. It tries to address the contingency tasks in the initial stage of mission execution based on their probabilities. There is a strong assumption on the nature of contingency tasks. The characteristics of the contingency tasks should not vary with time or be dependent on the percentage of mission completion. It is only then possible to address these contingency tasks independent of the progress in mission work ow. In this work, several heuristics are applied to allocate tasks to the robots for the contingency- aware multi-robot task allocation (MRTA) problem. These heuristics are improvements based upon the simple greedy method. A multi-heuristic scheduling framework has been developed which can choose the best heuristic for the mission scenario based on the task characteristics. The main goal is to account for contingency tasks eectively. Based on the above reasons, it is clear that one must adopt a proactive approach to manage contingency tasks. It is also evident that the proactive approach requires information coordination among the robots regarding the required tasks. This cannot be achieved using distributed and decentralized techniques. The 109 proactive approach is deliberative and can be implemented only through a centralized architecture. But the proactive approach is bound to be computationally expensive as will become evident in Section 6.5. For ve contingency tasks, an exhaustive method must run the mission planning module 4 5 1000 times. For a nominal scenario of 50 tasks and six robots, the running time is 0:1 seconds in Matlab. Thus for solving the contingency scenario consisting of ve contingency tasks, total time spent during computation is approximately 100 seconds. Assuming a ten-fold speed-up when deploying the technology in a low-level language, one gets a total running time of 10 seconds. Running times of such orders are completely agreeable for a centralized mission planning approach. But a heuristic is also developed to prune the decision search space of the contingency tasks, thus signicantly bringing down planning run-time further. This allows the handling of a larger number of contingency tasks. This chapter is organized as follows. Section 6.2 presents a motivating example of a multi-USV mission scenario to describe the nature of the problem solved in this chapter and its applicability. This is derived from the case study of Chapter 4. Section 6.3 provides the problem formulation. Section 6.4 describes the heuristics-based approach to solve the proposed MRTA problem while accounting for uncertainty. Section 6.5 outlines the approach of incorporating the contingency tasks in the nominal mission plan while minimizing the expected mission ending time. Results based on both the task allocation as well as contingency handling methodologies are then presented in Section 6.6. 6.2 Background Following is a motivating example that helps in setting up the problem scenario for this chapter. Suppose a team of USVs in a port region (u 1 ;u 2 ; ;u n ) has been assigned to the task of assisting an arriving ship to dock safely to its designated berth location. There may be multiple such sea vessels like cruise or container ships proceeding towards their berth. Busy ports are also 110 accompanied by the high trac of civilian/recreational vessels. The free region where the vessels are allowed to travel is known. Since the present chapter considers planning only at the mission level, one need not concern with any port-related restrictions on vessels speeds like minimum- wake zone. These are taken care of at the trajectory planning level. It is assumed that the entire environment is accessible to the central mission planner. The job of this centralized mission planner is to generate tasks that need to be executed and assign it to the USVs. 6.2.1 Environment Modeling The mission planner keeps track of the relevant environmental factors by maintaining a list of environmental variables which it keeps updating based on sensory information feedback. Even today, ground stations guiding sensitive military missions or NASA control center controlling space missions perform similarly by gathering sensory data from numerous sensors tted across the mission domain. These environmental variables are either continuous or Boolean. USV characteristics like the current vessel position and task characteristics like an assigned location where a task will be executed are modeled using continuous variables. These may be discretized due to constraints from computational resources. On the other hand, the Boolean variables are used to keep track of qualitative factors that are imperative for mission completion. They can only hold two values: true (>) or false (?). A complete enumeration of such variables for the example mission is as follows: • ` 1 : has ship arrived in port • ` 2 : has docking location been assigned to ship • ` 3 : has ship path been computed • ` 4 : have survey areas been computed • ` 5 : have USVs been assigned to survey areas 111 • ` 6 : have survey areas been partitioned among USVs • ` 7 : have coverage way-points been computed • ` 8 : have USVs surveyed regions of interest • ` 9 : have USVs approached the ship • ` 10 : have USVs escorted ship to the docking position • ` 11 : is ship at the docking position • ` 12 : has collision risk been detected 6.2.2 Resource Modeling Tasks require resources for their execution. Motion-based tasks are executed using USVs. Com- putational tasks are executed using the computing power available at the central mission planner location as well as the individual computers mounted on each USV. In general, it is assumed that the resources are homogeneous with respect to their tasks. Resources place additional constraints on task execution. In addition to a task's precondition for execution to be satised, it requires the assigned resources to be available at the beginning of task execution. Otherwise, the task remains halted until the required resources are freed up, leading to idle time during mission execution. Therefore it is important to utilize resources in such a way that the mission makespan is minimized. However, the tasks do not tend to grab as much free resource as is available at the beginning of their execution because this may result in ineciency. For example, even though three USVs may be available for surveying a region of interest, it may turn out to be faster to use only two USVs which are much closer to the region and hence can complete the area coverage in less time. 112 6.2.3 Task Modeling Tasks considered here are atomic or elemental. Once assigned to the agent(s), they can be nished without requiring any further decomposition. As already mentioned, tasks can be of two types. Motion-based are related to trajectory tracking or motion control of USVs. Computational tasks can only be done using the computing resources and result in the computation of data necessary for mission completion or even initiation of mission execution. These tasks may also be classied based on whether they are planned for before mission execu- tion (oine) or during mission execution (real-time). For example, if USVs suddenly come across a civilian vessel obstructing their paths or the ship's path, then the USV closest to the concerned civilian vessel must come near it and issue a warning to clear the path. This is an example of a contingency task which cannot be planned beforehand. The central mission planner plans only for the oine tasks initially and then keeps updating this plan by introducing contingency tasks in the task network as and when required. 6.2.3.1 Mission Tasks To be able to compute the task precedence network from the mission requirements, one must model the mission tasks 2T as 5tuple = (I;P;R;E;T) where: i. I denotes the information required as input for the robots to be able to execute the task. In the case of motion-based tasks, for example, one requires the USV positions. ii. P represents the preconditions that must be satised before the task can be executed using robots or computing agents. iii. R refers to the resource demand by the tasks to be nished. iv. E records the eects of task execution relevant to mission progress. Motion-based tasks may change the continuous variables representing environment states (external eects) as well 113 Table 6.1: Computational tasks Tasks Inputs Preconditions Eects monitor ship arrival in port ( 1 ) • Ship real-time position • Port region where ship must arrive • ` 1 =? • ` 1 updated assign docking position ( 2 ) • Ship docking position • Ship position • ` 1 => • ` 2 =? • ship assigned docking position • ` 2 => compute ship path ( 3 ) • Ship docking position • Ship position • ` 1 => • ` 2 => • ` 3 =? • ship path computed • ` 3 => compute areas to explore ( 4 ) • collision-free path • ` 1 => • ` 3 => • ` 4 =? • areas of interest computed • ` 4 => assign USVs to areas ( 5 ) • areas to be explored • available USVs • ` 1 => • ` 4 => • ` 5 =? • Assignment of USVs to areas • ` 5 => partition areas among USVs ( 6 ) • areas to be explored • available USVs • ` 1 => • ` 4 => • ` 5 => • ` 6 =? • Partitioning of areas of interest among USVs • ` 6 => compute coverage way-points for survey areas ( 7 ) • areas to be explored • available USVs • ` 1 => • ` 4 => • ` 5 => • ` 6 => • ` 7 =? • coverage way-points computed • ` 7 => as update boolean variables (internal eects). On the other hand, computational tasks may update boolean states or issue instructions to the available agents. v. T denotes the logical conditions which must be fullled in order to declare the successful termination of the task. This allows the handling of the temporal nature of tasks. When tasks being executed get nished, all the appropriate environment states must be updated, and this is achieved through the verication of these logical termination conditions. It allows the mission planner to keep track of the goal and perform feedback-based planning. The above task elements have been dened in Tables 6.1 and 6.2 for all the computational and motion-based tasks which are considered in the illustrative mission scenario. The column for the termination conditions for all computational tasks has been omitted because these refer to the completion of the corresponding computations. So it is trivial to specify the same every time. 114 The resources required by the tasks have also not been specied because it is assumed that the entire pool of agents that is free and relevant to the concerned task may be utilized. Once the task network has been computed, coalition formation and scheduling of tasks remain. To solve this MRTA problem, it suces to model the mission tasks 2 T more succinctly as 4tuple = (r in ; r out ; 2 [ min ; max ]; t e min ). The operator r() refer to the spatial position operator throughout this chapter. It is assumed that once the robots are assigned to task , they must arrive at the location r in () and when they have nished the task they end up at location r out (). In reality, it may be the case that dierent robots start their task execution from dierent positions depending on various other factors, and may then consequently end up being in dierent locations at the end of the task. However, such complexities may also be handled more eciently at the lower level planning once the mission has been planned and conrmed for execution. The number of robots assigned to the task must lie in the range [ min ; max ]. Such resource constraints are included to model tasks which cannot be started until a minimum number of robots have arrived at the task location r in (). There must also be an upper bound on the coalition size assigned to the task because multi- robot teams require higher overhead charges like communication robustness, accurate localization, and sucient battery levels. When robots work on task , they take time t e whose calculation is discussed in Section 6.4.1. The time t e min taken by min robots to nish the task is known beforehand and used to compute t e . To account for uncertainty, it is modeled as a Gaussian distributed variable t e min N(; 2 ). The mission proceeds by identifying feasible tasks, executing these identied tasks, and then recomputing the next set of feasible tasks. This keeps repeating until the mission goals have been successfully achieved. Figure 6.2 shows a task precedence graph of six mission tasks. This directed acyclic graph may be interpreted as follows. 115 Table 6.2: Motion tasks Tasks Inputs Preconditions Eects Termination survey region of interest ( 8 ) • coverage trajectories • USV positions • ` 1 => • ` 7 => • ` 8 =? • region surveyed by USV • ` 8 => • when survey completed approach ship ( 9 ) • Ship position • USV positions • ` 1 => • ` 2 => • ` 8 => • ` 9 =? • ship approached by USV • ` 9 => • when approach completed escort ship ( 10 ) • Polygon representing docking region • Ship position • USV positions • ` 1 => • ` 2 => • ` 8 => • ` 9 => • ` 10 =? • ship escorted by USV • ` 10 => • when ship enters docking region station-keeping ( 11 ) • Ship docking position • Ship position • USV positions • ` 2 => • ` 10 => • ` 11 =? • ship docking monitored by USV • ` 11 => • when ship docked issue warning ( 12 ) • Ship docking position • Ship position • USV positions • ` 2 => • ` 10 => • ` 11 =? • ` 12 => • danger to ship cleared • ` 11 => • when warning issued and acknowledged Initially only tasks 1 and 2 are feasible. After they are completed, three more tasks become feasible, viz. 3 , 4 and 5 . Once task 1 is nished, task 4 may be executed. Similarly, once task 2 is nished, task 5 may be executed. Only task 3 requires that both task 1 and 2 are completed. This implies that a sophisticated scheduling algorithm must take such constraints into account when trying to minimize the total duration for which the robots are idle. Finally, task 6 Figure 6.2: Example of task precedence graph 116 may only be begun when all three tasks 3 , 4 and 5 are completed. Thus, mission proceeds in stages of multiple tasks. 6.2.3.2 Contingency Tasks It is assumed that all contingency tasks with non-zero probability of impacting the mission are known as a set T c . One can model each contingency task c 2 T c as 5tuple c = (r; 2 [ min ; max ];t e c;min ;P c ; in ;). The rst three elements have similar meaning as for mission tasks but some specics will be elaborated below. It is assumed that before mission planning, one checks based on the environmental conditions which contingency tasks may potentially impact the mission execution adversely. All contingency tasks with zero probability are ruled out. For the remaining potential contingency tasks, one could maintain a prior probability distribution P c ( c ) as a function of time and project this probability distribution into time as far as may be needed for mission completion. During mission execution, some new information may become available that updates this into a new posterior distribution and then replanning is done again just as at the beginning of mission execution but with newly updated information. Contingency tasks are additional tasks inserted in the nominal task network from where it also derives its precedence constraints. The impact of a contingency task is analyzed through its eect on the task network and expected mission completion time t M . A contingency task transforms the task precedence graph P m by deleting specic edges and nodes and replacing them with new ones. This is shown in Figure 6.3 where the blue and green arrows are the directed edges from the preceding tasks of task in and the orange arrows represent the directed edges to the next-in-line tasks. Figure 6.3 shows that the contingency task c splits the interrupted mission task in into two fractions. In general, these fractions can be denoted by 1 and 2 . The 1 fraction of task in remains unaected by task c because its eects appear to hamper only the remaining task 117 Figure 6.3: General rule for graph renement to incorporate a contingency task execution. When a robot team nishes the rst 1 fraction of task in , one would naturally assume that along with the workload of contingency task c , only 1 1 task-fraction of task in remains to be done. However, in the generalized modeling of contingency tasks, one should include the cases where task completion requirements of the original task in have been changed so that it is now required to nish task-fraction 2 of task in which may be dierent than 1 1 . For simplicity, it is assumed that 1 = 1 2 = throughout this chapter. It is this case that has been depicted in the gure also. So it is assumed that the original workload of task in remains unchanged and task in has only been interrupted by a task c which is then simply an extra task to be inserted in the mission. The general case can be easily handled by extending the framework described here. So in this kind of graph renement, the uninterrupted task-fraction 1 is nished simulta- neously with the contingency task. After both work-loads have been nished, the remaining task-fraction of task in is nished. Figure 6.4 shows another type of graph renement. Here, the uninterrupted task-fraction 1 must be nished rst and only then the contingency task may proceed. This case is useful in modeling scenarios where the contingency task was not incor- porated at the beginning of the mission but is encountered during mission execution and hence Figure 6.4: Rule for graph renement when the contingency task was not properly incorporated in the nominal mission plan and encountered during mission execution 118 requires execution. In such cases, it is natural that the uninterrupted task-fraction is completed as expected, and then the agents reorganize to address the contingency task. There is one important dierence that sets the contingency tasks apart from the mission tasks. Contingency tasks must be nished before the interrupted task can be completed. On the other hand, the mission tasks constrain other mission tasks from starting unless they have been completed. Figure 6.5: Overview of mission planning 6.2.4 Nominal Mission Planning The initial mission specication generally includes an initial world state and the desired world state. Information regarding tasks and environment is assumed to be known beforehand. At the mission planning level, one starts by computing the set of all the feasible tasks. The nominal mission planner then allocates the right resources, instructs the robots to execute the feasible tasks, and records the new world state. If the new world state is the goal state, then the mission 119 terminates successfully. Otherwise, the current set of feasible tasks is again computed, and this loop must keep on repeating until the goal state is achieved. This scheme is shown in Figure 6.5. The external event generator is the simulator module used to simulate the occurrence of contingency tasks in real life. If tasks are visualized as nodes, this scheme naturally results in a directed acyclic graph which encodes the task precedence constraints. Unlike search nodes in classical planning scenarios, the tasks referred to here have time durations and resource constraints associated with their execution. In the present example, the task network computed using this procedure is serial (a linear chain of the tasks described in Tables 6.1 and 6.2) because each new task becomes feasible only when its preceding task is nished. However, task networks may be more complex as will be encountered in subsequent studies. After computing the precedence network, the general mission work ow is known and the robots can be assigned to their tasks. These task assignments are communicated to the USVs who plan their trajectories with respect to the International Regulations for Preventing Collisions at Sea (COLREGs) [224]. 6.3 Problem Statement The exact problem that is solved with regards to the mission scenarios elucidated above is stated below, along with its inputs and outputs. The entire framework comprises of solving two problems sequentially. First, one must compute the coalition-task assignments and nominal schedules for mission execution (Section 6.4). The inputs and outputs for the same may be summarized as follows. Given, i. R = [r 1 ;r 2 ; ;r n ], a set of n robots available for the entire mission ii. T = [ 1 ; ; m ], a set of mission tasks 120 iii. P m = (T;E m ), a directed acyclic graph denoting the task precedence graph. The set E m = [( i ; j ); ] consists of ordered pairs of tasks encoding the inter-dependence of the mission tasks. iv. Coalition ineciency Compute nominal mission execution plan comprising of task schedules for each robot so as to, i. Minimize the expected mission completion time while accounting for: • Traveling uncertainty • Task execution uncertainty ii. Satisfy all task precedence constraints Coalition ineciency factor will be explained in detail in Section 6.4.1. Expression 6.1 denotes an example task schedule for robot r i which is explained as follows. Robot r i travels to task a and then executes it [this is denoted by Trv( a )! Exc( a )]. The robot then waits for task b to become feasible and this action is denoted by [Wait(P( b ))]. When task b becomes feasible for execution, robot r i travels to the task's location denoted by [Trv( b )], waits for min robots to arrive because they are required by the task for execution. This waiting action is denoted by [Wait( min ( b ))]. Then the robot begins the task execution which is denoted by [Exc( b )]. This is how all the robots' schedules are expected to behave like. r i : Trv( a )! Exc( a )! Wait(P( b ))! Trv( b )! Wait( min ( b ))! Exc( b ) (6.1) After computing a nominal mission execution plan, one then incorporates the contingency information (Section 6.5). Given, i. Nominal mission completion planM n 121 ii. T c = [ c1 ; ; cmc ], a set of contingency tasks iii. Coalition ineciency iv. Fraction of contingency task completed for penalty type 2, (Algorithm 5) v. Uncertainty associated with robot when traveling: a (Section 6.4.4) vi. Uncertainty associated with task execution j : ej (Section 6.4.4) Compute, i. Task schedule for each robot that does the following: (a) Minimize the expected mission completion time accounting for: • Traveling uncertainty • Task execution uncertainty • Known probabilities for contingency tasks impacting the mission (b) Satisfy all task precedence constraints ii. Replanning trigger conditions for those contingency tasks which have been deferred for now Expression 6.2 provides an example task schedule for robot r i when contingency tasks are incorporated. According to the given schedule, robot r i rst travels to the contingency task location denoted by [Trv( ctg a )]. It then nishes the contingency task denoted by [Exc( ctg a )], and travels to the mission task location denoted by [Trv( a )]. Upon arrival, it waits for min robots to arrive at the task location denoted by [Wait( min ( a ))], and then begins task execution denoted by [Exc( a )]. r i : Trv( ctg a )! Exc( ctg a )! Trv( a )! Wait( min ( a ))! Exc( a ) (6.2) 122 6.4 Task Schedule Optimization In this section, strategies are described to form robot coalitions and assign them to the mission tasks. It is assumed that a robot performing its tasks is not aected by the environmental changes caused by other robots. To decide which coalitions to assign to which tasks, it is important to be able to score coalition-task pairs. For this purpose, a fractional task completion model [212] is used to compute the time required to nish the task by the assigned coalition. 6.4.1 Task Execution by Partial Teams When a higher number of robots are available, then the task takes lesser time to nish. It is assumed that an inverse relationship exists between the task execution time and the coalition size. However, in real-life, a bigger coalition size leads to additional overhead costs like communication channels, which pushes the duration of task execution upwards. To include this eect, a coalition ineciency factor is introduced whose value is assumed to be 30% when the coalition size is 10, and linearly interpolated for other coalition sizes. Thus, if robots have been assigned to task and it can be assumed that the robots are already at the task location so that traveling actions are not required, then the time taken for nishing the task is given by: t e () = t e min min (1 +) (6.3) Assume a coalition C R is assigned to task . If the coalition violates the task's resource constraints, then the pair is given a score of innity, essentially discarding it from future consid- erations. Otherwise, the arrival times for the robots of the coalition to reach the task location are computed, and the robots are sorted in the increasing order of their arrival times. This reordered robot list looks as follows: C s = [r (1) ;r (2) ; ] (6.4) 123 where :N\ [1;jCj ]!N\ [1;jRj] (6.5) Task execution does not begin until the robot labeled as r (min) arrives. Let the amount of unnished workload associated with task be , initialized to be 1, because the task has not yet been started. When the task is completed, then () will be assigned zero value. Otherwise, it varies between 0 and 1. The task execution time is computed based on the mean values (). When robot r (min) arrives then the team of assembled robots begins task execution. They may complete the task on their own, if no additional robots were to arrive, in time t e given by t e =(t e min )(1 +( min )) (6.6) Suppose the robot r (min+1) arrives in time t a . If t e > t a , then the task has only been nished partially by the time the ( min + 1) th robot arrives. To update the amount of workload remaining at this instant, one can assume a direct relationship between the amount of task nished and execution duration (Equation 6.7). t a (t e min )(1 +( min )) (6.7) The arriving robot r (min+1) teams up with the previously arrived robots to nish the re- maining task and this reasoning is repeated until the task has been nished. However, if t e < t a then the task is nished before the next robot arrives. However, the other robots assigned to the task are now rendered useless. To penalize such assignments, the score given is the time taken by the last robot to arrive at the task location. 124 Thus, every other robot remains idle for dierent duration of time, and this contributes to increasing the chances of discarding this coalition-task assignment. According to the above model, there are only two types of idle behaviors possible for the robots. The rst kind of scenario occurs when the rst ( min 1) robots wait for robotr (min) to arrive. This happens in all cases whether the task nished before or after all the robots arrived. The second kind of scenario occurs if the task nished before some robots have arrived at the task's location. In such cases, all robots other than the last arriving robot have to wait additionally till the time t a (r (jCj) ). Resource constraints may signicantly impact the scheduling process because these place con- straints on the number of robots that must be available for the mission execution. Inequalities 6.8 and 6.9 are also required for ecient mission execution. Otherwise, either the resource constraints must be revised, or more robots must be added. If Inequality 6.8 is not satised, then mission aborts unsuccessfully. Next, the rationale for these inequalities is provided. Observe that the task networks decom- pose mission work ow into stages. Suppose for stageS, the set of feasible tasks is denoted byT f . Then every stage requires at least the sum of the minimum number of robots required for each of its feasible task. The maximum over this minimum is computed over all stages. The stage that requires the most number of robots to be feasible for execution must require less than the number of robots available,jRj, hence Inequality 6.8. Similarly, every stage will require the maximum number of robots. Any more robots than that are just not needed because otherwise, they will remain idle during the execution of tasks in that stage. The second inequality is not strictly required, but its role becomes clearer during the discussion of benchmarking results presented in Section 6.6.1. One does not want to unnecessarily have a very large number of robots engaged in the mission only because it is feasible to do so, and the second inequality helps to enforce that. 125 max S X 2T f min ()jRj (6.8) min S X 2T f max ()jRj (6.9) For every stage, if P 2T f max () <jRj then mission can still proceed but some robots will have to be idle. One simple way to address situations of such nature more eciently may be the following. Suppose the mean position averaged over the locations of all tasks to which the robots have been currently assigned for a particular stage is r(T f ). The robots which are further away from the position r(T f ) are discarded and remain unassigned to any task. While the assigned robots are nishing their tasks, the unassigned robots advance to position r(T f ). This ensures that robots are not dispersed signicantly over time and remain relatively close to each other. However, the task resource constraints can also be revised to avoid such a situation. If robots are not being utilized, then the number of robots assigned to the mission can be decreased. Such a situation also indicates that one may increase max values for tasks chosen based on user-dened criteria. 6.4.2 Handling Divisible Tasks This section models an additional feature associated with task execution that allows tasks to be interrupted during their execution. In some real-world mission, such features are either specied by the user or strictly prohibited based on the nature of robots, tasks, and environment. It is useful to study the eect of task divisions. This is not exactly the same as task preemption studied in processor scheduling literature. The mission tasks considered in this chapter may be thought of, to some extent, as surveying tasks using unmanned robots. Such tasks can be interrupted during their execution, meaning the 126 robots are reassigned to other tasks. This feature is referred to as the divisibility of tasks because workload of the tasks are divided on the Gantt chart (Section 6.6.1) when they are redistributed among the robots. Algorithm 1 Compute Task Completion fraction (CTCF) Input: ;C ;;T 1 Output: 1: Task remaining, () 2: Robots at r(), ; 3: Time elapsed, t 0 4: R sorted C in the order in which they will arrive at r() 5: t a ( min ) time taken by min robots to arrive at r() 6: remove rst min robots fromR 7: if T 1 <t a ( min ) then 8: update robot positions 9: 1 10: else 11: while t<T 1 do 12: ifjj ==jC j then 13: r(r i ) r()8 r i 2C 14: update 15: t T 1 16: else ifjj<jC j then 17: T 2 FTCM(; ;) 18: T 3 t a (R:front ( )) 19: if T 3 ==T 1 t then 20: t T 3 21: jj jj + 1 22: [ (R:pop ( )) 23: t t +t 24: else 25: t T 1 t 26: t T 1 27: end if 28: if min(T 1 ;T 2 ;T 3 ) ==T 2 then 29: 0 30: else 31: update 32: end if 33: update robot positions based on work performed during duration t 34: end if 35: end while 36: end if Unlike processor tasks, the tasks are not being suspended to be resumed at a later point in time. However, robot states are updated and possible reassignments are considered to see if faster 127 mission completion can be achieved. Task execution may still get suspended if all robots assigned to a task are reassigned to another task and new robots are assigned to this task because the new robots will be traveling to the task and therefore task execution has been halted. Unless drastic scenarios are considered, such cases do not arise and are not observed in the experiments conducted for this chapter. Mostly, one or two robots from the current task are reassigned to another task based on optimality considerations. Also, the tasks undergoing execution by the robots are allowed to be interrupted only when at least one task has nished. The motivation behind allowing this feature is to reassign the robots that recently nished their task. All the robots are rescheduled, and not just those who got freed up after nishing their task. This helps to explore the search space for high-quality solutions as fully as possible. This procedure can also be interpreted as if robot schedules are getting interrupted and reas- signed as soon as any task is nished. For nding optimal reassignments, one must account for robot positions before rescheduling carefully. Those robots that got freed up after nishing their task must still be at the location of their current task. But the other robots may be working at their respective task locations or in transit to their task locations. It is required to compute the amount of workload remaining for each feasible task. Then the robots can be optimally rescheduled based on the updated workload remaining for each task. In such cases, the robots have worked on their assigned task until the time it took for the most recent task to nish, and this time is known. This requires solving of the inverse problem of the fractional task completion model: given a xed durationT 1 for the execution of task, how much workload has been nished by its assigned coalition C ? If the allowed duration T 1 < t a (r (min) ), meaning the duration lapses even before the rst min robots arrive at the task location, then the full workload is remaining. Otherwise, the approach is similar to Section 6.4.1, where robots that are arriving at the task location are tracked one by one, and they contribute to the completion of the task's workload. Once a new robot has 128 arrived, it combines with the previous robots. So, the workload nished by this extended team before the arrival of another robot must be computed. Suppose that this extended team requires time T 2 to nish the task whereas the next robot will arrive in time T 3 . If T 1 <T 2 <T 3 or T 1 <T 3 <T 2 then the task is partially nished by the coalition up to the allowed duration T 1 . The robot positions are also updated because some of them may be at intermediate positions while traveling to the task location. If T 2 <T 3 <T 1 then all robots have arrived and nished the task. If T 2 < T 1 < T 3 then the intermediate position of the robots is updated and the task is nished. If T 3 <T 1 <T 2 or T 3 <T 2 <T 1 , then the arrival of the next robot is recorded, the intermediate positions of the robots that have not yet arrived are updated, and the remaining workload is computed. Then, the same procedure is repeated until the allotted duration T 1 is exhausted. A succinct implementation of this methodology is shown in Algorithm 1. 6.4.3 Task Scheduling Strategies Algorithms in processor scheduling literature provide insights in designing techniques for multi- robot scheduling. These algorithms operate through two successive phases: task prioritization and robot selection. These algorithms can be modied for MRTA problems based on the fol- lowing analogy. The processing of tasks on processors is analogous to the execution of tasks by robot coalitions, and inter-processor communication is analogous to inter-task traveling by robots. StrategiesS 1 S 5 are inspired from the processor scheduling literature [122]. However, coalitions rather than individual robots are assigned to the tasks in the second phase. This is because this chapter assumes multi-robot (MR) tasks as noted in the beginning. 6.4.3.1 Heuristics Employed List scheduling algorithms use heuristics that provide an important method for developing fast algorithms for such intractable problems. Greedy heuristics are very common which order the 129 tasks in the increasing (or decreasing) order of their execution durations. However, there are more advanced heuristics designed in the processor scheduling literature to outperform the greedy meth- ods. We shall adopt these methods in the multi-robot context and compare their performance. StrategiesS 3 andS 4 use static levels to prioritize the tasks. Static levels for the mission tasks can be computed by summing up the execution durations of all the tasks along the longest path to the terminating tasks in the task precedence graph. Tasks with higher static levels are sequenced before tasks with lower static levels. If the execution duration of all the tasks is assumed to be one and then compute the static levels, it constitutes strategy S 3 , and this heuristic is based on the Levelized Min-Time (LMT) scheduling algorithm. On the other hand, if the actual execution duration provided for the mission tasks beforehand is used, then that strategy isS 4 , and this heuristic is based on the Highest Levels First with Estimated Times (HLFET) scheduling algorithm. Strategy S 5 computes upward (or downward) ranks of the tasks in such a way that the time spent in traveling to the task's location is also included in the heuristic used for prioritizing the tasks. This is based on the Heterogeneous Earliest Finish Time scheduling (HEFT) scheduling algorithm. If one computes upward ranks, then the tasks are ordered according to the decreasing order of their upward ranks and assign the coalitions which nish the tasks earliest in that order. All the mission tasks that form the leaf nodes in the task network (having zero out-degree) are assigned upward ranks asrank u ( E ) =e( E ). Here, the execution time of task E is obtained using the function e(). Let e() denote the average execution time of a task over all feasible coalitions. For other tasks in general, the ranks may be computed using the equation below rank u () =e() + max 0 2succ() (rank u ( 0 ) +a(; 0 )) (6.10) Where succ() refers to the direct successors of task in the precedence constraint graph, and a(; 0 ) refers to the travel time from task to task 0 . 130 Algorithm 2 StrategyS 0 without task division Input: R;T;P m ; Output: M n 1: M n ; 2: whileT not empty do 3: compute feasible task-setT f T from graph P m 4: whileT f not empty do 5: D all distributions of robots among tasks 2T f 6: compute setD r D of distributions violating task-resource constraints 7: D D r nD 8: t 1 9: for all d = (c;)2D do 10: t 1 11: if min ()jcj max () then 12: ifjRncj P 0 2T f n min ( 0 ) then 13: t FTCM(;c) 14: end if 15: end if 16: if t<t then 17: d d 18: c c 19: t t 20: 21: end if 22: end for 23: M n M n [ d 24: R Rnc 25: T f T f n 26: T Tn 27: update robot positions 28: end while 29: end while 6.4.3.2 Task Allocation Without Task Division A naive strategyS 0 would be to perform exhaustive enumeration over all possible coalition-task pairs and use this brute-force technique (Algorithm 2) to compute the optimal schedule. After computing the set of tasks whose preconditions are satised (Line 3), all the possible feasible distributions (Lines 5 7) of robots among the tasks (jT f jjRj) are computed. In Lines 8 23, the optimal coalition-task pair is found and then both the task and the assigned coalition are removed from further consideration (Line 24 26). The positions of the robots forming the 131 Table 6.3: Overview of strategies used for task allocation Strategy Task prioritization heuristic S 1 Pick tasks with lower execution duration rst S 2 Pick tasks with higher execution duration rst S 3 Start picking tasks with lower static levels computed by assuming unit execution duration S 4 Start picking tasks with lower static levels computed for actual execution duration S 5 Pick tasks with lower upward ranks rst assigned coalition are updated so that when the next batch of feasible tasks is generated, the optimal coalitions are selected appropriately. Using n robots, 2 n coalitions could be formed. Not all of them will be considered for task assignment because the resource constraints are used to discard the violating ones. Given a set of feasible tasks T f , all the distributions of n robots among the feasible tasks are computed. Since such computations are of the order O(jT f j n ), computers run out of memory for a high number of robots or tasks. For example, 16 GB of memory is insucient, even for nine tasks and nine robots. One could improve this further by only considering such distributions of the robots among tasks where at least one robot is assigned to a task because min > 182T. This leads ton !S(n;jT f j) distributions whereS(;) denotes the Stirling number of the second kind. This still hogs the entire 16 GB of RAM for nine tasks and eleven robots. Given a set of feasible tasksT f , those distributions of n robots among the feasible tasks are considered which do not violate resource constraints. If the problem specications are within RAM's capacity, then strategy S 0 computes the optimal mission assignment in the deterministic version of the problem that serves as the baseline. However, the memory requirements placed by strategy S 0 are very restrictive. To eciently solve problems of larger sizes, heuristics are used to search the space of high-quality solutions quickly. The heuristics used in Sections 6.4.3.2 and 6.4.3.3 are described using Table 6.3. Based on heuristics described in Section 6.4.3.1, strategies S 1 S 5 are designed, which can handle problems of larger sizes much more eciently. 132 Two additional strategies, strategiesS 6 andS 7 , are also introduced for benchmarking purpose. They do not comprise a task prioritization phase. They directly start selecting coalitions while handling the interference of coalitions among themselves. Two coalitions interfere if at least one robot overlaps between them. StrategiesS 6 andS 7 are based on the MinStepSum and MinInterfere heuristics described in [1]. The implementation details of strategies S 1 S 5 are provided in the context of task divisions in the next section. 6.4.3.3 Task Allocation with Task Division It can be seen that when task divisions are allowed, lower mission completion times may be achieved because this only results in reducing the duration for which the robots remain idle. This is explained in detail in the context of strategies S 1 S 5 . Strategy S 1 (described as Algorithm 3) greedily orders the feasible tasks generated in the ascending order of their execution duration every time they are generated (Lines 3 4). Then, the task with the highest priority is selected for assigning its optimal coalition. The set of all coalitions that can be formed using the available robots is computed and the coalition sizes are restricted to the minimum between the maximum robots allowed for the task under consideration and the number of robots available for assignment (Lines 7 8). For the coalition selection phase, another simple heuristic is used. The coalition that takes the minimum time to nish the chosen task is then computed (Lines 17 20). The time taken by the coalition to nish the task using the fractional task completion model is computed (Line 14). The selected coalition should satisfy the resource constraints (Line 12). The selected coalition should be such that the number of robots available after the robots in the selected coalition are used up is at least as much as the sum of minimum numbers of robots required for the remaining tasks (Line 13). Once the coalition assignments M n have been computed (Line 22), the task which will get nished the earliest is determined (Line 25). Task executions are paused once any task gets 133 Algorithm 3 StrategyS 1 (andS 2 ) with task division Input: R;T;P m ; Output: M n 1: M n ; 2: whileT not empty do 3: compute feasible task-setT f T from graph P m 4: sortT f in ascending (descending forS 2 ) order by task duration 5: whileT f not empty do 6: T f :pop() 7: N f min( max ();jRj) 8: C set of all coalitions in set R up to N f robots 9: t 1 10: for all c2C do 11: t 1 12: if min ()jcj max () then 13: ifjRncj P 0 2T f n min ( 0 ) then 14: t FTCM(;c;) 15: end if 16: end if 17: if t<t then 18: c () c 19: t () t 20: end if 21: end for 22: M n M n [ (c ;) 23: R Rnc 24: end while 25: arg min t () 26: T Tn 27: for all 2T f n do 28: () CTCF(;c ();t ( )) 29: end for 30: update robot positions 31: end while nished and all the robots are redistributed among the new set of feasible tasks T new f . This is done to prevent robots from being idle while waiting for the last task in the original set of the feasible tasksT old f to be nished. Instead, the coalition assignmentM n is only adopted until the completion of the earliest task. Then task is marked as nished (Line 26 27) and compute the workload remaining for other tasks belonging to the setT old f n (Lines 28 30). This entire procedure of computing the new set of feasible tasksT new f and nding its corresponding coalition assignment is then repeated. By 134 Algorithm 4 Task prioritization phase for strategiesS 3 ;S 4 ;S 5 with task division 1: M n ; 2: T p prioritizedT (Section 6.4.3.3) 3: whileT not empty do 4: compute feasible task-setT f T from graph P m 5: T f T p \T f 6: whileT f not empty do 7: REST IS SAME AS ALGORITHM 3 8: end while 9: end while striving to reduce the amount of time spent by robots being idle, task divisions lead to mission completing faster. Other than the task prioritization heuristic, strategies S 3 S 5 also dier slightly regarding how they prioritize the tasks as shown in Algorithm 4. These strategies prioritize the tasks (Line 2) even before the currently feasible tasks are computed, i.e., before Line 3. When the feasible tasks are computed, then they are ordered by taking an intersection with the prioritized task listT p while preserving the order in the task-listT p . Rest of the steps are the same as in Algorithm 3. The strategies described in this section are derived from the list scheduling algorithms by adding the features of coalition formation, fractional task completion, and task division. 6.4.4 Accounting for Uncertainty Robots suer from controller noise. Accurate models of tasks are generally not available before- hand. Environmental factors cause deviation from conditions assumed at the beginning of mission execution for its planning. Due to the interplay between these characteristics of robots, tasks, and environment, uncertainty associated with mission execution must be addressed. The time taken by robots to arrive at their designated locations to execute tasks becomes uncertain and is modeled as Gaussian distributed variables. For simplicity, it may be assumed that the variance of the associated normal distribution is the same for all robots regardless of the 135 tasks and environment. Execution durations of the mission tasks are also modeled as Gaussian distributed variables whose variances are again assumed to only depend on the tasks for simplicity. The task allocation methods discussed above explore dierent possible mission assignments and choose the best solution based on some optimizing criteria. Mission makespan is generally used as the metric to be minimized. Using the principles of risk-neutral decision-making, that mission assignment is chosen, which requires minimum expected mission completion time for its completion. When the mission tasks are divisible and stochastic in nature, mission completion time is computed as follows. Generally, a task precedence graph decomposes the mission into stages. In each stage multiple tasks are simultaneously being nished. One has to assign a coalition to each of the feasible tasks of that stage, say for the s th stage 1 ; ; ns . To select the optimal coalition, the distribution of the time taken by the coalition to complete the task is computed. For concreteness, let C k = [r 1 ;r 2 ; ;r n k ] be the k th coalition being considered for mission task j . Robotr i must travel distanced ij between robot position r(r i ) and spatial location of the task r( j ) at mean speed v. The time taken to arrive at the task's spatial location is modeled as a Gaussian variable T a ij N(d ij =v; 2 a ). Thereafter it must perform the task for duration T e ij N(t e ij ; 2 ej ) which can be calculated using the fractional task completion model. The total task completion time of robot r i is T k ij =T a ij +T e ij . The total task completion time for the coalition is given by s T k j = max(T 1j ; ;T n k j ) (6.11) We can approximate the maximum of two Gaussian distributed variables to be another Gaus- sian distributed variable. This does not hold true in the strict sense but works for approximate calculations. 136 Assuming the Pearson correlation coecient corr between the two Gaussian variables X N( X ; 2 X ) and Y N( Y ; 2 Y ) to be zero, their maximum is the distribution Z = max(X;Y ) whose mean and standard deviation can be computed using [225]. Z = X X Y + Y Y X + X Y (6.12) The value for mean can be computed as shown above. The value of the standard deviation can be computed as follows 2 Z = ( 2 X + 2 X ) X Y + ( 2 Y + 2 Y ) Y X + ( X + Y ) X Y 2 Z (6.13) where (x) = 1 2 1 + erf x p 2 (6.14) (x) = 1 p 2 exp x 2 2 (6.15) = q 2 X + 2 Y 2 corr X Y (6.16) The maximum of multiple Gaussian variables is computed based on the approximation [226] given below. s T j = max max(max(T 1j ;T 2j );T 3j )T n k j ) (6.17) The coalitionC k is assigned to task j where k = arg min k ((T k j )) (6.18) 137 Stage completion times are calculated by minimizing over the stage tasks because as soon as any stage task nishes, all the robots are rescheduled and, hence another stage begins. To calculate the mission completion time, one must sum over the stage completion times. s T = min(T 1 ; ;T ns ) (6.19) The minimum of two Gaussian variablesXN( X ; 2 X ) andY N( Y ; 2 Y ) is the distribution W = min(X;Y ) which is given as follows W = X Y X + Y X Y Y X (6.20) 2 W = ( 2 X + 2 X ) Y X + ( 2 Y + 2 Y ) X Y ( X + Y ) Y X 2 W (6.21) And the minimum of multiple Gaussian variables is approximated by using the iterative method that was used for calculating maximum over Gaussian variables. 6.5 Incorporating Potential Contingency Tasks 6.5.1 Background Three dierent approaches to incorporate potential contingency tasks in mission planning were previously discussed. The conservative approach adds the contingency tasks into the queue if they have been reported with any non-zero probabilities. On the other hand, a reactive approach is based on just-in-time behavior. According to this approach, if there is a non-zero probability for 138 Table 6.4: Dierent types of penalty associated with available actions for contingency tasks Action on potential contingency task c c does not aect mission c aects mission Defer action on c for now Case A (correct choice) Case B (penalty type 1) Immediately act on c Case D (penalty type 2) Case C (correct choice) contingency task c to impact mission task, then just when task becomes feasible the planner checks whether contingency task c presents a clear danger. If task c requires execution based on the query at that instant, then the uninterrupted fraction of the task is rst completed, next the contingency task is nished and nally, the remaining portion of the task is nished. However, the proactive approach explicitly takes into account the probabilistic nature of the contingency tasks to decide which contingency tasks should be incorporated in the mission plan and which should be ignored till beliefs about the occurrence of contingency tasks change. When- ever a new potential contingency task is reported, or the beliefs associated with already reported contingency tasks change, the entire decision-making process for all contingency tasks must be redone. If a contingency task does not aect the mission, then the reactive approach will always perform better than or at least as well as the proactive approach. However, if the contingency task impacts the mission, the reactive approach pays a higher penalty and performs worse than the proactive approach. Two actions are available for every contingency task c : either immediately incorporate it in the task network or defer its incorporation. The planner evaluates all options and assesses their impact on the expected mission completion time. Table 6.4 shows dierent outcomes associated with the two actions based on the real-world scenario. To decide whether to defer incorporation or not, the hypothesis testing approach is followed. If one decides to defer action on task c and it ends up impacting the mission, a penalty is incurred for keeping the robots associated with 139 Algorithm 5 Computation of contingency task networks through renement of nominal task network Input: T c ,T cr c , P m Output: P c 1: T 0 c T c nT cr c 2: W h f>g jT cr c j ;f>;?g jT 0 c j i 3: C d WW 4: P c ; 5: for all (a;w)2C d do 6: P 0 c P m 7: for all c 2T c do 8: N node in graph P 0 c for the mission task impacted by task c 9: if a defers action on c and c does not impact w then 10: leave graph P 0 c unchanged (case A) 11: else if a defers action on c and c impacts w then 12: add c serially between the two task fractions (case B) 13: else if a acts on c and c impacts w then 14: rene network as in Figure 6.3 (case C) 15: else if a acts on c and c does not impact w then 16: (t e c ) =(t e c ) ( = 70 100%) 17: rene network as in Figure 6.3 (case D) 18: end if 19: end for 20: P c add(P 0 c ) 21: end for the interrupted mission task idle because the contingency task becomes a bottleneck (Type 1 penalty). On the other hand, immediate action on c may incur a penalty if contingency action did not impact the mission tasks because some resources were unnecessarily diverted which could have been used to continue mission work ow (Type 2 penalty). If the number of contingency tasks that are reported is m c , then there are N c = 2 mc possible decision outcomes. There are alsoN c dierent world scenarios that could materialize when robots are completing the assigned mission depending on whether a particular contingency task impacts the mission. A naive exhaustive method proceeds as follows. For each decision, one could compute the ex- pected mission completion times for all theN c scenarios. This leads toN c N c score matrixS. To compute the expected mission completion time for a particular action-world pair (a;w), the frac- tional task completion model is applied on a task network derived from the original task network 140 excluding the contingency tasks. The set of all possible task networks P c = P c1 ; ;P c(Nc1) is computed when contingency tasks are incorporated. Each graph P ci = (T c [T;E c ) is also a directed acyclic graph. The task networksP m andP c comprise of allN c decision alternatives. To obtain setP c , Algorithm 5 based on the case-by-case analysis in Table 6.4 is used. It uses the two types of graph renement shown in Figures 6.3 and 6.4. It is assumed that if mission task is interrupted by contingency task c , then a single robot is commanded to resolve the contingency task. According to this operational model, min = max = 1 for all contingency tasks. Given the probability vector P = [p 1 ; ;p Nc ] of the dierent world scenarios and the matrix S, the vector T d = [T d1 ; ;T dNc ] storing expected mission completion times for each decision is computed where T di N(; 2 ). The optimal choice of decision i is risk-neutral. T di = Nc X j=1 p j S ij (6.22) i = arg min i (T di ) (6.23) 6.5.2 Search-space Pruning Heuristic In this section, a heuristic is proposed to reduce the computations in the exhaustive method. For every contingency task, 1 ( ci ) is computed by taking the dierence between expected mission completion time for deferring and executing it while all other contingency tasks are deferred. Similarly, 2 ( ci ) is the dierence between expected mission completion time when deferring and executing it while all other contingency tasks are executed. A contingency task that satises min( 1 ; 2 ) 0:1t e c ( ci ) > 0 is considered urgent and must be immediately incorporated in the task network. A contingency task that satises max( 1 ; 2 ) + 0:1t e c ( ci ) < 0 is considered deferred and must be revisited when its probability changes. For other regular contingency tasks, 141 a combinatorial search is performed to determine their optimal actions. The rationale for the heuristic is described next. 6.5.3 Analysis of Pruning Heuristic Using certain illustrative cases, the validity of the heuristic is demonstrated. Analysis for a general task network encompassing all possible scenarios cannot be performed, it is analytically shown for ve nominal cases that this heuristic holds. In all these cases, contingency tasks are assumed to take more time to nish than mission tasks, but the observations hold even if the assumption is reversed. Consider a scenario involving three high-probability contingency tasks such that only one contingency task impacts the mission per stage of mission execution. Then the expected mission completion time for the decision to defer all contingency tasks is represented as T M (000). Suppose the time to nish the mission according to the nominal task network is T . Then for each contingency task, the nominal execution is modied according to case B of Algorithm 5 implying that the network is serially extended by each contingency task soT M (000)T +t e c1 +t e c2 +t e c3 where t e ci is the time taken to nish task ci . To compute T M (100), observe that the rst contingency task will be treated according to case C, so the rst stage of task execution is extended by time t e c1 t max where t max represents the maximum time taken by the mission tasks of the rst stage to nish. HenceT M (100)T + (t e c1 t max ) +t e c2 +t e c3 and 1 ( c1 ) =T M (000)T M (100)t max . Even if one chooses an ordering for the contingency task execution times, the conclusion holds regardless of which ordering is chosen. Suppose t e c1 t e c2 t e c3 . To compute T M (011) the rst stage adds t e c2 t max to T because c2 is clearly the bottleneck task. Similarly, T M (111) T + (t e c1 t max ) and 2 ( c1 ) = T M (011)T M (111) t e c2 . The other pairwise dierences, viz. T M (010)T M (110) andT M (001)T M (101) aret e c2 andt e c3 respectively. These values are clearly contained in the range [ 1 ; 2 ]. The same is observed when analysis is performed for the other two tasks. 142 Suppose all contingency tasks have low probability. Then T M (000) T because this corre- sponds to application of caseA. To computeT M (100) one can invoke caseD for task c1 and also assume = 1 for simplicity so T M (100) T + (t e c1 t max ). Therefore 1 (t e c1 t max ) and similarly 2 (t e c1 t e c2 ). The other pairwise dierences are(t e c1 t e c2 ) and(t e c1 t e c3 ) which clearly lie in the range between the rst two values. The third case considers four high-probability contingency tasks where one stage has two mission tasks impacted by tasks c1 and c2 and other contingency tasks only impact one mission task per stage (t e c1 t e c2 t e c4 t e c3 ). The fourth case has two stages where two mission tasks are impacted by a total of four high-probability contingency tasks: c1 and c2 in one stage, and c3 and c4 in another (t e c1 t e c3 t e c4 t e c2 ). The fth case diers from fourth case only in assuming them to be low-probability (t e c4 t e c3 t e c1 t e c2 ). The results for these are summarized in Table 6.5. Case 3:1 represents the analysis for task c1 of third case. The rst two entries in the second column for each row are 1 and 2 followed by other pairwise dierences. Thus, all possible pairwise dierences between deferring and doing a contingency task seem to lie inside the range dened by 1 and 2 . Statistical experiments were performed to verify this pattern using randomly generated task networks and found this bound to be overshot at most by 10% of the considered contingency task duration. This slight overshot is well expected because approximations were used in the above analysis. 6.5.4 Contingency Handling Algorithm The above heuristic helps in choosing the right action for urgent and deferred contingency tasks which total to m out of m c contingency tasks. For the rest m c m regular contingency tasks, a sampling-based combinatorial search is performed. For each of the N c = 2 (mcm ) possible decisions, the expected mission completion times over K world scenarios out of the N c that are possible (for experiments in this chapter, choose K = 50 if m c > 5 else min(N c ; 2 4 )) is 143 Case Dierence between deferring and executing contingency task 1:1 t max ;t e c2 ;t e c2 ;t e c3 1:2 t max ;t e c2 ;t e c2 ;t e c3 1:3 t max ;t e c3 ;t e c3 ;t e c3 2:1 (t e c1 t max );(t e c1 t e c2 );(t e c1 t e c2 );(t e c1 t e c3 ) 2:2 (t e c2 t max ); 0; 0;(t e c2 t e c3 ) 2:3 (t e c2 t max ); 0; 0; 0 3:1 (t e c2 t max );t e c2 ;t e c2 ;(t e c2 t e c4 );t e c2 ; (t e c2 t e c4 );(t e c2 t e c4 );t e c2 3:2 (t e c2 t max );t e c2 ;(t e c2 t e c4 );t e c2 ;t e c2 ; t e c2 ;(t e c2 t e c3 );(t e c2 t e c4 ) 3:3 t max ;t e c3 ;t e c3 ;t e c3 ;t e c3 ;t e c3 ;t e c3 ;t e c3 3:4 t max ;t e c4 ;t e c3 ;t e c4 ;t e c4 ;t e c4 ;t e c4 ;t e c4 4:1 (t e c2 t max );t e c3 ;t e c2 ;(t e c2 t e c4 );t e c4 ; t e c3 t e c2 ;t e c4 t e c2 ;t e c3 4:2 (t e c2 t max );t e c2 ; 0;t e c2 ;t e c2 ;t e c2 ; 0; 0 4:3 (t e c2 t max );t e c3 ;t e c4 ;t e c3 t e c4 ; t e c3 t e c4 ;t e c3 ;(t e c4 t e c2 );t e c4 4:4 (t e c4 t max );t e c4 ;t e c4 ; 0; 0;t e c4 ; (t e c4 t e c2 );t e c4 5:1 (t e c1 t max ); 0;(t e c1 t e c2 ); 0; 0; 0; 0; 0 5:2 (t e c2 t max ); 0; 0; 0; 0; 0; 0; 0 5:3 (t e c3 t max ); 0; 0;(t e c3 t e c1 );(t e c3 t e c1 ); 0;(t e c3 t e c2 ); 0 5:4 (t e c4 t max );(t e c4 t e c3 );(t e c4 t e c3 ); (t e c4 t e c1 );(t e c4 t e c1 );(t e c4 t e c3 ); (t e c4 t e c2 );(t e c4 t e c3 ) Table 6.5: Analysis of various task networks computed. The set of task networks P c = [P c1 ;P c2 ; ;P cK ] representing each action-world scenario is computed using Algorithm 5 based on the case-by-case analysis in Table 6.4. The probabilities of these world scenarios are computed using the product rule. Then, the probabilistic sum of the expected mission completion time over K scenarios is computed. Thus, the vector T d = [T d1 ; ;T dN c ] storing expected mission completion times for each decision is obtained where T di N(; 2 ). That choice of contingency actions is selected which minimizes the mean time. For each contingency task, one must evaluate K task networks four times to compute 1 and 2 . Thus, the rst phase of proactive planning is of the order O(Km c ). If a total of m urgent and deferred contingency tasks were identied, then only 2 mcm decisions need to be analyzed. 144 Algorithm 6 Proactive planning for contingency tasks Input: T c ;P m Output: Optimal decision x 1: for all c 2T c do 2: if min( 1 ; 2 ) 0:1t e c ( c )> 0 then 3: x ( c ) => 4: else if max( 1 ; 2 ) + 0:1t e c ( c )< 0 then 5: x ( c ) =? 6: end if 7: end for 8: Compute P c using Algorithm 5 9: for all N c decisions do 10: for all K sampled world scenarios do 11: Compute T m using FTCM 12: S xj T m 13: end for 14: end for 15: T dx K P j=1 p j S xj 16: x arg min x (T dx ) The complexity of the second phase is clearly O(K2 mcm ) as described above. Thus the overall complexity of the proactive approach (Algorithm 6) is O(K(m c + 2 mcm )) = O(2 mcm ). For example, if a mission is reported to concurrently have ten contingency tasks and four contingency tasks can be pruned so that they do not need to be considered during combinatorial analysis, then this reduces the number of decisions to be considered from 1024 to 64, and thus enables the proposed approach to handle around ten contingency tasks. If an action on a particular contingency task is deferred, one must generate replanning trigger conditions and monitor the associated probability values being updated for the ignored contin- gency tasks in case they may become critical in the future. The replanning trigger condition computed for the deferred tasks is basically a range of probability values [p r l ;p r u ] [0; 1]. The lower bound is not of much importance and is set as the current probability of the contingency task. It is assumed that if the probability associated with the contingency task decreases, then the execution of the contingency task continues to get deferred. So one has to nd the increase in the probability that is critical for replanning. The upper bound however indicates that when 145 the actual probability of the contingency task exceeds this value replanning of the mission must be performed because low-risk contingency tasks may have become critical for successful mission completion. The range is determined using a heuristic method of sensitivity analysis. All the other information regarding the environment is kept xed. The current probability value, say p c of the contingency task, is increased in suitable step-sizes until a change in decision occurs at p . The trigger condition is then given by the range [p c ;p ]. The methods of incorporating contingency tasks are illustrated with two examples. Figures 6.12 and 6.13 show a simple and more complicated task network respectively followed by their corresponding execution schedules. Observe that this approach helps robots spend less time traveling and remaining idle (gray and black bars). (a) (b) Figure 6.6: Task networks based on neural networks used for simulation experiments in Table 6.8 6.6 Results and Illustrations 6.6.1 Multi-robot Task Allocation The strategies listed in Section 6.4 can be rst compared without introducing the feature of task divisions. In Table 6.6, the optimized mission completion time is shown for ten dierent mission scenarios whose task network topologies were manually generated, but all other features were randomly generated some of which are described in Table 6.7. The task networks chosen for these 146 Table 6.6: Benchmarking strategy S 2 against S 6 and S 7 . Table entries are mission makespan in minutes. S 0 S 2 S 6 S 7 1 298.28 308.63 325.04 322.77 2 589.80 610.47 648.09 642.30 3 888.92 917.81 968.34 957.79 4 1190.44 1226.04 1295.36 1281.81 5 94.75 116.11 119.41 124.54 6 303.65 372.93 364.50 379.13 7 489.86 616.13 612.72 639.86 8 676.26 863.86 850.31 878.72 9 1779.97 1831.29 1924.50 1916.47 10 2163.44 2222.54 2343.84 2336.89 experiments were random repetitions of graphs derived from articial neural networks (ANNs), as shown in Figure 6.6. Task execution durations are randomly sampled for each task. Resource constraints are tight meaning that the equality in Inequality 6.9 is satised or the corresponding inequality is satised by a tiny margin. As expected, the exhaustive method achieves the overall minimum. The second column shows the time achieved by strategy S 2 . This is because of all the list scheduling algorithms described, strategy S 2 performs signicantly better. All ve strategies were simulated for the ten mission scenarios. It was found that strategyS 2 performed the best and strategyS 4 was the second best. The mean percentage by which the mission completion time computed by strategy S 2 was less than that of strategyS 4 over the ten experiments was 3:28% with a standard deviation of 0:78%. StrategyS 2 performs better because it takes into account the task execution durations in a local way, i.e., after the feasible tasks have been identied for parallel execution. On the other hand, strategy S 4 prioritizes the tasks globally before task allocation begins. However, once tasks are allowed to be interrupted and reassigned to robots, strategy S 4 starts working better. A detailed explanation for the case of task division follows in the next section. The last two columns represent the results for strategiesS 6 andS 7 . Strategy S 2 outperforms strategies S 6 and S 7 in seven cases out of ten. This is because in these cases, resource constraints are tighter, and there are fewer robots. Having tight resource 147 constraints means equality in Equation 6.9 or the inequality holding by a tiny margin which di- minishes the advantage provided by using same robots for nishing multiple tasks (as in strategies S 6 andS 7 ). When this is not the case as in the other four cases, the list scheduling heuristics still outperform strategyS 7 but notS 6 . However, the runtime data shows that the list scheduling strategies are much faster because the mean ratio of runtime for strategyS 6 toS 2 over the ten experiments is 62:47 and the standard deviation is 62:5 and S 6 is faster than S 7 . Given the motivation to develop fast contingency handling techniques, one must devote as less time as possible during nominal mission planning. The last two strategies were only included for benchmarking purposes. All seven strategies are compared explicitly. In Figure 6.7a, the optimized mission completion time reported for ve dierent experiments is plotted. The exhaustive method takes the least time. The second element that is plotted is the minimum achieved by the list scheduling strategies. The last two elements represent the results for strategies S 6 and S 7 . The list scheduling strategies outperform strategiesS 6 andS 7 . In Figure 6.7b, ten dierent scenarios were simulated and their normalized mission completion time as well as running time are reported for all the seven strategies. A Pareto frontier is found to be taking shape because reducing running time produces more suboptimal results. Note that strategies S 1 S 5 take less time to complete than strategies S 6 and S 7 . In fact, some runs show that the last two strategies take even more time than the exhaustive strategy S 0 . A simple coalition selection heuristic was chosen for strategiesS 1 S 5 . The major weightlifting was done by the task prioritization heuristics. On the other hand, strategiesS 6 andS 7 used more sophisticated coalition selection heuristics to handle coalition interferences and discarded the task prioritization phase. The overall eect is that strategies S 1 S 5 perform faster and produce better results than the last two strategies. This is because tight resource constraints downgrade the possibility of using overlapping coalitions for nishing concurrently feasible tasks. The last two strategies were included only for benchmarking purposes and so can be safely removed from 148 Table 6.7: Description of scenarios used for experiments in Table 6.8 jTj jRj jE(P)j 1 175 8 246 2 350 8 496 3 525 8 746 4 700 8 996 5 120 8 155 6 360 8 475 7 600 8 795 8 840 8 1115 9 1080 10 1435 10 1320 10 1755 further consideration based on the above observations. The exhaustive strategy is also excluded from further consideration because it does not scale eciently and restricts the problem sizes. (a) (b) Figure 6.7: Performance of list scheduling heuristics compared to MinStepSum and MinInterfere from [1] 149 Table 6.8: Results for ten simulation runs with task division T c (in seconds) T m (in seconds) S 1 S 2 S 3 S 4 S 5 S 1 S 2 S 3 S 4 S 5 1 2.38 2.27 2.26 2.38 2.30 10510 9890 10265 9370 9950 2 13.52 13.51 13.51 13.52 13.53 21235 20795 21192 19376 20819 3 41.41 41.21 41.28 41.17 41.23 33138 31145 31181 28365 30823 4 93.08 93.60 93.08 93.06 93.44 43905 40952 41184 38949 40812 5 1.25 1.05 1.11 1.08 1.07 7276 6702 6537 5961 6582 6 14.80 14.91 14.97 14.77 14.86 22214 19734 19561 18139 19463 7 59.57 59.75 59.52 60.12 60.52 41269 36895 34296 33212 34556 8 155.76 164.67 166.98 173.03 183.42 55175 51759 47996 48023 47992 9 371.18 368.08 367.98 369.31 340.42 65144 60541 56752 54735 56361 10 612.50 622.37 627.38 623.98 592.20 80335 75028 69324 69413 68750 Next, the feature of task division is introduced and strategies S 1 S 5 are compared. Ten experiments were run whose specications are given in Table 6.7. Again the task networks chosen for these experiments are based on the articial neural networks architecture as shown in Figure 6.6. All ve strategies were applied on the ten scenarios. Here, the running times are denoted by T c and optimized mission completion times are denoted by T m . The results are shown in Table 6.8. Since the complexity of the rst seven experiments is signicantly lower compared to the last three (Table 6.7), the running times for the strategies in these cases do not dier signicantly. The last three experiments show signicantly dierent running times. From the data collected in Table 6.8, strategies which could be further discarded are decided. A strategyS i dominates another strategyS j if: i. both T m (S i )T m (S j ) and T c (S i )T c (S j ) ii. either T m (S i )<T m (S j ) or T c (S i )<T c (S j ) Out of the ten experiments, strategyS 4 is non-dominated eight times which is double that of the runner-up. However, no strategy is dominated in every experiment so none can be discarded based on Pareto eciency. Strategy S 4 outperforms the other four strategies except in experiments 8 and 10 where it was dominated by strategiesS 3 andS 5 respectively. If a strategy is dominated in 150 (a) (b) (c) (d) Figure 6.8: Application of strategyS 4 on task networks (a,c), and corresponding robot schedules (b,d) every experiment, then it is discarded. However, no such strategy exists based on the collected data. The rationale for the above observations is as follows: The missions considered here are based on spatially separated tasks that may be generally encountered by mobile robots. It is reasonable to assume that the time spent by the robots during traveling is signicantly less than the time spent by them during task execution, and therefore, high task duration was assigned to all the mission tasks. StrategyS 5 includes traveling durations in its guiding heuristic. Strategies S 1 and S 2 rst identify the stage-wise feasible tasks and then order them using task execution durations. 151 (a) (b) (c) (d) Figure 6.9: Application of strategyS 4 on task networks (a,c), and corresponding robot schedules (b,d) But strategy S 4 orders tasks based on their execution durations rst and the identication of feasible tasks and assignment of their optimal coalitions proceed later. In terms of running time, it is approximately equal to others. Based on these arguments, strategyS 4 is chosen as the optimal choice for contingency handling. The solutions generated by strategy S 4 are illustrated for four dierent mission scenarios in Figures 6.8 and 6.9. Task networks diering from ANNs are eciently handled. Color coding is used for better visualization of the solutions produced. Having robots idle for large amounts of time is not benecial for the mission, so such a duration is colored black. One can also consider 152 traveling as a task of low signicance and color traveling duration gray. The signicance of this coloring scheme is that when one looks at the solutions, the collective quantities of gray and black blocks are expected to be smaller than those representing the execution of the mission tasks. The task networks dene a particular color for each mission task which is used in the Gantt chart for the mission execution. Computational analysis is performed for strategy S 4 . In both Figures 6.11a and 6.11b, the running time is recorded by varying the number of mission tasks and robots respectively. In both (a) (b) (c) (d) Figure 6.10: Eect on performance of strategies by varying task duration compared to traveling duration 153 (a) (b) Figure 6.11: Computational performance of strategy S 4 Figures, the running time data is plotted for three dierent cases. In Figure 6.11b, on the yaxis, the running time normalized by the square of the number of mission tasks is shown (because O(m 2 ) is a good approximation, and doing this ts all three data-sets in one plot). 154 Table 6.9: Eect of uncertainty on mission makespan a e T m 1 T m 2 jT m 1 T m 2 j " (as %) 1 3 0.1 7427 7693 267 3.59 2 5 0.1 7363 7793 431 5.85 3 10 0.1 7613 8440 827 10.87 4 12 0.1 7564 8452 887 11.73 5 15 0.1 7585 8721 1136 14.97 6 0.1 10 7438 8309 870 11.70 7 0.1 20 7544 8910 1366 18.10 8 0.1 30 7578 9460 1882 24.84 9 0.1 40 7644 9935 2291 29.97 10 0.1 50 7702 10038 2336 30.32 For the experiments of Table 6.8, it is assumed that task execution duration is signicantly higher than travel times because real-life tasks have similar characterizations and observed that strategyS 4 is a better optimizer in most cases. But then one must also study the eect of varying task execution duration with respect to traveling time. The randomly generated ten task networks are used as before. Specically, the ratio of average task durationht e i to average travel timeht a i is varied. For strategy s, the meanht e i is computed over the number of tasks. So, the meanht a i is computed over the number of tasks. For a given problem scenario and strategy s, the ratio r s =ht e i=ht a i is computed. One can then compute the average ratio for the entire scenario as hr s i = ( P r s )=5 because ve strategiesS 1 S 5 are considered. These experiments are repeated on 100 problem scenarios and the underlying task network is chosen from among the ten generated task networks. The task duration is randomly varied within specic bounds such that the computed average ratio for each problem scenario is roughly the same. Then the mean as well as the standard deviation of the average ratios computed for the 100 experiments is reported (Figure 6.10). The histogram representing the frequency with which the strategies outperform others is also plotted. This is repeated for four dierent scenarios as shown in Figure 6.10a-6.10d by reducing the task-travel ratio successively. For high task durations, strategyS 4 wins as already discussed above. For all ten task networks considered here for benchmarking purposes, similar patterns were observed. 155 However, when task duration is comparable to travel time or even much shorter, then strategy S 4 is outperformed by strategyS 3 . When travel duration becomes more signicant, it is expected that strategyS 4 which relies partly on task duration as its task prioritizing heuristic will perform worse than strategy S 3 which disregards task duration by assuming it to be one for all tasks. Intuitively, strategy S 5 is expected to perform better in cases where there is signicant traveling involved because it tries to include traveling duration in its heuristic. But this strategy naively averages travel times over all possible routes the robots may take. The experiments show this is not good enough to beat strategyS 3 . For analyzing uncertainty results, a scenario comprising of 120 tasks, 9 robots is simulated. The task network is similar to Figure 6.6b and has 155 edges. The average task execution duration for the 120 tasks is around 450 seconds. The average time spent by robots to travel to task locations is computed by dividing the total time spent by all robots during traveling by the number of mission tasks and is around 70 seconds. In Table 6.9, the standard deviation associated with noise in traveling times and execution durations are allowed to reach up to 10 20% of their average values. T m 1 denotes the expected mission completion time computed by only considering the mean time expected for traveling and mission tasks. If the durations are instead modeled as Gaussian distributed variables and both the mean and variance information are used, then the computed expected mission completion time is denoted by T m 2 . The last column records the disparity caused by ignoring the variance information associated with task durations. Ignoring the eects of uncertainty leads to highly inaccurate decisions. 6.6.2 Contingency Management The methods of incorporating contingency tasks are illustrated through two examples in Figures 6.12 and 6.13. The problem scenarios for the two examples are shown in Figures 6.12a and 6.13a respectively. Results of nominal mission planning without taking contingency tasks into 156 (a) (b) (c) (d) Figure 6.12: (a) Mission scenario, (b) Robot schedules obtained without considering contingency tasks, (c) Incorporating reported contingency tasks in robot schedules, (d) Replanning schedules for robots when probability of contingency tasks exceed desired limits consideration are shown in Figures 6.12b and 6.13b respectively. When contingency tasks are considered, the resulting robot schedules are shown in Figures 6.12c and 6.13c respectively. When the probability values of the contingency tasks change so much that they trigger the replanning conditions, the newer schedules for robots are computed based on the updated information in Figures 6.12d and 6.13d respectively. In Figure 6.14, the time spent by robots during task execution is compared with the time spent in other activities including traveling, remaining idle or performing contingency tasks. Figures 157 (a) (b) (c) (d) Figure 6.13: (a) Mission scenario, (b) Robot schedules obtained without considering contingency tasks, (c) Incorporating reported contingency tasks in robot schedules, (d) Replanning schedules for robots when probability of contingency tasks exceed desired limits 6.14a and 6.14b show the doubly stacked bar-plots for the missions of Figures 6.12a and 6.13a respectively. The durations are normalized by dividing with the nominal mission completion time for each case. The set of left stacked columns in Figures 6.14a and 6.14b refer to the nominal scenarios of Figures 6.12b and 6.13b respectively. The set of right stacked columns in Figures 6.14a and 6.14b refer to the contingency scenarios of Figures 6.12d and 6.13d respectively. The average idle times of the robots are also computed for the scenarios shown in Table 6.8 and found to be well under 1% of the mission time. 158 (a) (b) Figure 6.14: Comparison of time spent during task execution and otherwise for problems of (a) Fig. 6.12a, (b) Fig. 6.13a This optimization scheme should ensure that the robots spend more time during task execution as opposed to other activities. All activities colored with a shade of gray are sought to be minimized by minimizing makespan. In Figure 6.14, note that most robots spend higher times executing tasks than otherwise. Note also that the contingency tasks adversely impact the robots because their engagement in mission-related activities gets prolonged. The sensitivity analysis results are shown for the contingency tasks based on six experiments. In Figure 6.17, the upper bounds are shown that are supposed to trigger replanning for those contingency tasks whose immediate execution was deferred. These experiments suggest that if the probability of contingency tasks adversely impacting the mission is higher than 50%, then it is highly likely that its execution is not deferred. However, the upper thresholds also depend on other characteristics of the contingency tasks impacting the mission like their locations, durations, and uncertainties. Therefore, contingency tasks can also have much smaller upper threshold, as shown in the rst three experiments. 159 (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) Figure 6.15: Comparison of conservative (C), reactive (R) and proactive (P ) approaches 160 Figure 6.15 compares the conservative, reactive, and proactive approaches for ten dierent problems. Generally, the proactive approach is expected to perform better than both the conser- vative and reactive approaches. In Figures 6.15a6.15c, proactive approach outperforms conser- vative approaches by around 30% and reactive approaches by around 15%. However, if the probability of contingency tasks impacting the mission is low, then the reactive approach is much better than the conservative approach, and the proactive approach will at least be as good as the reactive approach on average (Figure 6.15d). On the other hand, if (a) Exhaustive method (b) Pruning-based method Figure 6.16: Variation of running time with number of concurrently reported contingency tasks for strategyS 4 161 the probability is high, then the conservative approach will beat the reactive approach, and the proactive approach will at least be as good as the conservative approach on average (Figures 6.15e6.15i). There may also be cases where the three approaches perform approximately the same (Figure 6.15j). The three approaches were also compared on the ten task networks from Table 6.7 and it was found that the proactive approach is at least as good as the other two approaches on average. In Figure 6.16, the pruning-based method outperforms the exhaustive method by reducing the running times. The number of contingency tasks pruned for each of the experiments in serial order for the case of ve contingency tasks is: 2; 1; 2. Figure 6.17: Replanning trigger conditions obtained from sensitivity analysis 6.7 Summary The current chapter studies an approach of forming coalitions among robots and scheduling them for the execution of spatially distributed tasks having inter-task precedence constraints while incorporating contingency tasks. Uncertainty associated with traveling and task execution is taken into account while making decisions for the contingency tasks. In the present mission model, partial teams can commence task execution depending on the resource constraints of dierent mission tasks. Also, the robots assigned to a task can be interrupted mid-way and 162 rescheduled to another task. It can be concluded that the strategy that orders the tasks based on their execution durations using the concept of static levels provides the best trade-o. Running time is around 5 minutes for approximately 1000 tasks and 10 robots in MATLAB. Thus, it can eciently handle a large number of mission tasks during nominal mission planning. For a scenario involving six robots, ten mission tasks, and ve contingency tasks, the running time was around 20 seconds. In real missions involving multiple USVs, it is not feasible to consider higher numbers at least for centralized approaches. This should also hold for complex missions involving other kinds of mobile robots. 163 Chapter 7 Decomposition of Collaborative Surveillance Tasks with Uncertainty in Environmental Conditions and Robot Availability In this chapter 1 , an optimization approach is presented for partitioning a single, complex task among multiple robots. 7.1 Introduction Most categories of collaborative tasks involving one or more team(s) of robots require spatial partitioning of the region of interest. For example, for tasks like search and rescue, patrolling, and landmine detection, the robots need to sweep and clear the region of interest as early as possible. In such scenarios, it will be quite dicult for a single robot to accomplish the task in an acceptable amount of time. It will be advantageous to use a team of collaborative robots that can eciently decompose the task as early as possible. Decomposition of tasks among the teams of robots is a very challenging problem because a large variety of factors need to be considered. Some of these factors may be imperfect information of the environment, changing environmental conditions, varying performance of the robot as a result 1 The work in this chapter is derived from the work published in [227,228] 164 Figure 7.1: This is an illustration of a port scenario where a region has been demarcated for exploration by multiple USVs. of its interaction with the environment or risk of collision with external entities (e.g., civilian trac). The algorithm used to decompose the task has to ensure that the workload has been eectively distributed among all the robots according to their capabilities. Also, the algorithm should be intelligent enough to re-plan dynamically when it receives more information about the robots' operating condition as well as environmental conditions. Figure 7.1 shows an illustrative problem scenario. Suppose one is interested in exploring some region inside the port eciently using multiple USVs. The region of interest (green polygon) may have obstacles (red polygons). For an ecient exploration of the region to take place, one can partition the environment among the USVs to ensure that when the USVs start covering their assigned sub-regions, they complete exploring the region of interest eciently. Once such a partitioning has been completed, planning for the motion of the USVs can be performed using methods described in [229{232]. The proposed method helps to decompose the exploration task for a given region among multiple USVs. It is dierent from the geometrical area partitioning because one has to take into 165 account the dierent constraints to USV motion in the region of interest. To achieve this, the time taken by the bottleneck USV is minimized, which in turn helps in achieving workload balancing among all the USVs. Such cases are also considered where information regarding environmental conditions is unknown. 7.2 Problem Formulation 7.2.1 Denitions Let A be the planar region that is to be surveyed by the team of USVs. Region A is the closure of a subset of the Euclidean spaceR 2 in the usual topological sense. A = (x;y)j (x;y)2AR 2 (7.1) It is assumed that the region A is a simply-connected region of the Euclidean space R 2 . Simply-connected means there exists a path completely contained inA between any two arbitrary locations that lie inside the region. In other words, the planar region A does not consist of any disjoint regions. Let, O A be the set of all non-traversable or obstacle regions that lie inside the planar region A. These are the land areas that cannot be traversed by the USVs or port areas that must be avoided. Each obstacle O i 2 O is a simply-connected region like the region of interest A. Now, the traversable area within the area of interest A can be computed as A p = AnO A . Here, A p is the traversable area that needs to be surveyed by the team of USVs. The simply-connected property of region A remains intact even after removal of the simply-connected obstacles O from the region. Thus, it can be stated that the region A p is also a simply-connected traversable area. The boundary (denoted by @A p ) of the region A p is assumed to be non-traversable wherever it intersects with obstacles O. 166 It may happen that for a given dynamic model of a USV, a path respecting nonholonomic constraints may not exist between two points in the region A p even if a geometrical path exists. This problem only occurs when each USV starts coverage of its claimed sub-regions. In such cases, the USVs should ignore covering areas unreachable physically and then return the non-traversed regions to the mission planner so that the area can be reassigned to another team of USVs for exploration if required. If the obstacle-deducted region is not path-connected, a mission planner should have assigned the unconnected regions to dierent teams in the rst place. However, since the current chapter is mainly focused on exploratory task decomposition among multiple USVs, the nonholonomic coverage problem is not addressed here. Let the team of unmanned surface vehicles (USVs) be denoted by T n , where n is the total number of available USVs that may be used to survey the region A p . Each USV in the team is denoted byU i 2 T n , wherein. In this chapter, the primary focus is on studying a partitioning problem where a team of USVs T n is assigned to perform a collaborative exploration task of region A p . Here, it is assumed that the USVs are homogeneous and have the same physical capabilities. It is also assumed that a functioning global communication network exists among the USVs in the assigned team. While performing the collaborative exploration task, each USV U i 2 T n will explore a subset of region A p . Hence, the team of USVs has to divide the exploration tasks among themselves in an optimal manner. Essentially, this particular case reduces to a variant of area partitioning problem. Each USV claims sub-regions, A p1 ;A p2 ;:::;A pjTnj , in region A p such that the disjoint union of each sub-region sums up to the original area to be explored. A p =A p1 [A p2 [[A pjTnj (7.2) Equation 7.2 implies that all locations within the region of interest A p are explored by one USV only. Moreover, the USVs are not allowed to revisit the location it has already visited. This 167 is enforced in the way each USV is allowed to expand its claimed area, which will be discussed later. It is assumed that when a region A p was assigned to team T p , then the region was small enough so that by the expected amount of time in which the exploration has been completed, the conditions in the region A p have not changed signicantly. Hence the area of region A p suitable for exploration by a team of USVs will vary from port to port as well as within the port. During the exploration task, the motion of each USV will be aected by the surface currents, sea depth, and wakes generated by other civilian vessel and USVs. These factors introduce un- certainty in the positions of the USVs and also lead to noise in the controller. The positional uncertainties and controller noises increase with the increase in the USV's speed. Thus, the USVs operating at high speeds have large uncertainty in their positions and reduced controllability of the vessel, and vice versa. Therefore, the USVs operating at high speeds have a higher probability of collision with other civilian vessels and the static obstacles O. To handle the collision-risk, a velocity-mapV (a i ) is introduced that decides the maximum operating speed for the USV operat- ing in the region a i 2A p by keeping the collision-risk within acceptable bounds. The maximum operating speeds of a USV governs the rate at which the USV can explore the sub-region. In the regions with higher operating speeds (or lower risk), the USV will cover larger areas as compared to the regions with lower operating speeds (or higher risk). 7.2.2 State Space Representation The continuous region of interest A p is discretized with a uniform grid of l. During the division of region A p into sub-regions, it is assumed that each USV can traverse at least one grid cell in a unit time interval. Thus, the size of l is decided by the length of duration of discrete time interval (t) and the minimum operating speed of USV (v min ), i.e.,l =v min t. The discretized region of interest is denoted by A p;d . Each cell in A p;d is represented by state s = (x;y)2 Z 2 , where 1xx max and 1yy max . LetS be the set of all the states from region A p;d . 168 Neighbors of each state s are determined by performing a single step of Manhattan moves from s in each direction, i.e., North, South, East, and West. Let Neigh(s) be a function that returns all the feasible neighbors of state s. Finally, the discrete regionA p;d is approximated by a polygonB A , such that all the cells of regionA p;d intersect with or lie in the interior region ofB A . The discrete velocity-map V d (s) provides the maximum velocity the USV can operate at state s. 7.2.3 Problem Statement The goal is to seek an optimal division of area such that the area is covered in minimum time and work-load is optimally balanced among the USVs. This is achieved by minimizing the time taken by the bottleneck USV based on Minimax style of decision-making [233]. Formally, given (a) Region of interestA p;d , (b) Team of USVs,T n , and (c) Velocity-map,V d (), then one must compute optimal partitions of the region P i 2P for each USV U i 2 T n , where i2 [1;n]. Each partitioned region P i 2P is computed such that, the time taken by the team of USVs T n is minimized to explore then entire region A p;d . 7.3 Overview of Approach Below, the technique is described in detail. It is used to partition a region among multiple USVs based on a spatially distributed speed prole stored in a velocity-map. In this method, dierent initial placements of the USVs along the boundary of the polygon give rise to dierent partitions for the same region. So one must nd that initial placement of USVs for which the time taken by the last USV to nish claiming its area is minimum. No other initial placement should result in lower time for the last USV to nish. It is with regards to this criterion that a particular partitioning result will be referred to as optimal or not. The rest of the chapter is organized as follows: Section 7.3.1 describes how each USV grows its area based on speed constraints. Section 7.3.2 gives the objective function to be minimized. 169 Section 7.4.1 describes how the area partitioning simulations are implemented for known velocity- maps. Then, some helpful illustrations (Sec. 7.4.1) and computational performance plots (Sec. 7.4.2) are provided. In Section 7.5, the strategies are employed to handle unknown velocity-maps. 7.3.1 Partitioning the Region of Interest The discretized region of interest A p;d consists of discrete states s2S. Let N total be the total number of discrete states inS. It is desirable to compute the partitioned region for each USV U i 2T n while minimizing the time taken by bottleneck USV during its exploration of the region A p;d . A partition P of the region is a collection of n mutually exclusive and exhaustive subsets of discrete statesS, here n is total number of USVs in team T n . Algorithm 7 Determination of discretized region of interest Input: Polygon representing region of interest, obstacles, and grid structure Output: Set of grid-cells A p;d 1: Compute all cell-centers that collide with polygon and store as Boolean matrix 2: Use matrix to calculate which cell has at least one vertex intersecting with the polygon and mark such cells as part of discrete polygon A p;d 3: for all cells not in discrete polygon do 4: for all vertices in polygon do 5: Check intersection of polygon vertex with cell 6: if intersection found then 7: Add cell to discrete polygon A p;d 8: end if 9: end for 10: end for 11: Repeat Lines 3 10 for all obstacle polygons 12: for all cells in discrete polygon A p;d do 13: if cell not intersecting with an obstacle as found in Step 11 then 14: Cell remains in A p;d 15: else 16: Remove cell from A p;d 17: end if 18: end for Computing the discretized region of interest A p;d involves using polygonal collision checking routines. A p;d denotes a contiguous set of cells in the grid imposed on top of the continuous region. The recipe for calculating this set of cells is shown in Algorithm 7 which is described as follows. In computational geometry, point-in-polygon (PIP) algorithms are computationally very 170 ecient and can quickly decide whether a given point in the 2-D plane lies inside the boundary of a polygon or not. One call of such a software routine may be used to determine all grid cells whose centers lie inside the polygon and store the information from this collision-checking in the form of a Boolean matrix. Using standard geometric tricks, that matrix may be used to determine all cells whose corners may be inside the polygon. All these cells are marked as included in the set A p;d . Of the excluded cells, one must again determine computationally as to whether any vertex of the polygon representing regionA lies inside these cells. All those cells with which at least one polygon vertex intersects is also included in the set A p;d . But now one must remove all those cells which are intersecting with the obstacle cells. For this purpose, the same procedure is repeated that was carried out using the polygon representing regionA, but this time using the polygons representing each obstacle sequentially. This generates a set of cells that intersect with the obstacle polygons. All these obstacle cells must be excluded from the previous set of cells identied only using the polygonal region A. Then only the set of cells A p;d must be explored using the USVs. Algorithm 8 Boundary Characterization Input: n A 2 matrix of polygon vertices Output: Array of edge lengths l i , polygon perimeter peri(A) 1: peri(A) 0 2: for all edges of polygon do 3: Compute and store edge length l i 4: peri(A) peri(A) +l i 5: end for The boundary of the polygon representing the regionA p is parameterized by variable2 [0; 1). Here, = 0 represents the starting point of the closed polygon. Note that = 1 is excluded because it corresponds to the same point as = 0, i.e., the starting point on the boundary. The boundary itself is also discretized with a suitable step-size . This essentially reduces the boundary to an array of vertices from where area exploration is started by a team of USVs. For example, if = 0:1 then a USV may start from the boundary point denoted by = 0:9. Note a ner assignment such as = 0:92 is not possible because of the chosen scheme of boundary 171 discretization. So choosing a correct step-size to discretize the boundary is an important design decision taken to bring the approximate optimal boundary assignments to the USV as close as possible to the true optima without increasing the computational costs by a huge margin. For computing the partition of a USV that has been assigned the boundary point = 0:9, one would require further characterization of the boundary. This is shown in Algorithm 8. Considering the polygonal region A consisting ofjAj = n A vertices, the perimeter of the polygonal boundary is computed by traversing over n A edges while also storing the individual length of each edge. This is important as then one can use this information to map a particular value to its abscissa and ordinate. One needs to track the distance measuring peri(A) units from the starting vertex = 0. The edge lengths are used to determine which edge this vertex will lie in, and then the unitary method is used to determine the exact intermediate point on this edge. Once each USV has been assigned a boundary point, one readily obtains the starting states for each USV, s i;init . However, the mapping from space to states in grid-region A p;d is generally not injective though this also depends on the step size chosen to discretize the grid. Also, those boundary points which are inside obstacles will never be assigned as starting points for any of the USVs. At each time-step, the USVs can only move at speeds prescribed by the velocity-map, V d (s). The results shown below are for velocity-maps based on Cartesian, or polar system as well as contour lines. The speed is expressed in the units of number of cells traversed per unit time. Suppose for the USV starting at grid cell s, the maximum allowed velocity is denoted as V d (s). So, the USVU i will attempt to claimV d (s) states next time step to augment its currently claimed area. Also, denote the area claimed by USV U i at iteration t by a(t;i). Initially, all the USVs are assigned states along the boundary polygon B A of the region A p;d . All states are labeled open. At this point, each USVU i 2T n has claimed the state s i;init at which they are initialized. Each USV has claimed area a(1;i) = 1. In order to partition the region A p;d into n convex shaped simply-connected regions, it will be advantageous for each USV to select 172 (a) (b) Figure 7.2: (a) This image shows the example scenario discussed in Figure 7.3, (b) Contour-lines based velocity-map is used to model low speeds for USVs near land. states that are connected and nearest to current state of the USV. Let, C Ui be the cache of all the possible states that USV U i can claim in next time-step by maintaining the convexity of the partitioned region A Ui . Here, A Ui be the partitioned region for USV U i andS i be the set of all states belonging to region A Ui . In the initial step,S i will contain s i;init , and C Ui will contain all the neighboring states provided byNeigh(s i;init ). Letc 1;i denote the maximum number of states that USV U i can claim at time step t = 2. Clearly, c 1;i = V d (s i;init ). The cache must have at leastc 1;i states. Initially, cache of each USV, C U (i), is empty so at time t = 1, neighbors of state s i;init are added to the cache C U (i). Once the state s i;init has been claimed by one of the USVs and its neighbors are generated, the state is labeled as closed. Since the starting state is a boundary point, at least one neighboring state will lie outside the grid-region A p;d . But because non-convex polygons are allowed, it is possible to havej Neigh(s i;init )j 3. If c 1;i Neigh(s i;init ), then the partitioning procedure continues with claiming states next time step from the current cache. States unclaimed at time t = 2 from the cache will remain in cache for future times. But if c 1;i >Neigh(s i;init ), then those states in the cache, C U (i) that are open have their neighbors generated and stored in the cache. This process continues till the cache population,jC U (i)j, equals or exceeds c 1;i . 173 The actual number of states c 0 2;i that the USV claims for time step t = 2 depends on con icts with other USVs and also the availability of free states in its neighborhood to explore. Thus, 0c 0 2;i c 1;i (7.3) The frontier cells may be described as follows. At any time t, for a USV U i it refers to the c 0 t;i states it actually claims. Thus, the set of states that the USV U i has claimed at time step t = 2 can be denoted by s i;2 = [s 1 i;2 ; s 2 i;2 ; ; s c 0 2;i i;2 ]. These form the frontier cells for the USV U i at time step t = 2. These frontier cells augment the current claimed area for the USV to a(2;i) =a(1;i) +c 0 2;i . The rst cell s 1 i;2 is chosen from the contending open states in the cache such that it augments the current claimed area of USV U i with minimum loss of convexity. The chosen state should have minimum center-to-center distance from the center of the starting state of USV U i . Such an expansion in an obstacle-free and unconstrained square lattice invariably leads to a circle-like expansion. So, this heuristic is also expected to give good results for non-convex regions. The distance heuristic referred to above is not based on the classical Euclidean metric because the boundary contour itself maybe non-convex or contain non-convex polygonal obstacles inside it. Euclidean distance heuristic works well only in the absence of these concavities and holes in the map world. Instead, the Dijkstra's algorithm is applied [234] to every lattice point, thereby computing the lengths of shortest collision-free paths through the region, A p between each pair of lattice points as shown in Algorithm 9. The input for this algorithm is the undirected connectivity graph of the grid world along with its weighted edges. Weights are the Euclidean distances between neighboring states, hence either 1 or p 2. It is prudent here to allow diagonal connectivity as well to compute paths more tuned to the local concavities. The distances generated using such a look-up table is used as the heuristic in 174 guiding the growth of USV's claimed area, a(t;i). The sub-regions claimed by USVs will not be convex in general because of the non-convex free region A p . Algorithm 9 Compute all-pairs shortest path-lengths Input: Discretized region of interest A p;d Output: Look-up table 1: Initialize an empty undirected graphG 2: for all cells in A p;d do 3: Add cell as a node in the graphG 4: Compute neighbors and add as nodes if they lie in A p;d 5: Add weighted edges to neighbors from cells 6: end for 7: Compute all-pairs shortest distances by applying Dijkstra's algorithm on G 8: Store the distances in the form of a look-up table Figure 7.3 demonstrates how a USV grows its area. The values shown represent the maximum speed allowed at each cell. Figure 7.3a shows the snapshot of iteration 29 where the frontier cells appear as green. There are 2 frontier cells having velocity values of 1 and 2. Thus their average is 1:5, which is rounded to 2. So, in the next iteration, the USV tries to claim 2 more states among the free states such that maximum convexity is maintained. Thus, in Figure 7.3b, 2 more states have been added to the existing claimed area of the USV. This procedure is summed up in equations below. In the second iteration, there are possibly multiple frontier cells. Each cell corresponds to a value of maximum speed that can be attained around that cell. So one can take an average of these multiple maximum speeds to determine how many states should the USV U i claim next time step. Hence, the following equation is obtained: c 2;i = c 0 2;i P k=1 V d (s k i;2 ) c 0 2;i (7.4) Thereafter, new neighbors are generated as before from the current frontier cells in a breadth-rst search pattern. If more states are needed, then other open states in the cache can be expanded. 175 (a) Iteration 1 (b) Iteration 2 (c) Iteration 29 (d) Iteration 30 Figure 7.3: In the above images, the brown cells are outside the region of interest, the light blue cells represent the free, unexplored region and the dark blue cells represent regions already claimed by an USV for exploration. (c) The green cells represent the frontier cells for iteration 29. The scheme used for the above exploration is explained in detail in Section 7.3.1. 176 This process continues till the grid-region A p;d has been completely explored. Thus, the number of cells claimed evolve for each USV as follows: c 1;i =V d (s i;init ) (7.5) As pointed out earlier, the actual number of claimed cellsc 0 t;i may not be equal to the maximum number of cells that can be claimed, c t;i . c t;i = c 0 t;i P k=1 V d (s k i;t ) c 0 t;i c 0 t;i =f(c t1;i ) (7.6) The exact number c 0 t;i is however computed by doing the simulations which is represented by function f() above. Also, area claimed by USV U i evolves as follows: a(1;i) = 1 a(t;i) =a(t 1;i) +c 0 t;i (7.7) Next, a summary of this elaborate procedure is provided from the implementation point-of- view in Algorithm 10 to compute the partitions of a region given the initial boundary assignments to the USVs. Area partitioning proceeds in iterations, and for each iteration, a random ordering of USVs is generated, and USVs start claiming their cells in that order. For this purpose, each USV maintains its own open list. This list is populated rstly by the starting cell of each USV. In the rst iteration, each USV claims the only cell in the open list, which is its starting cell. This is a trivial step. Each USV then adds the feasible neighbors of the starting cell into the open list. Each USV also computes the number of cells that it should claim in the second iteration in accordance with the velocity-map. 177 In subsequent iterations, each USV rst populates its open list with enough cells to claim in that iteration. It does this by adding the neighbors of the cells in the open list. If there are still not enough cells in the open list, then one must include the neighbors of the cells in the open list at the start of this iteration. Algorithm 10 Compute Partitions Input: Discretized polygon, USV starting points Output: Partitioned polygons 1: while region not explored do 2: O U = random ordering of USVs 3: iter 1 4: for USV u i in O U do 5: if iter == 1 then 6: remove current cell from ith open list 7: assign current cell to USV i 8: mark current cell as claimed 9: generate neighbors of current cell and populate ith open list 10: compute number of cells n i to claim next time step using velocity map 11: else 12: populate open list with enough cells to claim 13: if success in populating then 14: sort unexplored cells in open list by shortest path-lengths 15: for rst n i cells do 16: remove current cell from ith open list 17: assign current cell to USV i 18: mark current cell as explored 19: end for 20: compute number of cells n i to claim next time step using velocity map 21: else 22: for all unexplored cells in the ith open list do 23: remove current cell from ith open list 24: assign current cell to USV i 25: mark current cell as claimed 26: end for 27: compute number of cells n i to claim next time step using velocity map 28: end if 29: end if 30: end for 31: check if all cells in A p;d have been claimed 32: iter iter + 1 33: end while The number of times each USV may recursively keep exploring its neighbors in this fashion to populate its open list is kept xed. For the simulations conducted for the present chapter, 178 partitioning proceeded smoothly if this procedure of expanding neighbors recursively stopped at the fourth level. Mostly the open list gets populated way before reaching the fourth level of neighbor expansion. So neighbor search gets terminated mostly in rst or second levels. If velocity maps assign very high area growth rates, then one may have to allow the possibility of higher levels of neighbor expansion. If the open list is successfully populated with enough cells, then one must sort them in increas- ing order according to their distance in the geodesic sense from the starting cell for each USV and then select the cells in the open list for the corresponding USV according to this new ranking. Then the number of cells that may be claimed by each USV in the next iteration is computed based on the velocity-map. It may happen that the open list of some of the USVs does not get populated as desired; then those USVs claim all the cells in their open lists. 7.3.2 Optimizing Partition The states in the region A p;d are divided among multiple USVs in a nite number of time steps, t finish . At each time step, the USVs search their neighboring unclaimed states and keep expanding their claimed sub-region for exploration. The iterations continue until all free states have been claimed. Depending on the shape and size of the region A p;d , each USV U i 2 T n will take a dierent amount of time to claim its sub-region. The USV which claims the last unclaimed state is considered to be the bottleneck agent. In the scenario where multiple USVs nish in the nal iteration, any one of them can be considered the bottleneck agent. The time taken by each USV to claim all the possible free states depends upon the initial location of the USV on the boundary B A . It is assumed that each USV is allotted a dierent initial state. 179 Suppose USV U i takes time t if to complete claiming states for exploration. Then the time step at which the process of area partitioning terminates as well as the bottleneck USV U i is determined as follows: t finish = max i (t if ) i = arg max i (t if ) (7.8) The initial starting states map to anndimensional vector, U = [ 1 ; n ]. The vector elements should be suciently spaced so that two or more USVs do not claim to the same initial state. If the minimum time taken by the bottleneck USV is allowed to only be a function of vector U , then this problem can be formulated in terms of optimal initial placement U as follows U = arg min U max i (t if ) t finish = min U max i (t if ) (7.9) One can also modify the objective function to take care of initial locations of the USVs. The USV U i will be at some state outside region A p;d before exploration of the region has actually begun. The distance it will travel to reach state s i;init should also be included in the cost incurred. Based on the maximum speed allowed (from velocity-map) at each state that will be traversed by USV U i along the shortest path to reach state s i;init , one can calculate an optimistic estimate of time taken, t i0 . U = arg min U max i (t if +t i0 ) (7.10) 7.4 Area Partitioning Using Known Velocity-map 7.4.1 Approach While implementing the partitioning algorithm, the USVs are not implemented to behave like concurrently operating agents that claim states in the region of interest simultaneously. However, 180 if the USVs sequentially claim states, then there will be a bias in the partitions created among the USVs in favour of the USV which is assigned at the top of the list of USVs. Instead, a random order is generated for each discrete time-step using which the USVs claim the states from the set of free available states. This is the only source of randomness in this task decomposition technique. During area partitioning, this protocol will lead to dead-locks for those time steps when mul- tiple USVs lay claims on the same cell. To resolve such deadlocks, one again makes use of the random ordering of the USVs assigned to each iteration of the simulation. In case of a deadlock, the USV with higher priority at that time step wins the bid to its claim. The scenarios used to evaluate the performance of the developed approach were assigned increasing degree of concavity to represent regions in ports such as narrow channels or small, enclosed regions in the harbor (see Figures 7.5, 7.6, 7.7 and 7.8). A scenario of size 500 500 meters is selected and discretized with a uniform grid-size of 10 meters. Four types of velocity-maps are generated for testing this area partitioning scheme. The simplest example of such a velocity-map would be a uniform, constant map such that all states have the same value of maximum speed allowed. Then, maps where velocities for each state are dened as a function of its Cartesian (Figure 7.5a) or polar (Figure 7.7a) coordinates are also generated. The former was based on using linear gradients to signify speed zones. By applying a linear gradient horizontally and superposing it with another linearly graded risk prole in a vertical direction, one can obtain the velocity-map, as shown in Figure 7.5a. The horizontal prole prescribes maximum risk on the left, whereas the vertical prole prescribes maximum risk to the top of the area. Figure 7.7a uses a velocity-map dened in terms of polar coordinates such that the risk con- straints are described in terms of radius and azimuth rather than the usual Cartesian coordinates. The most useful is the contour map (Figure 7.9a), which assigns lower speeds near land and higher speeds away from land. 181 Topographic maps use contour lines to denote the elevation. It is not only an excellent visu- alization tool but also a very convenient way of recording features of the terrain. In the present chapter, the velocity constraints and underlying risk factors at dierent locations in the port re- gion A p are caused by features of the seabed such as depth, and features of the sea surface such as current. Trac and unaccounted objects owing on sea surface also should be taken into account. Velocity-maps can also be represented as contour maps. One such velocity-map along with its example is shown in Figure 7.9. The time taken by the bottleneck USV is only optimized in the present chapter over the boundary points assigned to each USV. This decision variable has already been dened by the vector, U = [ 1 ; 2 ; n ]. For a given scenario, the initial choice of n boundary points determines the partitions along with the randomness introduced in prioritizing the USVs at each time step. Other factors like world size and region of interest are held constant in the current simulation. Thus, the mission completion time is minimized over the vector of starting states U . The objective function, ap( U ) = max i (t if ( U ) +t i0 ) is non-linear. The elements of the decision vector are bounded in the subset [0; 1). The objective itself is an ill-dened mapping as far as gradient-based optimization methods are concerned. ap : ( U 2 [0; 1] n )!Z (7.11) Since the mapping associates every bounded vector to a positive integer, it is expected not to be dierentiable. Classical gradient-based methods are expected to perform poorly. Therefore, the PSO method is employed [235] to minimize the time taken by bottleneck USVs to explore the region. PSO is used because it eciently handles higher dimensions and does not require the objective function to be dierentiable. 182 In the present chapter, the cognitive trust parameter is chosen to be 1:7 and the social trust parameter to be 2. A population size of 12 was chosen for optimization over four USVs, and PSO was run for 75 iterations. The tolerance for the convergence of the solution was chosen to be 0:001. The initial population is chosen as the uniformly spaced placement of USVs along the polygonal contour in terms of the parameter, viz: 0; 0:25; 0:5; 0:75. The above-discussed scheme of decomposing exploration tasks among multiple USVs is coded in MATLAB(R) software [236] on Ubuntu MATE 18.04.1 operating system. The workstation used was Dell(R) Precision Tower 3620 including eight Intel(R) Xeon(R) E3-1245V5 CPUs having 3.5GHz speed and a total of 32GB RAM. Computation time for generating partitioned regions for a team of four USVs exploring the region (see Figure 7.4c) and velocity-map (see Figure 7.5a) is approximately 17 seconds. The partitioning of the area by the USVs to minimize the time taken by the bottleneck USV depends heavily on the velocity-map available. This is because velocity-map determines the maximum velocity allowed at each cell, and hence the maximum number of states that can be claimed by the USVs at each time step. A constant velocity-map is compared with the linear velocity-map in Figure 7.4. For the constant risk case such that maximum speed allowed is 4m=s for each cell, as in Figure 7.4a, the optimal bottleneck time obtained through simulation is 113 seconds whereas for Figure 7.4b it is 89 seconds. The addition of obstacle reduces the number of states to be explored and hence results in less time taken for the latter case. Also, the constant velocity-map leads to four fairly equally divided regions for the four USVs. The number of USVs is only representative here and can be varied as needed. In Figure 7.4c, the total number of iterations required for dividing the region among multiple USVs is 131 whereas for Figure 7.4d it is 103. Again as expected, the linear case without obstacles requires larger time than the corresponding constant-map case. This is due to the risk prole in Figure 7.5a, which prescribes maximum speeds of magnitudes as low as 2m=s. In contrast, the 183 (a) (b) (c) (d) Figure 7.4: (a-b) Uniform, constant velocity-map, (c-d) Linear velocity-map constant velocity-map prescribes speeds of magnitude 4m=s, irrespective of the cell location. And, though some states in the non-uniform case do achieve speeds of magnitude as high as 6, it is clear from the simulation that the constant-map case completes faster. This is because only a few cells with such high speeds fall inside the region of interest. This pattern is observed in corresponding obstacle cases as well (Figure 7.4b and 7.4d). Note that in the obstacle cases, the region of interest is not divided equally among all the four USVs. This is because the presence of obstacle results in high concavities in the free region that will be explored. Similar behavior will be seen in the absence of obstacles if the region of interest A p itself is highly concave. Also, since the top and 184 (a) (b) (c) (d) Figure 7.5: (a) Linearly graded velocity-map, (b-d) Application on manually created regions. left states of the map have lower speeds while the right and bottom states have higher speeds, so USVs originating in the lower right zones claim larger area for exploration than the USVs starting on the left in Figure 7.4c. Referring back to Figures 7.4a and 7.4c, if the polar map is applied to the same region in Figure 7.7b, then the optimal partition in this case is very dierent. In Figure 7.5(c-d), the linear map is applied to a convex and non-convex region. Then, the map is applied to regions modeled using real scenarios obtained from Google(R) Maps in Figure 7.6. 185 (a) (b) (c) (d) Figure 7.6: Application of the linearly graded velocity-map on regions based on real maps. In Figure 7.7(c-d), the polar map is applied to two highly non-convex regions. The polar map is then also applied to two regions obtained from Google(R) Maps in Figure 7.8 but a smaller spatial discretization of 5 meters is chosen to model the narrow channels correctly. Otherwise, they get treated as obstacles when spatial step size is chosen to be 10 meters. 7.4.2 Computational Results In this section, the eectiveness of the algorithm is evaluated by varying the number of USVs from 2 to 10. Minimization of time taken by the bottleneck USV requires each USV in the team 186 (a) (b) (c) (d) Figure 7.7: (a) Polar velocity-map, (b-d) Application on manually created regions. to nish claiming all the possible remaining unexplored states at the same time as the bottleneck USV. Figure 7.10(a) shows a box-plot for dierent ranges of the nal exploration time required by the team of USVs over 200 simulations. The range is computed by taking a dierence between the maximum and minimum value of the data. Theoretically, the range values for the nal exploration times of the USVs should be zero, but non-convexity associated with a region leads to deviation from ideal results as can be seen in the simulation data recorded. The values shown on the y-axis (see Figure 7.10(a)) are discrete values of time-step. In Figure 7.10(a), the median of the data is 2 while the minimum is zero, which is expected. The maximum range is capped at 6, and the 187 (a) (b) (c) (d) Figure 7.8: Application of the polar velocity-map on regions based on real maps. interquartile range is 2. These results point to the successful minimization of time taken by the bottleneck USV. The reasoning behind the zero range is related to the idea of optimal load balancing. If the bottleneck USV takes much higher time to compute its explorable region compared to other USVs, it is implied that other USVs could be prompted to claim more area to reduce the load on the bottleneck USV. This process can then ideally be continued until all USVs nish at approximately the same time. However, this still does not guarantee that the area will be equally divided. In particular, if a USV is in a high-risk zone, then it will cover less area than another USV operating 188 (a) (b) Figure 7.9: (a) Contour-lines based velocity-map, (b) Corresponding area partition in a low-risk zone for the same amount of time. Additionally, if a region is highly non-convex, then it may not be possible to partition the area among the USVs such that they nish almost at the same time. In such cases, the optimal bottleneck time may signicantly be higher than the nish times of the other USVs. In spite of such issues, optimal load balancing is possible for most cases by computing the appropriate initial placement of the USVs. Figure 7.10(b) shows the box-plot of the computational times obtained for partitioning the area by varying the number of USVs. The y-axis plots the percentage reduction in computation time with respect to the mean computation time required by the algorithm to compute the optimal partitions for two USVs. Ordinarily, the optimization with more number of USVs should take more computational time. However, with the increase in the number of USVs, the areas assigned to each USV reduce. This results in a decrease in the number of time-steps required to compute optimal partitions for each USV. So, the time to compute optimum partitions of the region for each optimization iteration reduces. This eect is more signicant and leads to a reduction in overall computation time. 189 (a) (b) Figure 7.10: (a) This image shows the range of times required by the USVs for exploration of given regions from over 200 simulations in the form of box-plot, (b) Here, the variation of computational time required for generating area partitions with respect to the number of USVs involved in exploration is shown. 7.5 Area Partitioning Using Unknown Velocity-map 7.5.1 Approach In this section, the robustness of the area partitioning algorithm is evaluated when the velocity- map is unknown initially. The algorithm does not have perfect knowledge of the velocity-map while computing the initial exploration plan. However, while the USVs are exploring the region, they gather more information about the achievable maximum velocity and obtain the correct velocity-map. So, two approaches are developed to handle such scenarios of imperfect or unknown velocity-maps. In the rst approach, the optimal starting boundary points U for the USVs is determined based on uniform, constant velocity-map. USVs begin covering the region starting from the boundary points U but claim states based on speeds from the correct velocity-map. This is because even if the USVs do not know about the correct velocity-map but while exploring the region, USVs will move according to the actual environmental conditions. While the USVs are covering the areas in real-time, the central mission planner waits for the correct velocity-map to 190 be received. The planner then uses the correct velocity-map to compute the optimal partitions for the remaining region. During this computation, the planner excludes the states of the region A p;d that have already been covered by the USVs. In the second approach, the USVs proceed with a noisy estimate of velocity-map until the correct velocity-map is obtained. The noisy velocity-map is assumed to be the best estimate of the correct velocity-map based on the sensor data that is available at that moment. It is assumed that the time taken by the USVs to estimate the correct velocity-map ist estimate . The optimal execution time required by the team of USVs to explore the region A p;d with perfect information of the velocity-map is denoted byt optimal . Finally, the optimal execution time required by the team of USVs for the region remaining after timet estimate using the updated velocity-map is given by t remaining . So one can say that t optimal < t remaining +t estimate . If the values of the estimation time t estimate are low, then one can also state that t optimal t remaining . 7.5.2 Computational Results In this section, computational experiments are performed by varying the time taken, t estimate , by the USVs to estimate the correct velocity-map. To simplify the implementation, the results are presented by varying the percentage area covered by the USVs before it acquires the correct estimate of the velocity-map (shown on thex-axis of the graphs in Figure 7.11). Figure 7.11 plots the percentage increase in execution time taken for exploration by the USVs from the true-optimal timet optimal on they-axis. The simulation data presented in the form of box-plots are taken over 100 simulations with medians denoted by the red lines. The simulation scenario used for the experiments is shown in Figure 7.4. Figure 7.11 gives plots that indicate that with more delay in estimating the correct values of the velocity map, there is a steady increase in the execution time incurred by the team of USVs. Both the approaches perform quite similar when the correct estimates of velocity-map are received before the USVs cover about 10% of the area. However, when the USVs cover more 191 (a) (b) Figure 7.11: In the absence of correct velocity-map, one can either use (a) constant map, or (b) noisy map; and record the time taken to complete partitioning the region based on how much time into the area partitioning process the correct map becomes available. The observed trend is explained in detail in Section 7.5. than 20% of the area without correct estimates, then the approach which computes the initial plan using the noisy estimates performs marginally poor as compared to the rst approach which starts with a uniform, constant map. This suggests that it is better to use the mean estimate of the velocity-map computed over time as compared to using currently available noisy estimates. 192 It is seen from the graph (shown in Figure 7.11) that the % increase in execution time taken by the USVs from the true-optimal time dips below zero. This is because the median time taken by the team of USVs when correct velocity-map is available from the beginning is set to zero. 7.6 Area Partitioning Under Variable Robot Availability In this section, the eects of robots becoming suddenly unavailable in the middle of task execution is studied. It must be analyzed how robust this method is against such contingencies, and what are the eects of such events on the resulting task decomposition. Suppose three USVs are assigned to a region for exploration. This region is partitioned and assigned to the USVs for exploration. Assume that while exploring, one of the USVs becomes unavailable. So, the rest of the region must again be partitioned between only the two remaining USVs. Let the time-steps it would take two USVs to partition the region completely be t 2 , and for three USVs be t 3 . Suppose that % of the region was covered by the time one of the USVs becomes unavailable. Then the steps to partition the region with three USVs up to %, and with two USVs for the rest could be linearly approximated as t 3 + (1)t 2 . At least, this is what would be expected for a constant unit map and a convex region of interest. Following simulation experiments are run to analyze the eects of such robot unavailability. Figure 7.12a shows plots for a scenario involving a convex region of interest R a and a uniform velocity map with a constant speed of 4 cells per time steps. If this map is denoted with an index of 1, thent 1 2a denotes the time taken by two robots to cover region R a using the 4uniform map. A nonuniform map which is similar to a linearly graded map shown before is used and denoted by index of 2. It has speeds varying from 2 to 6 cells per time step. Figure 7.12a shows two pairs of four horizontal black lines. These horizontal lines indicate the linear approximations obtained from the weighted averaging as shown above for both map scenarios. 193 (a) (b) Figure 7.12: Eect of a robot becoming unavailable in the middle of task execution on the time to partition a (a) convex , and (b) nonconvex region The yaxis shows the time-steps taken to partition the given region. It is represented in the percentage form with respect tot 1 3a . Thexaxis shows the percentage of area covered after which one of the agent becomes unavailable. The plots show that transitioning from a uniform map to a nonuniform map worsens the time required to partition the region but it is well within 30% in each case. Again this is expected because the linear approximation will break for nonuniform maps. In the case of the nonuniform map, the deviation from the horizontal line is clearly seen to be more than the uniform case. Figure 7.12b shows similar patterns for a manually created nonconvex region R b . It is expected that the linear approximation will perform even worse for a nonconvex region. Thus, the time to partition the region worsens between the uniform and nonuniform map but still stays around 30% with respect to the time taken by three robots to nish the task without interruption for the uniform map scenario. Thus, despite unfavourable scenarios the task completion times do not deviate erratically and remain bounded within reasonable limits. 194 7.7 Summary In this chapter, an algorithm is developed that solves the area partitioning problem while taking into account the physical constraints applied to the USVs. The algorithm can optimally partition a large spatial region among multiple USVs using underlying velocity-maps well under one minute. 195 Chapter 8 Conclusions This chapter presents the expected intellectual contributions and anticipated benets from the work proposed in this dissertation. 8.1 Intellectual Contributions This dissertation built computational foundations for handling contingencies proactively during missions involving multiple robots. Nominal mission features can be handled using fast heuris- tic methods. But for handling uncertain contingency information, optimization-based methods must be employed. Thus, to proactively incorporate contingencies into nominal mission plan in real-time, the problem must be partitioned to strike the right balance between optimization and heuristic domain. The right interplay between the use of optimization and heuristics can help obtain solution with high quality in a computationally ecient manner. This approach is used in Chapters 6 and 7. Another approach to create ecient contingency-aware plans is to develop a multi-layered structure for handling dierent levels of the problem. This was demonstrated through Chapters 3 to 5 where a model checking framework was used for encoding the mission model and queries using formal methods. On top of this, a sampling-based Monte Carlo simulator was used to quickly evaluate the performance of dierent contingency resolution strategies. And 196 nally a high-level reasoning framework was deployed to perform aggregate analysis for prob- lems of larger sizes. Such multi-layered and combined approaches can be applied to solve other computationally challenging problems arising in multi-robot mission planning domain. The key contributions according to the three main problems dealt within this dissertation are summarized in the following sections. 8.1.1 Modelling and Verication of Contingency Resolution Strategies for Multi-robot Missions Chapter 3 describes a framework to do modeling of multi-robot missions with deterministic and probabilistic transitions based on a nominal mission description. The modeling of contingency resolution strategies is also demonstrated with specic examples of contingencies for two case studies. In the rst case study, the correctness of contingency resolution strategies is veried, and NuSMV is used to generate mission execution plans for dierent scenarios. In the second case study, PRISM is used as a computational tool to provide a probabilistic assessment for mission success and analyze the eect of relevant mission variables on mission execution. It is demonstrated that subsystems can be analyzed individually, and their results can be aggregated to evaluate the performance of the proposed operations plan for the entire system, thereby help- ing to design more optimized systems. A modeling approach is presented for a production line set up across multiple robotic assembly cells. PRISM software is used to combine probabilistic modeling techniques with temporal logic based evaluation of the system. It is demonstrated how the cell settings could be modied based on feedback from the encoded mission model to optimize production performance. 197 8.1.2 Incorporation of Contingency Tasks in Nominal Task Allocation The multi-robot task allocation approach developed in Chapter 6 eciently handles contingency tasks by incorporating probabilistic analysis coupled with pruning to reduce the combinatorial explosion of computational requirements. The proactive approach is shown to outperform the conservative and reactive approaches to handling contingency tasks. This dissertation presents an algorithm for incorporating contingency tasks in a mission plan. It is expected that the proposed algorithm will be feasible for planning up to 10 contingency tasks because it takes under a minute in MATLAB implementation for ve contingency tasks, and one could reasonably expect at least 8090% reduction for a parallelized C++ implementation. A higher number of contingency tasks such as those that were considered in this work will generally not be encountered in such missions. 8.1.3 Decomposition of Collaborative Surveillance Tasks Chapter 7 develops an algorithm that solves the area partitioning problem while taking into account the constraints on the rates at which USVs can perform exploration tasks. The algorithm can partition a large spatial region among multiple USVs using underlying velocity-maps well under one minute. It is shown that by increasing the number of USVs, the computational time reduces. The performance of the algorithm was evaluated in scenarios where the velocity-map is unknown initially for a given length of time. Subsequently, the accurate velocity-maps becomes available after the USVs have explored part of the desired region. It is observed that the percentage increase in the optimal partitioning time remains below 5% even when the accurate velocity-map becomes available after 10 20% of the area has been explored. 8.2 Anticipated Benets This dissertation presents methods to incorporate unexpected, contingency tasks that may ad- versely impact the progress of multi-robot missions. Autonomous systems are being deployed 198 rapidly in various scenarios to automate the operations and increase eciency in the long run. However, the deployment of autonomous systems is not possible without guarantees against con- tingency events. Reactive methods of handling contingency events cannot provide such guaran- tees. Contingencies need to be handled proactively, which implies that it must be incorporated in nominal mission planning tasks like model verication, task allocation, and task decomposition. The approach in this dissertation is to set up simulation models that incorporate contingency information into nominal mission planning techniques in a deliberative and proactive manner, and then study and analyze the performance of this approach. Seeking such solutions provide a robust degree of condence and guarantee to proceed with the operations of autonomous systems, especially when the operation site is complex and is frequently traversed by humans. 8.3 Future Work The present work can be extended in the following directions: • The work presented in this dissertation does not take unanticipated contingencies into con- sideration. Enabling the ability to process environmental data and predict new types of con- tingencies that may occur can help such automated contingency handling systems achieve much better performance than existing state-of-the-art systems. This would possibly re- quire humans in the decision loop but computational tasks may still be carried out by the algorithms. • A potential research direction for extending this dissertation is to develop interaction tech- niques for humans to eectively supervise robot teams and provide input on contingency handling. It would be also interesting to integrate the formal modeling approach presented in this dissertation with scheduling software packages or custom-designed algorithms. This is because scheduling allows the investigation of the behavior of involved agents in a more 199 detailed manner. In scheduling literature, operation performance is enhanced by studying the agent behavior, identifying undesirable behaviors, and addressing them. • Limitations of the approach presented in this dissertation include the use of manually de- signed heuristics to plan for multi-robot missions. One could deploy machine learning al- gorithms for discovering task allocation heuristics. One could also deploy machine learning algorithms so that newer and better contingency resolution strategies are automatically generated based on sensory feedback and the history of human interventions. This will be eective for scenarios where contingencies occur at high frequencies amid rapidly changing mission conditions. • Another current limitations of the approach presented in this dissertation is that robot teams of large sizes are not taken into consideration. With the advancement of technologies, it is however possible to deploy large teams of robots to accomplish complex and higher duration missions. One could develop scalable techniques that account for contingency occurrence in large robot teams using techniques from swarm robotics. Future work could also model issues arising for heterogeneous teams. 200 Reference List [1] Yu Zhang and Lynne E Parker. Multi-robot task scheduling. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pages 2992{2998, 2013. [2] James S Jennings, Greg Whelan, and William F Evans. Cooperative search and rescue with a team of mobile robots. In Proceedings of IEEE International Conference on Advanced Robotics (ICAR), pages 193{200, 1997. [3] Antonio Barrientos, Julian Colorado, Jaime del Cerro, Alexander Martinez, Claudio Rossi, David Sanz, and Jo~ ao Valente. Aerial remote sensing in agriculture: A practical approach to area coverage and path planning for eets of mini aerial robots. Journal of Field Robotics, 28(5):667{689, 2011. [4] Joseph L Baxter, EK Burke, Jonathan M Garibaldi, and Mark Norman. Multi-robot search and rescue: A potential eld based approach. In Autonomous robots and agents, pages 9{16. Springer, 2007. [5] George Kantor, Sanjiv Singh, Ronald Peterson, Daniela Rus, Aveek Das, Vijay Kumar, Guilherme Pereira, and John Spletzer. Distributed search and rescue with robot and sensor teams. In Field and Service Robotics, pages 529{538. Springer, 2003. [6] Michal P echou cek and Vladim r Ma r k. Industrial deployment of multi-agent technologies: review and selected case studies. Autonomous agents and multi-agent systems, 17(3):397{ 431, 2008. [7] John Enright and Peter R Wurman. Optimization and coordinated autonomy in mobile fulllment systems. In Proceedings of AAAI Workshop on Automated Action Planning for Autonomous Mobile Robots, pages 33{38, 2011. [8] Simon Bogh, Casper Schou, Thomas R uhr, Yevgen Kogan, Andreas D omel, Manuel Brucker, Christof Eberst, Riccardo Tornese, Christoph Sprunk, Gian Diego Tipaldi, et al. Integration and assessment of multiple mobile manipulators in a real-world industrial production facility. In Proceedings of International Symposium on Robotics (ISR/Robotik), pages 1{8, 2014. [9] Hadi Ardiny, Stefan Witwicki, and Francesco Mondada. Construction automation with autonomous mobile robots: A review. In Proceedings of RSI International Conference on Robotics and Mechatronics (ICROM), pages 418{424, 2015. [10] Qin Li, Alexander Pogromsky, Teun Adriaansen, and Jan Tijmen Udding. A control of colli- sion and deadlock avoidance for automated guided vehicles with a fault-tolerance capability. International Journal of Advanced Robotic Systems, 13(2):64, 2016. [11] Brual C Shah and Satyandra K Gupta. Long distance path planning for unmanned surface vehicles in complex marine environment. IEEE Journal of Oceanic Engineering, Accepted for publication. 201 [12] Ariyan Kabir, Alec Kanyuck, Rishi K. Malhan, Aniruddha V. Shembekar, Shantanu Thakar, Brual C. Shah, and Satyandra K. Gupta. Generation of synchronized conguration space trajectories of multi-robot systems. In IEEE International Conference on Robotics and Automation (ICRA), Montreal, Canada, May 2019. [13] Wolfram Burgard, Mark Moors, Dieter Fox, Reid Simmons, and Sebastian Thrun. Col- laborative multi-robot exploration. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), volume 1, pages 476{481, 2000. [14] Nicholas Roy and Gregory Dudek. Collaborative robot exploration and rendezvous: Algo- rithms, performance bounds and observations. Autonomous Robots, 11(2):117{136, 2001. [15] Ioannis M Rekleitis, Gregory Dudek, and Evangelos E Milios. Multi-robot collaboration for robust exploration. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), volume 4, pages 3164{3169, 2000. [16] Ashley Stroupe, Terry Huntsberger, Avi Okon, Hrand Aghazarian, and Matthew Robinson. Behavior-based multi-robot collaboration for autonomous construction tasks. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1495{1500, 2005. [17] Dalong Wang, Dikai Liu, and Gamini Dissanayake. A variable speed force eld method for multi-robot collaboration. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2697{2702, 2006. [18] Y Uny Cao, Alex S Fukunaga, and Andrew Kahng. Cooperative mobile robotics: An- tecedents and directions. Autonomous robots, 4(1):7{27, 1997. [19] J-H Kim, H-S Shim, H-S Kim, M-J Jung, I-H Choi, and J-O Kim. A cooperative multi-agent system and its real time application to robot soccer. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), volume 1, pages 638{643, 1997. [20] R. K. Malhan, A. M. Kabir, B. Shah, T. Centea, and S. K. Gupta. Determining feasi- ble robot placements in robotic cells for composite prepreg sheet layup. In ASMEs 14th Manufacturing Science and Engineering Conference, Erie, PA, USA, June 2019. [21] R. K. Malhan, A. M. Kabir, A. V. Shembekar, B. C. Shah, T. Centea, and S. K. Gupta. Hy- brid cells for multi-layer prepreg composite sheet layup. In IEEE International Conference on Automation Science and Engineering (CASE), Munich, Germany, Aug 2018. [22] R. K. Malhan, A. M. Kabir, B. C. Shah, T. Centea, and S. K. Gupta. Automated prepreg sheet placement using collaborative robotics. In 2018 SAMPE, Long Beach, CA, USA, 2018. [23] Prahar M. Bhatt, Ariyan M. Kabir, Rishi K. Malhan, Brual C. Shah, Aniruddha V. Shem- bekar, Yeo J. Yoon, and Satyandra K. Gupta. A robotic cell for multi-resolution additive manufacturing. In IEEE International Conference on Robotics and Automation (ICRA), Montreal, Canada, May 2019. [24] Prahar M Bhatt, Ariyan M Kabir, Max Peralta, Hugh A Bruck, and Satyandra K Gupta. A robotic cell for performing sheet lamination-based additive manufacturing. Additive Manu- facturing, 2019. [25] Prahar M Bhatt, Rishi K Malhan, and Satyandra K Gupta. Computational foundations for using three degrees of freedom build platforms to enable supportless extrusion based addi- tive manufacturing. In International Manufacturing Science and Engineering Conference. ASME, 2019. 202 [26] A. M. Kabir, K. N. Kaipa, J. Marvel, and S. K. Gupta. Automated planning for robotic cleaning using multiple setups and oscillatory tool motions. IEEE Transactions on Automa- tion Science and Engineering, 14(3):1364{1377, July 2017. [27] A. M. Kabir, J. D. Langsfeld, S. Shriyam, V. S. Rachakonda, C. Zhuang, K. N. Kaipa, J. Marvel, and S. K. Gupta. Planning algorithms for multi-setup multi-pass robotic clean- ing with oscillatory moving tools. In 2016 IEEE International Conference on Automation Science and Engineering (CASE), pages 751{757, Fort Worth, Texas, USA, Aug 2016. [28] J. D. Langsfeld, A. M. Kabir, K. N. Kaipa, and S. K. Gupta. Integration of planning and deformation model estimation for robotic cleaning of elastically deformable objects. IEEE Robotics and Automation Letters, 3(1):352{359, Jan 2018. [29] P. Rajendran, T. Moscicki, J. Wampler, and S. K. Gupta K. D. von Ellenrieder. Wave-aware trajectory planning for unmanned surface vehicles operating in congested environments. In IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), Philadel- phia, PA, U.S.A., August 2018. [30] Pradeep Rajendran, Shantanu Thakar, and Satyandra Gupta. User-guided path planning for redundant manipulators in highly constrained work environments. In IEEE International Conference on Automation Science and Engineering (CASE), Vancouver, Canada, August 2019. [31] Pradeep Rajendran, Shantanu Thakar, Ariyan Kabir, Brual Shah, and S. K. Gupta. Context-dependent search for generating paths for redundant manipulators in cluttered en- vironments. In IEEE International Conference on Intelligent Robots and Systems (IROS), Macau, China, November 2019. [32] Sarah Al-Hussaini, Jason M. Gregory, and Satyandra K. Gupta. A policy synthesis-based framework for robot rescue decision-making in multi-robot exploration of disaster sites. In IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR), Philadel- phia, PA, U.S.A., August 2018. [33] Sarah Al-Hussaini, Jason M. Gregory, and Satyandra K. Gupta. Generation of context- dependent policies for robot rescue decision-making in multi-robot teams. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, October 2018. [34] Lynne E Parker. Multiple mobile robot systems. Springer Handbook of Robotics, pages 921{941, 2008. [35] Yi Guo, Lynne E Parker, and Raj Madhavan. Towards collaborative robots for infras- tructure security applications. In Proceedings of International Symposium on Collaborative Technologies and Systems, volume 2004, pages 235{240, 2004. [36] HR Everett, RT Laird, DM Carroll, GA Gilbreath, and TA Heath-Pastore. Multiple re- source host architecture (mrha) for the mobile detection assessment response system (mdars) revision a. Technical report, Space And Naval Warfare Systems Center San Diego CA, 2000. [37] Lynne E Parker. Alliance: An architecture for fault tolerant multirobot cooperation. IEEE transactions on robotics and automation, 14(2):220{240, 1998. [38] Cai Luo, Andre Possani Espinosa, Danu Pranantha, and Alessandro De Gloria. Multi- robot search and rescue team. In Proceedings of IEEE International Symposium on Safety, Security, and Rescue Robotics, pages 296{301, 2011. 203 [39] Rachid Alami, Sara Fleury, Matthieu Herrb, F elix Ingrand, and Fr ed eric Robert. Multi- robot cooperation in the martha project. IEEE Robotics & Automation Magazine, 5(1):36{ 47, 1998. [40] Raaele Limosani, Raele Esposito, Alessandro Manzi, Giancarlo Teti, Filippo Cavallo, and Paolo Dario. Robotic delivery service in combined outdoor{indoor environments: technical analysis and user evaluation. Robotics and Autonomous Systems, 103:56{67, 2018. [41] Ashley Stroupe, Avi Okon, Matthew Robinson, Terry Huntsberger, Hrand Aghazarian, and Eric Baumgartner. Sustainable cooperative robotic technologies for human and robotic outpost infrastructure construction and maintenance. Autonomous Robots, 20(2):113{123, 2006. [42] Ali Narenji Sheshkalani and Ramtin Khosravi. Verication of visibility-based properties on multiple moving robots in an environment with obstacles. International Journal of Advanced Robotic Systems, 15(4):1729881418786657, 2018. [43] Alan FT Wineld, Jin Sa, Mari-Carmen Fern andez-Gago, Clare Dixon, and Michael Fisher. On formal specication of emergent behaviours in swarm robotic systems. International Journal of Advanced Robotic Systems, 2(4):39, 2005. [44] Matthew L Bolton, Ellen J Bass, and Radu I Siminiceanu. Using formal verication to evaluate human-automation interaction: A review. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 43(3):488{503, 2013. [45] Yash Vardhan Pant, Houssam Abbas, Rhudii A Quaye, and Rahul Mangharam. Fly-by- logic: control of multi-drone eets with temporal logic objectives. In Proceedings of the 9th ACM/IEEE International Conference on Cyber-Physical Systems, pages 186{197. IEEE Press, 2018. [46] Zhiyu Liu, Bo Wu, Jin Dai, and Hai Lin. Distributed communication-aware motion plan- ning for multi-agent systems from STL and SpaTeL specications. In Proceedings of IEEE Conference on Decision and Control (CDC), pages 4452{4457, 2017. [47] Zhiyu Liu, Jin Dai, Bo Wu, and Hai Lin. Communication-aware motion planning for multi- agent systems from signal temporal logic specications. In Proceedings of IEEE American Control Conference (ACC), 2017, pages 2516{2521, 2017. [48] Alexandros Nikou, Dimitris Boskos, Jana Tumova, and Dimos V Dimarogonas. Cooper- ative planning for coupled multi-agent systems under timed temporal specications. In Proceedings of IEEE American Control Conference (ACC), pages 1847{1852, 2017. [49] Yuchen Zhou, Dipankar Maity, and John S Baras. Optimal mission planner with timed temporal logic constraints. In Proceedings of IEEE European Control Conference (ECC), pages 759{764, 2015. [50] Joseph Kim, Christopher J Banks, and Julie A Shah. Collaborative planning with encoding of users' high-level strategies. In Proceedings of AAAI Conference on Articial Intelligence, pages 955{962, 2017. [51] Douglas C MacKenzie, Ronald C Arkin, and Jonathan M Cameron. Multiagent mission specication and execution. In Robot colonies, pages 29{52. Springer, 1997. [52] Alessio Lomuscio and Franco Raimondi. Model checking knowledge, strategies, and games in multi-agent systems. In Proceedings of the fth international joint conference on Au- tonomous agents and multiagent systems, pages 161{168. ACM, 2006. 204 [53] Yunus Emre Sahin, Petter Nilsson, and Necmiye Ozay. Synchronous and asynchronous multi-agent coordination with cLTL+ constraints. In Proceedings of IEEE Conference on Decision and Control (CDC), pages 335{342, 2017. [54] Jonathan A DeCastro, Javier Alonso-Mora, Vasumathi Raman, Daniela Rus, and Hadas Kress-Gazit. Collision-free reactive mission and motion planning for multi-robot systems. In Robotics Research, pages 459{476. Springer, 2018. [55] Lars Lindemann, Christos K Verginis, and Dimos V Dimarogonas. Prescribed performance control for signal temporal logic specications. arXiv preprint arXiv:1703.07094, 2017. [56] Yu Ke, Antonios Tsourdos, Brian White, and Peter Silson. Dynamic mission planning of multiple unmanned autonomous platforms for sensing and autonomous urban reconnais- sance. In Proceedings of AIAA Guidance, Navigation, and Control Conference, page 6216, 2009. [57] Gordon Fraser, Franz Wotawa, and Paul E Ammann. Testing with model checkers: a survey. Software Testing, Verication and Reliability, 19(3):215{261, 2009. [58] Georgios E Fainekos, Antoine Girard, Hadas Kress-Gazit, and George J Pappas. Temporal logic motion planning for dynamic robots. Automatica, 45(2):343{352, 2009. [59] Morteza Lahijanian, Joseph Wasniewski, Sean B Andersson, and Calin Belta. Motion plan- ning and control from temporal logic specications with probabilistic satisfaction guaran- tees. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pages 3227{3232, 2010. [60] Cameron Finucane, Gangyuan Jing, and Hadas Kress-Gazit. LTLMoP: Experimenting with language, temporal logic and robot control. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1988{1993, 2010. [61] Savas Konur, Clare Dixon, and Michael Fisher. Analysing robot swarm behaviour via probabilistic model checking. Robotics and Autonomous Systems, 60(2):199{213, 2012. [62] Fujio Miyawaki, Ken Masamune, Satoshi Suzuki, Kitaro Yoshimitsu, and Juri Vain. Scrub nurse robot system-intraoperative motion analysis of a scrub nurse and timed-automata- based model for surgery. IEEE Transactions on Industrial Electronics, 52(5):1227{1235, 2005. [63] Thao Nguyen Van, Nugroho Fredivianus, Huu Tam Tran, Kurt Geihs, and Thi Thanh Binh Huynh. Formal verication of ALICA multi-agent plans using model checking. In Proceed- ings of the Ninth International Symposium on Information and Communication Technology, pages 351{358. ACM, 2018. [64] Rong Gu, Raluca Marinescu, Cristina Seceleanu, and Kristina Lundqvist. Formal veri- cation of an autonomous wheel loader by model checking. In Proceedings of IEEE/ACM International FME Workshop on Formal Methods in Software Engineering (FormaliSE), pages 74{83, 2018. [65] Sertac Karaman and Emilio Frazzoli. Linear temporal logic vehicle routing with applications to multi-UAV mission planning. International Journal of Robust and Nonlinear Control, 21(12):1372{1395, 2011. [66] Meng Guo and Dimos V Dimarogonas. Task and motion coordination for heterogeneous multiagent systems with loosely coupled local tasks. IEEE Transactions on Automation Science and Engineering, 14(2):797{808, 2017. 205 [67] Philipp Schillinger, Mathias B urger, and Dimos V Dimarogonas. Simultaneous task allo- cation and planning for temporal logic goals in heterogeneous multi-robot systems. The International Journal of Robotics Research, page 0278364918774135, 2017. [68] Yasser Shoukry, Pierluigi Nuzzo, Ayca Balkan, Indranil Saha, Alberto L Sangiovanni- Vincentelli, Sanjit A Seshia, George J Pappas, and Paulo Tabuada. Linear temporal logic motion planning for teams of underactuated robots using satisability modulo convex pro- gramming. In Proceedings of IEEE Conference on Decision and Control (CDC), pages 1132{1137, 2017. [69] Maryam Kamali, Louise A Dennis, Owen McAree, Michael Fisher, and Sandor M Veres. Formal verication of autonomous vehicle platooning. Science of Computer Programming, 148:88{106, 2017. [70] Mathilde Machin, J er emie Guiochet, H elene Waeselynck, Jean-Paul Blanquart, Matthieu Roy, and Lola Masson. SMOF: A safety monitoring framework for autonomous systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 48(5):702{715, 2018. [71] Liangguo Liu, Jun Peng, Xiaoyong Zhang, Rui Zhang, Bin Chen, Kai Gao, and Yingze Yang. Reactive behavioral strategy for unmanned ground vehicle under liner temporal logic specications. In Proceedings of IEEE International Conference on Big Data and Smart Computing (BigComp), pages 133{140, 2018. [72] Ankur M Mehta, Joseph DelPreto, Kai Weng Wong, Scott Hamill, Hadas Kress-Gazit, and Daniela Rus. Robot creation from functional specications. In Robotics Research, pages 631{648. Springer, 2018. [73] Jiyoung Choi, Seungkeun Kim, and Antonios Tsourdos. Verication of heterogeneous multi- agent system using mcmas. International Journal of Systems Science, 46(4):634{651, 2015. [74] Laura Humphrey. Model checking UAV mission plans. In Proceedings of AIAA Modeling and Simulation Technologies Conference, page 4723, 2012. [75] Clayton Rothwell, Alexa Eggert, Michael J Patzek, George Bearden, Gloria L Calhoun, and Laura R Humphrey. Human-computer interface concepts for veriable mission speci- cation, planning, and management. In Proceedings of AIAA Infotech@ Aerospace (I@ A) Conference, page 4804, 2013. [76] Javier Alonso-Mora, Jonathan A DeCastro, Vasumathi Raman, Daniela Rus, and Hadas Kress-Gazit. Reactive mission and motion planning with deadlock resolution avoiding dy- namic obstacles. Autonomous Robots, 42(4):801{824, 2018. [77] Hasan Sinan Bank, Sandeep D'souza, and Aditya Rasam. Temporal logic (TL)-based au- tonomy for smart manufacturing systems. Procedia Manufacturing, 26:1221{1229, 2018. [78] Philipp Schillinger, Mathias B urger, and Dimos V Dimarogonas. Multi-objective search for optimal multi-robot planning with nite LTL specications and resource constraints. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pages 768{774, 2017. [79] Matthew R Maly, Morteza Lahijanian, Lydia E Kavraki, Hadas Kress-Gazit, and Moshe Y Vardi. Iterative temporal motion planning for hybrid systems in partially unknown envi- ronments. In Proceedings of the international conference on Hybrid systems: computation and control, pages 353{362. ACM, 2013. 206 [80] Indranil Saha, Rattanachai Ramaithitima, Vijay Kumar, George J Pappas, and Sanjit A Seshia. Automated composition of motion primitives for multi-robot systems from safe LTL specications. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1525{1532, 2014. [81] Meng Guo, Karl H Johansson, and Dimos V Dimarogonas. Motion and action planning un- der LTL specications using navigation functions and action description language. In Pro- ceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 240{245, 2013. [82] Yi Li, Jing Sun, Jin Song Dong, Yang Liu, and Jun Sun. Planning as model checking tasks. In Proceedings of IEEE Software Engineering Workshop, pages 177{186, 2012. [83] Sebastian Thrun, Wolfram Burgard, and Dieter Fox. Probabilistic robotics. MIT press, 2005. [84] Dong Liu, Ming Cong, Qiang Zou, and Yu Du. A biological-inspired episodic cognitive map building framework for mobile robot navigation. International Journal of Advanced Robotic Systems, 14(3):1729881417705922, 2017. [85] Shengbing Jiang and Ratnesh Kumar. Diagnosis of repeated failures for discrete event systems with linear-time temporal-logic specications. IEEE Transactions on Automation Science and Engineering, 3(1):47{59, 2006. [86] Krishnendu Chatterjee, Martin Chmel k, Raghav Gupta, and Ayush Kanodia. Qualitative analysis of pomdps with temporal logic specications for robotics applications. In Proceed- ings of IEEE International Conference on Robotics and Automation (ICRA), pages 325{330, 2015. [87] Bruno Lacerda, David Parker, and Nick Hawes. Optimal and dynamic planning for Markov decision processes with co-safe LTL specications. In Proceedings of IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS), pages 1511{1516, 2014. [88] Igor Cizelj, Xu Chu Ding, Morteza Lahijanian, Alessandro Pinto, and Calin Belta. Proba- bilistically safe vehicle control in a hostile environment. In Presented at World Congress of the International Federation of Automatic Control (IFAC), 2011. [89] Xuchu Ding, Stephen L Smith, Calin Belta, and Daniela Rus. Optimal control of markov decision processes with linear temporal logic constraints. IEEE Transactions on Automatic Control, 59(5):1244{1257, 2014. [90] Fatma Faruq, Bruno Lacerda, Nick Hawes, and David Parker. Simultaneous task allocation and planning under uncertainty. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018. [91] Lu Feng, Clemens Wiltsche, Laura Humphrey, and Ufuk Topcu. Synthesis of human-in-the- loop control protocols for autonomous systems. IEEE Transactions on Automation Science and Engineering, 13(2):450{462, 2016. [92] Marius Kloetzer and Cristian Mahulea. LTL-based planning in environments with probabilistic observations. IEEE Transactions on Automation Science and Engineering, 12(4):1407{1420, 2015. [93] Yushan Chen, Jana Tumov a, and Calin Belta. LTL robot motion control based on automata learning of environmental dynamics. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pages 5177{5182, 2012. 207 [94] Xuchu Dennis Ding, Brendan Englot, Alessandro Pinto, Alberto Speranzon, and Amit Surana. Hierarchical multi-objective planning: From mission specications to contingency management. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pages 3735{3742, 2014. [95] Tichakorn Wongpiromsarn and Richard M Murray. Distributed mission and contingency management for the DARPA urban challenge. In Proceedings of International Workshop on Intelligent Vehicle Control Systems (IVCS), 2008. [96] Jerry L Franke, Stephen M Jameson, Rosemary D Paradis, and Robert J Szczerba. Hierar- chical contingency management system for mission planners, December 13 2011. US Patent 8,078,319. [97] Anders Lyhne Christensen, Rehan O'Grady, Mauro Birattari, and Marco Dorigo. Fault detection in autonomous robots based on fault injection and learning. Autonomous Robots, 24(1):49{67, 2008. [98] Steve McGuire, P Michael Furlong, Christoer Heckman, Simon Julier, Daniel Szar, and Nisar Ahmed. Failure is not an option: Policy learning for adaptive recovery in space operations. IEEE Robotics and Automation Letters, 3(3):1639{1646, 2018. [99] Matthew T Long, Robin R Murphy, and Lynne E Parker. Distributed multi-agent diagnosis and recovery from sensor failures. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), volume 3, pages 2506{2513, 2003. [100] Patrick Ulam and Ronald C Arkin. When good communication go bad: communications re- covery for multi-robot teams. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), volume 4, pages 3727{3734, 2004. [101] Siddharth Mayya and Magnus Egerstedt. Safe open-loop strategies for handling intermittent communications in multi-robot systems. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pages 5818{5823, 2017. [102] Liang Tang, Gregory J Kacprzynski, Kai Goebel, Abhinav Saxena, Bhaskar Saha, and George Vachtsevanos. Prognostics-enhanced automated contingency management for ad- vanced autonomous systems. In Proceedings of International Conference on Prognostics and Health Management (PHM), pages 1{9, 2008. [103] Liang Tang, Bin Zhang, Jonathan DeCastro, and Eric Hettler. An integrated health and contingency management case study on an autonomous ground robot. In Proceedings of IEEE International Conference on Control and Automation (ICCA), pages 584{589, 2011. [104] Dong-Hyun Lee, Sheir Afgen Zaheer, and Jong-Hwan Kim. A resource-oriented, decentral- ized auction algorithm for multirobot task allocation. IEEE Transactions on Automation Science and Engineering, 12(4):1469{1481, 2015. [105] Changjoo Nam and Dylan A Shell. Assignment algorithms for modeling resource contention in multirobot task allocation. IEEE Transactions on Automation Science and Engineering, 12(3):889{900, 2015. [106] Lingzhi Luo, Nilanjan Chakraborty, and Katia Sycara. Distributed algorithms for multi- robot task assignment with task deadline constraints. IEEE Transactions on Automation Science and Engineering, 12(3):876{888, 2015. 208 [107] Amanda Whitbrook, Qinggang Meng, and Paul WH Chung. Reliable, distributed scheduling and rescheduling for time-critical, multiagent systems. IEEE Transactions on Automation Science and Engineering, 15(2):732{747, 2018. [108] Shudong Liu, Huayu Wu, Shili Xiang, and Xiaoli Li. Mobile robot scheduling with multiple trips and time windows. In Proceedings of International Conference on Advanced Data Mining and Applications, pages 608{620, 2017. [109] Chihyun Jung and Tae-Eog Lee. An ecient mixed integer programming model based on timed petri nets for diverse complex cluster tool scheduling problems. IEEE Transactions on Semiconductor Manufacturing, 25(2):186{199, 2012. [110] Mary Koes, Illah Nourbakhsh, and Katia Sycara. Constraint optimization coordination architecture for search and rescue robotics. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pages 3977{3982, 2006. [111] Eduardo Feo Flushing, Luca M Gambardella, and Gianni A Di Caro. Simultaneous task allocation, data routing, and transmission scheduling in mobile multi-robot teams. In Pro- ceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1861{1868, 2017. [112] Chongjie Zhang and Julie A Shah. Co-optimizating multi-agent placement with task as- signment and scheduling. In Proceedings of International Joint Conference on Articial Intelligence (IJCAI), pages 3308{3314, 2016. [113] Xavier Lorca, Charles Prud'Homme, Aur elien Questel, and Beno^ t Rottembourg. Using constraint programming for the urban transit crew rescheduling problem. In Proceedings of International Conference on Principles and Practice of Constraint Programming, pages 636{649, 2016. [114] Ridvan Gedik, Emre Kirac, Ashlea Bennet Milburn, and Chase Rainwater. A constraint programming approach for the team orienteering problem with time windows. Computers & Industrial Engineering, 107:178{195, 2017. [115] Kyle EC Booth, Goldie Nejat, and J Christopher Beck. A constraint programming ap- proach to multi-robot task allocation and scheduling in retirement homes. In Proceedings of International Conference on Principles and Practice of Constraint Programming, pages 539{555, 2016. [116] Juan M Novas and Gabriela P Henning. Integrated scheduling of resource-constrained exible manufacturing systems using constraint programming. Expert Systems with Appli- cations, 41(5):2286{2299, 2014. [117] Moacir Godinho Filho, Clarissa Fullin Barco, and Roberto Fernandes Tavares Neto. Using genetic algorithms to solve scheduling problems on exible manufacturing systems (FMS): a literature survey, classication and analysis. Flexible Services and Manufacturing Journal, 26(3):408{431, 2014. [118] J Jerald, Prabaharan Asokan, G Prabaharan, and R Saravanan. Scheduling optimisation of exible manufacturing systems using particle swarm optimisation algorithm. The Interna- tional Journal of Advanced Manufacturing Technology, 25(9-10):964{971, 2005. [119] S Meysam Mousavi and Reza Tavakkoli-Moghaddam. A hybrid simulated annealing algo- rithm for location and routing scheduling problems with cross-docking in the supply chain. Journal of Manufacturing Systems, 32(2):335{347, 2013. 209 [120] Torbjrn S Dahl, Maja Matari c, and Gaurav S Sukhatme. Multi-robot task allocation through vacancy chain scheduling. Robotics and Autonomous Systems, 57(6-7):674{687, 2009. [121] Shiyuan Jin, Guy Schiavone, and Damla Turgut. A performance study of multiprocessor task scheduling algorithms. The Journal of Supercomputing, 43(1):77{97, 2008. [122] Haluk Topcuoglu, Salim Hariri, and Min-you Wu. Performance-eective and low-complexity task scheduling for heterogeneous computing. IEEE transactions on parallel and distributed systems, 13(3):260{274, 2002. [123] Ernesto Nunes, Mitchell McIntire, and Maria Gini. Decentralized multi-robot allocation of tasks with temporal and precedence constraints. Advanced Robotics, 31(22):1193{1207, 2017. [124] Ernesto Nunes, Marie Manner, Hakim Mitiche, and Maria Gini. A taxonomy for task allocation problems with temporal and ordering constraints. Robotics and Autonomous Systems, 90:55{70, 2017. [125] Kenli Li, Xiaoyong Tang, Bharadwaj Veeravalli, and Keqin Li. Scheduling precedence con- strained stochastic tasks on heterogeneous cluster systems. IEEE Transactions on comput- ers, 64(1):191{204, 2015. [126] Binghai Zhou and Jiahui Xu. An adaptive SVM-based real-time scheduling mechanism and simulation for multiple-load carriers in automobile assembly lines. International Journal of Modeling, Simulation, and Scientic Computing, 8(04):1750048, 2017. [127] Jerey R Peters and Luca F Bertuccelli. Robust scheduling strategies for collaborative human-UAV missions. In Proceedings of American Control Conference (ACC), 2016, pages 5255{5262, 2016. [128] Matthew C Gombolay, Ronald Wilcox, and Julie A Shah. Fast scheduling of multi-robot teams with temporospatial constraints. In Proceedings of Robotics: Science and Systems, 2013. [129] Lantao Liu and Dylan A Shell. Optimal market-based multi-robot task allocation via strate- gic pricing. In Proceedings of Robotics: Science and Systems, volume 9, pages 33{40, 2013. [130] Drew Wicke, David Freelan, and Sean Luke. Bounty hunters and multiagent task alloca- tion. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems, pages 387{394, 2015. [131] Michael Otte, Michael Kuhlman, and Donald Sofge. Multi-robot task allocation with auc- tions in harsh communication environments. In Proceedings of IEEE International Sympo- sium on Multi-Robot and Multi-Agent Systems (MRS), pages 32{39, 2017. [132] Han-Lim Choi, Luc Brunet, and Jonathan P How. Consensus-based decentralized auctions for robust task allocation. IEEE transactions on robotics, 25(4):912{926, 2009. [133] Wanqing Zhao, Qinggang Meng, and Paul WH Chung. A heuristic distributed task alloca- tion method for multivehicle multitask problems and its application to search and rescue scenario. IEEE transactions on cybernetics, 46(4):902{915, 2016. [134] Gautham P Das, Thomas M McGinnity, Sonya A Coleman, and Laxmidhar Behera. A dis- tributed task allocation algorithm for a multi-robot system in healthcare facilities. Journal of Intelligent & Robotic Systems, 80(1):33{58, 2015. 210 [135] Carla Mouradian, Jagruti Sahoo, Roch H Glitho, Monique J Morrow, and Paul A Polakos. A coalition formation algorithm for multi-robot task allocation in large-scale natural disas- ters. In Proceedings of IEEE International Wireless Communications and Mobile Computing Conference (IWCMC), pages 1909{1914, 2017. [136] Anton Dukeman and Julie A Adams. Hybrid mission planning with coalition formation. Autonomous Agents and Multi-Agent Systems, 31(6):1424{1466, 2017. [137] Bing Xie, Shaofei Chen, Jing Chen, and LinCheng Shen. A mutual-selecting market-based mechanism for dynamic coalition formation. International Journal of Advanced Robotic Systems, 15(1):1729881418755840, 2018. [138] Onn Shehory and Sarit Kraus. Methods for task allocation via agent coalition formation. Articial intelligence, 101(1-2):165{200, 1998. [139] Jos e Guerrero and Gabriel Oliver. Multi-robot coalition formation in real-time scenarios. Robotics and Autonomous Systems, 60(10):1295{1307, 2012. [140] Tyler Gunn and John Anderson. Dynamic heterogeneous team formation for robotic urban search and rescue. Journal of Computer and System Sciences, 81(3):553{567, 2015. [141] Zhen Yang Lim, SG Ponnambalam, and Kazuhiro Izui. Multi-objective hybrid algorithms for layout optimization in multi-robot cellular manufacturing systems. Knowledge-Based Systems, 120:87{98, 2017. [142] Chao Lu, Liang Gao, Xinyu Li, Quanke Pan, and Qi Wang. Energy-ecient permutation ow shop scheduling problem using a hybrid multi-objective backtracking search algorithm. Journal of cleaner production, 144:228{238, 2017. [143] Elkin Castro and Sanja Petrovic. Combined mathematical programming and heuristics for a radiotherapy pre-treatment scheduling problem. Journal of Scheduling, 15(3):333{346, 2012. [144] Wei Tan and Behrokh Khoshnevis. Integration of process planning and scheduling - a review. Journal of Intelligent Manufacturing, 11(1):51{63, 2000. [145] Vipul Jain and Ignacio E Grossmann. Algorithms for hybrid MILP/CP models for a class of optimization problems. INFORMS Journal on computing, 13(4):258{276, 2001. [146] Haitao Li and Keith Womer. Scheduling projects with multi-skilled personnel by a hybrid MILP/CP benders decomposition algorithm. Journal of Scheduling, 12(3):281, 2009. [147] Qiao Zhang, Herv e Manier, and M-A Manier. A genetic algorithm with tabu search proce- dure for exible job shop scheduling with transportation constraints and bounded processing times. Computers & Operations Research, 39(7):1713{1723, 2012. [148] Stefano Di Alesio, Lionel Briand, Shiva Nejati, and Arnaud Gotlieb. Combining genetic algorithms and constraint programming to support stress testing of task deadlines. ACM Transactions on Software Engineering & Methodology, 25(1), 2015. [149] Xinyan Ou, Qing Chang, Nilanjan Chakraborty, and Junfeng Wang. Gantry scheduling for multi-gantry production system by online task allocation method. IEEE Robotics and Automation Letters, 2(4):1848{1855, 2017. [150] Amanda Whitbrook, Qinggang Meng, and Paul WH Chung. A novel distributed scheduling algorithm for time-critical multi-agent systems. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6451{6458, 2015. 211 [151] Pedro M Shiroma and Mario FM Campos. CoMutaR: A framework for multi-robot co- ordination and task allocation. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4817{4824. IEEE, 2009. [152] Matthew Turpin, Nathan Michael, and Vijay Kumar. CAPT: Concurrent assignment and planning of trajectories for multiple robots. The International Journal of Robotics Research, 33(1):98{112, 2014. [153] Ernesto Nunes and Maria L Gini. Multi-robot auctions for allocation of tasks with temporal constraints. In Proceedings of AAAI Conference on Articial Intelligence, pages 2110{2116, 2015. [154] Abderraouf Maoudj, Brahim Bouzouia, Abdelfetah Hentout, and Redouane Toumi. Multi- agent approach for task allocation and scheduling in cooperative heterogeneous multi-robot team: Simulation results. In Proceedings of IEEE International Conference on Industrial Informatics (INDIN), pages 179{184, 2015. [155] Anshul Kanakia, Behrouz Touri, and Nikolaus Correll. Modeling multi-robot task allocation with limited information as global game. Swarm Intelligence, 10(2):147{160, 2016. [156] Smriti Chopra and Magnus Egerstedt. Heterogeneous multi-robot routing. In Proceedings of IEEE American Control Conference (ACC), pages 5390{5395, 2014. [157] Yueyue Deng, Pierre-Philippe J Beaujean, Edgar An, and Edward Carlson. Task allocation and path planning for collaborative autonomous underwater vehicles operating through an underwater acoustic network. Journal of Robotics, 2013, 2013. [158] Alejandro R Mosteo, Luis Montano, and Michail G Lagoudakis. Multi-robot routing under limited communication range. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pages 1531{1536, 2008. [159] Bilal Y Kaddouh, William J Crowther, and Peter Hollingsworth. Dynamic resource allo- cation for ecient sharing of services from heterogeneous autonomous vehicles. Journal of Aerospace Information Systems, pages 450{474, 2016. [160] Jacopo Ban, J er^ ome Guzzi, Francesco Amigoni, Eduardo Feo Flushing, Alessandro Giusti, Luca Gambardella, and Gianni A Di Caro. An integer linear programming model for fair multitarget tracking in cooperative multirobot systems. Autonomous Robots, pages 1{16, 2018. [161] Chen K Tham and Richard W Prager. A modular q-learning architecture for manipulator task decomposition. In Proceedings of International Conference on Machine Learning, pages 309{317. Elsevier, 1994. [162] Naiqi Wu, Ning Mao, and Yanming Qian. An approach to partner selection in agile manu- facturing. Journal of Intelligent Manufacturing, 10(6):519{529, 1999. [163] Nabil NZ Gindy and Tsvetan M Ratchev. Cellular decomposition of manufacturing facilities using resource elements. Integrated Manufacturing Systems, 8(4):215{222, 1997. [164] Shimon Whiteson, Nate Kohl, Risto Miikkulainen, and Peter Stone. Evolving soccer keep- away players through task decomposition. Machine Learning, 59(1-2):5{30, 2005. [165] Peter Stone and Manuela Veloso. Task decomposition, dynamic role assignment, and low-bandwidth communication for real-time strategic teamwork. Articial Intelligence, 110(2):241{273, 1999. 212 [166] Chau Su and Yuan F Zheng. Task decomposition for a multilimbed robot to work in reachable but unorientable space. IEEE transactions on robotics and automation, 7(6):759{ 770, 1991. [167] Jerey Too Chuan Tan, Feng Duan, Ye Zhang, and Tamio Arai. Task decomposition of cell production assembly operation for man-machine collaboration by hta. In Proceedings of IEEE International Conference on Automation and Logistics (ICAL), pages 1066{1071, 2008. [168] Jacob Huckaby and Henrik I Christensen. Toward a knowledge transfer framework for pro- cess abstraction in manufacturing robotics. In ICML Workshop on Theoretically Grounded Transfer Learning, 2013. [169] Susan Hert and Vladimir Lumelsky. Polygon area decomposition for multiple-robot workspace division. International Journal of Computational Geometry & Applications, 8(04):437{466, 1998. [170] Hannah Bast and Susan Hert. The area partitioning problem. In Proceedings of Canadian Conference on Computational Geometry, 2000. [171] Jyh-Ming Lien and Nancy M Amato. Approximate convex decomposition of polygons. Computational Geometry, 35(1-2):100{123, 2006. [172] Marco Pavone, Alessandro Arsie, Emilio Frazzoli, and Francesco Bullo. Distributed algo- rithms for environment partitioning in mobile robotic networks. IEEE Transactions on Automatic Control, 56(8):1834{1848, 2011. [173] Praneel Chand and Dale A Carnegie. Mapping and exploration in a hierarchical heteroge- neous multi-robot system using limited capability robots. Robotics and autonomous Sys- tems, 61(6):565{579, 2013. [174] Vittorio A Ziparo, Alexander Kleiner, Bernhard Nebel, and Daniele Nardi. Rd-based exploration for large robot teams. In Proceedings of IEEE International Conference on Robotics and Automation, pages 4606{4613, 2007. [175] Arnoud Visser and Bayu A Slamet. Including communication success in the estimation of information gain for multi-robot exploration. In Proceedings of IEEE International Sym- posium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless (WiOPT), pages 680{687, 2008. [176] La etitia Matignon, Laurent Jeanpierre, and Abdel-Illah Mouaddib. Distributed value func- tions for multi-robot exploration: a position paper. In Multi-Agent Sequential Decision Making in Uncertain Multi-Agent Domain (MSDM)(AAMAS workshop), 2012. [177] Jing Yuan, Yalou Huang, Tong Tao, and Fengchi Sun. A cooperative approach for multi- robot area exploration. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1390{1395, 2010. [178] Kian Hsiang Low, Georey J Gordon, John M Dolan, and Pradeep Khosla. Adaptive sampling for multi-robot wide-area exploration. In Proceedings of IEEE International Con- ference on Robotics and Automation, pages 755{760, 2007. [179] Kian Hsiang Low, John M Dolan, and Pradeep Khosla. Adaptive multi-robot wide-area exploration and mapping. In Proceedings of International Joint Conference on Autonomous Agents and Multiagent Systems, pages 23{30, 2008. 213 [180] Mazda Ahmadi and Peter Stone. A multi-robot system for continuous area sweeping tasks. In Proceedings of IEEE International Conference on Robotics and Automation, pages 1724{ 1729, 2006. [181] Jie Zhao, Xiangguo Su, and Jihong Yan. A novel strategy for distributed multi-robot coordination in area exploration. In Proceedings of International Conference on Measuring Technology and Mechatronics Automation, volume 2, pages 24{27, 2009. [182] Nicola Basilico and Francesco Amigoni. Exploration strategies based on multi-criteria deci- sion making for searching environments in rescue operations. Autonomous Robots, 31(4):401, 2011. [183] Agusti Solanas and Miguel Angel Garcia. Coordinated multi-robot exploration through unsupervised clustering of unknown space. In Proceedings of IEEE/RSJ International Con- ference on Intelligent Robots and Systems, pages 717{721, 2004. [184] Kai M Wurm, Cyrill Stachniss, and Wolfram Burgard. Coordinated multi-robot explo- ration using a segmentation of the environment. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1160{1165, 2008. [185] Yuanteng Pei, Matt W Mutka, and Ning Xi. Coordinated multi-robot real-time explo- ration with connectivity and bandwidth awareness. In Proceedings of IEEE International Conference on Robotics and Automation, pages 5460{5465, 2010. [186] Reid Simmons, David Apfelbaum, Wolfram Burgard, Dieter Fox, Mark Moors, Sebastian Thrun, and H akan Younes. Coordination for multi-robot exploration and mapping. In Proceedings of Conference on Innovative Applications of Articial Intelligence, pages 852{ 858, 2000. [187] Arnaud Doniec, Noury Bouraqadi, Michael Defoort, Serge Stinckwich, et al. Distributed constraint reasoning applied to multi-robot exploration. In Proceedings of IEEE Interna- tional Conference on Tools with Articial Intelligence (ICTAI), pages 159{166, 2009. [188] Weihua Sheng, Qingyan Yang, Jindong Tan, and Ning Xi. Distributed multi-robot coordi- nation in area exploration. Robotics and Autonomous Systems, 54(12):945{955, 2006. [189] Jose Vazquez and Chris Malcolm. Distributed multirobot exploration maintaining a mobile network. In Proceedings of IEEE Conference on Intelligent Systems, volume 3, pages 113{ 118, 2004. [190] Yehuda Elmaliach, Noa Agmon, and Gal A Kaminka. Multi-robot area patrol under fre- quency constraints. Annals of Mathematics and Articial Intelligence, 57(3-4):293{320, 2009. [191] Andrew Howard, Maja J Matari c, and Gaurav S Sukhatme. Mobile sensor network deploy- ment using potential elds: A distributed, scalable solution to the area coverage problem. In Proceedings of International Symposium on Distributed Autonomous Robotic Systems (DARS), pages 299{308. 2002. [192] Maxim A Batalin and Gaurav S Sukhatme. Spreading out: A local approach to multi-robot coverage. In Proceedings of International Symposium on Distributed Autonomous Robotic Systems (DARS), pages 373{382. 2002. [193] Ali Marjovi, Jo~ ao Gon calo Nunes, Lino Marques, and An bal de Almeida. Multi-robot exploration and re searching. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 1929{1934, 2009. 214 [194] KS Senthilkumar and KK Bharadwaj. Multi-robot exploration and terrain coverage in an unknown environment. Robotics and Autonomous Systems, 60(1):123{132, 2012. [195] Robert Zlot, Anthony Stentz, M Bernardine Dias, and Scott Thayer. Multi-robot explo- ration controlled by a market economy. In Proceedings of IEEE International Conference on Robotics and Automation, volume 3, pages 3016{3023, 2002. [196] Julian De Hoog, Stephen Cameron, and Arnoud Visser. Role-based autonomous multi- robot exploration. In Proceedings of IEEE Computation World: Future Computing, Service Computation, Cognitive, Adaptive, Content, Patterns, pages 482{487, 2009. [197] Julian De Hoog, Stephen Cameron, Arnoud Visser, et al. Selection of rendezvous points for multi-robot exploration in dynamic environments. In Proceedings of International Confer- ence on Autonomous Agents and Multi-Agent Systems (AAMAS), 2010. [198] Ruofei Ouyang, Kian Hsiang Low, Jie Chen, and Patrick Jaillet. Multi-robot active sens- ing of non-stationary gaussian process-based environmental phenomena. In Proceedings of International Conference on Autonomous Agents and Multi-agent Systems, pages 573{580, 2014. [199] Mahdi Hassan, Dikai Liu, Shoudong Huang, and Gamini Dissanayake. Task oriented area partitioning and allocation for optimal operation of multiple industrial robots in unstruc- tured environments. In Proceedings of International Conference on Control Automation Robotics & Vision (ICARCV), pages 1184{1189, 2014. [200] Markus Jager and Bernhard Nebel. Dynamic decentralized area partitioning for cooper- ating cleaning robots. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), volume 4, pages 3577{3582, 2002. [201] John Gunnar Carlsson. Dividing a territory among several vehicles. INFORMS Journal on Computing, 24(4):565{577, 2012. [202] Juan Camilo Gamboa Higuera and Gregory Dudek. Fair subdivision of multi-robot tasks. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pages 3014{3019, 2013. [203] A. Agarwal, Meng-Hiot Lim, and Meng Joo Er. Proportional partition of holed rectilin- ear region amongst multiple uravs. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pages 1779{1784, 2005. [204] Brian Yamauchi. A frontier-based approach for autonomous exploration. In Proceedings of IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA), pages 146{151, 1997. [205] Weihua Sheng, Qingyan Yang, Song Ci, and Ning Xi. Multi-robot area exploration with limited-range communications. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), volume 2, pages 1414{1419, 2004. [206] Wolfram Burgard, Mark Moors, and Frank Schneider. Collaborative exploration of unknown environments with teams of mobile robots. In Advances in Plan-Based Control of Robotic Agents, pages 52{70. Springer, 2002. [207] Xin Ma, Qin Zhang, and Yibin Li. Genetic algorithm-based multi-robot cooperative ex- ploration. In Proceedings of IEEE International Conference on Control and Automation (ICCA), pages 1018{1023, 2007. 215 [208] Yiheng Wang, Alei Liang, and Haibing Guan. Frontier-based multi-robot map exploration using particle swarm optimization. In Proceedings of IEEE Symposium on Swarm Intelli- gence (SIS), pages 1{6, 2011. [209] Yi Zhou, Kai Xiao, Yiheng Wang, Alei Liang, and Aboul Ella Hassanien. A pso-inspired multi-robot map exploration algorithm using frontier-based strategy. International Journal of System Dynamics Applications (IJSDA), 2(2):1{13, 2013. [210] Yeun-Soo Jung, Kong-Woo Lee, Seong-Yong Lee, Myoung Hwan Choi, and Beom-Hee Lee. An ecient underwater coverage method for multi-auv with sea current disturbances. In- ternational Journal of Control, Automation and Systems, 7(4):615{629, 2009. [211] Marta Kwiatkowska, Gethin Norman, and David Parker. Stochastic model checking. In Proceedings of International School on Formal Methods for the Design of Computer, Com- munication and Software Systems, pages 220{270. Springer, 2007. [212] Shaurya Shriyam and Satyandra K Gupta. Incorporating potential contingency tasks in multi-robot mission planning. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA, Brisbane, Australia, May 21-25), 2018. [213] Alessandro Cimatti, Edmund Clarke, Enrico Giunchiglia, Fausto Giunchiglia, Marco Pistore, Marco Roveri, Roberto Sebastiani, and Armando Tacchella. NuSMV 2: An opensource tool for symbolic model checking. In Proceedings of International Conference on Computer Aided Verication, pages 359{364. Springer, 2002. [214] Shaurya Shriyam and Satyandra K Gupta. Modeling and analysis of subsystem interactions in robotic assembly. In Proceedings of ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, 2019. [215] Abdullah Alsharhan, Ariyan Kabir, Rishi Malhan, Brual Shah, and Satyandra K Gupta. A exible hybrid cell for low production volume assembly. https://www.youtube.com/ watch?v=yqT2XHwkpOo, 2017. [216] Shantanu Thakar, Pradeep Rajendran, Vivek Annem, Ariyan Kabir, and Satyandra K. Gupta. Accounting for part pose estimation uncertainties during trajectory generation for part pick-up using mobile manipulators. In IEEE International Conference on Robotics and Automation (ICRA), Montreal, Canada, May 2019. [217] Shantanu Thakar, Liwei Fang, Brual Shah, and Satyandra K. Gupta. Towards time-optimal trajectory planning for pick-and-transport operation with a mobile manipulator. In 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE), pages 981{987. IEEE, 2018. [218] Vivek Annem, Pradeep Rajendran, Shantanu Thakar, and Satyandra K. Gupta. Towards remote teleoperation of a semi-autonomous mobile manipulator system in machine tending tasks. In ASME Manufacturing Science and Engineering Conference (MSEC), Erie, USA, June 2019. [219] Shantanu Thakar, Ariyan Kabir, Prahar Bhatt, Rishi Malhan, Pradeep Rajendran, Brual Shah, and Satyandra K. Gupta. Task assignment and motion planning for bi-manual mobile manipulation. In IEEE International Conference on Automation Science and Engineering (CASE), Vancouver, Canada, August 2019. [220] Marta Kwiatkowska, Gethin Norman, and David Parker. Prism 4.0: Verication of prob- abilistic real-time systems. In Proceedings of International conference on computer aided verication, pages 585{591. Springer, 2011. 216 [221] Shaurya Shriyam and Satyandra K Gupta. Task assignment and scheduling for mobile robot teams. In Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC-CIE, Quebec City, Quebec, August 26-29), 2018. [222] G Ayorkor Korsah, Anthony Stentz, and M Bernardine Dias. A comprehensive taxonomy for multi-robot task allocation. The International Journal of Robotics Research, 32(12):1495{ 1512, 2013. [223] Brian P Gerkey and Maja J Matari c. A formal analysis and taxonomy of task allocation in multi-robot systems. The International Journal of Robotics Research, 23(9):939{954, 2004. [224] International Maritime Organization. COLREG: Convention on the International Regula- tions for Preventing Collisions at Sea, 1972. International Maritime Organization, 2002. [225] Saralees Nadarajah and Samuel Kotz. Exact distribution of the max/min of two Gaus- sian random variables. IEEE Transactions on very large scale integration (VLSI) systems, 16(2):210{212, 2008. [226] Charles E Clark. The greatest of a nite set of random variables. Operations Research, 9(2):145{162, 1961. [227] Shaurya Shriyam, Brual C Shah, and Satyandra K Gupta. On-line task decomposition for collaborative surveillance of marine environment by a team of unmanned surface vehi- cles. In Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC-CIE, Cleveland, Ohio, August 6-9), 2017. [228] Shaurya Shriyam, Brual C Shah, and Satyandra K Gupta. Decomposition of collaborative surveillance tasks for execution in marine environments by a team of unmanned surface vehicles. Journal of Mechanisms and Robotics, 10(2):025007, 2018. [229] Brual C Shah, Petr Svec, Ivan R Bertaska, Armando J Sinisterra, Wilhelm Klinger, Karl von Ellenrieder, Manhar Dhanak, and Satyandra K Gupta. Resolution-adaptive risk-aware trajectory planning for surface vehicles operating in congested civilian trac. Autonomous Robots, 40(7):1139{1163, 2016. [230] Petr Svec, Atul Thakur, Eric Raboin, Brual C Shah, and Satyandra K Gupta. Target following with motion prediction for unmanned surface vehicle operating in cluttered envi- ronments. Autonomous Robots, 36(4):383{405, 2014. [231] Brual C Shah and Satyandra K Gupta. Speeding up A* search on visibility graphs dened over quadtrees to enable long distance path planning for unmanned surface vehicles. In Proceedings of International Conference on Automated Planning and Scheduling, pages 527{ 535, 2016. [232] Eric Raboin, Petr Svec, Dana S Nau, and Satyandra K Gupta. Model-predictive asset guard- ing by team of autonomous surface vehicles in environment with civilian boats. Autonomous Robots, 38(3):261{282, 2015. [233] Stuart Russell, Peter Norvig, and Articial Intelligence. A modern approach. Articial Intelligence. Prentice-Hall, Egnlewood Clis, 25:27, 1995. [234] Edsger W Dijkstra. A note on two problems in connexion with graphs. Numerische math- ematik, 1(1):269{271, 1959. 217 [235] J. Kennedy and R. Eberhart. Particle swarm optimization. In Proceedings of IEEE Inter- national Conference on Neural Networks, volume 4, pages 1942{1948, 1995. [236] MATLAB. Version 9.5 (R2018b). The MathWorks Inc., Natick, Massachusetts, 2018. 218
Abstract (if available)
Abstract
In order to be deployed in real-world applications, autonomous multi-robot systems need to be able to handle contingency events. Many multi-robot missions are time-critical, so using overly conservative methods to handle contingencies will not be acceptable in such applications. Robots themselves may be prone to unreliable task execution and may require human assistance for handling such contingencies, which must be properly planned and coordinated. This dissertation models the handling of contingency events in the form of execution of contingency tasks. When an unexpected event occurs that adversely impacts the mission workflow, then the multi-robot system, with or without the guidance of human operators, must perform contingency tasks to ensure the safety and completion of the mission. Contingency tasks can arise due to robot failure occurring due to battery failure or communication lapse. Contingency tasks may also occur explicitly as an additional task that becomes a bottleneck for nominal mission execution. For example, if an autonomous vehicle were to run out of fuel, the vehicle must be taken to a filling center to refuel it. Additional tasks that may be required for mission completion are referred to as contingency tasks. Contingency tasks may also arise due to uncertain operating conditions or bad information about operating conditions. The general approach to handling contingencies in robotics has been to design failure recovery protocols which kick in when a specific contingency occurs. However, such a reactive approach does not work well in time-critical missions. ❧ This dissertation presents the computational foundations for proactively handling contingencies encountered during multi-robot missions. A formal model checking framework is developed to evaluate the adequacy of contingency resolution strategies that may be encountered during multi-robot missions. In this dissertation, the developed framework was successfully applied to an assembly line setup and a ship escorting mission which used unmanned surface vehicles, and the performance of the strategies designed to handle different contingencies was evaluated. This dissertation also presents a proactive approach for incorporating contingency tasks in nominal mission planning. It was found through simulation experiments that the proactive approach out-performed the reactive and conservative approaches. Finally, the dissertation proposes a method for task decomposition required for completing collaborative surveillance tasks assigned to multiple robots under the conditions of uncertain information and varying robot availability. It was found that the developed methods for task decomposition were robust against these uncertainties and were able to successfully guide the robots to successful task completion while minimizing the time taken.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
AI-driven experimental design for learning of process parameter models for robotic processing applications
PDF
Speeding up trajectory planning for autonomous robots operating in complex environments
PDF
Automated alert generation to improve decision-making in human robot teams
PDF
Multi-robot strategies for adaptive sampling with autonomous underwater vehicles
PDF
Target assignment and path planning for navigation tasks with teams of agents
PDF
Algorithms and systems for continual robot learning
PDF
Motion coordination for large multi-robot teams in obstacle-rich environments
PDF
Planning for mobile manipulation
PDF
Process planning for robotic additive manufacturing
PDF
Data scarcity in robotics: leveraging structural priors and representation learning
PDF
Trajectory planning for manipulators performing complex tasks
PDF
Efficiently learning human preferences for proactive robot assistance in assembly tasks
PDF
Robot trajectory generation and placement under motion constraints
PDF
Data-driven acquisition of closed-loop robotic skills
PDF
Robust loop closures for multi-robot SLAM in unstructured environments
PDF
Efficient and effective techniques for large-scale multi-agent path finding
PDF
Identifying and leveraging structure in complex cooperative tasks for multi-agent reinforcement learning
PDF
Stochastic oilfield optimization under uncertainty in future development plans
PDF
Novel soft and micro transducers for biologically-inspired robots
PDF
Leveraging cross-task transfer in sequential decision problems
Asset Metadata
Creator
Shriyam, Shaurya
(author)
Core Title
Contingency handling in mission planning for multi-robot teams
School
Viterbi School of Engineering
Degree
Doctor of Philosophy
Degree Program
Mechanical Engineering
Publication Date
09/23/2019
Defense Date
07/01/2019
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
contingency management,heuristic optimization,mission planning,model checking,multi-robot task allocation,multi-robot task decomposition,OAI-PMH Harvest
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Gupta, Satyandra (
committee chair
), Jin, Yan (
committee member
), Madni, Azad (
committee member
)
Creator Email
shauryashri@gmail.com,shriyam@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c89-219800
Unique identifier
UC11673231
Identifier
etd-ShriyamSha-7838.pdf (filename),usctheses-c89-219800 (legacy record id)
Legacy Identifier
etd-ShriyamSha-7838.pdf
Dmrecord
219800
Document Type
Dissertation
Rights
Shriyam, Shaurya
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
contingency management
heuristic optimization
mission planning
model checking
multi-robot task allocation
multi-robot task decomposition