Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Advancing robot autonomy for long-horizon tasks
(USC Thesis Other)
Advancing robot autonomy for long-horizon tasks
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
ADV ANCING ROBOT AUTONOMY FOR LONG-HORIZON TASKS by Isabel M. Rayas Fernández A Dissertation Presented to the FACULTY OF THE USC GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (COMPUTER SCIENCE) August 2023 Copyright 2023 Isabel M. Rayas Fernández Acknowledgements On its surface, this dissertation appears to be the culmination of 5 years of my work at USC, but it is impossible to overlook the huge community of people throughout my life who have helped me to get to this point. I hope to acknowledge a small number of them here. Firstly, I’d like to thank Gaurav, my PhD advisor, for achieving the perfect balance of hands-on and hands-off mentoring which has allowed me to learn from his deep wisdom and yet grow on my own. His is a voice of calm reason, which was often exactly what I needed. I am grateful to have had a PhD experience where, though it certainly had its stressful aspects, my advisor was never one of them. Thank you for believing in me, and telling me so, when I felt I couldn’t possibly do it. I’d also like to thank the other members of my PhD dissertation committee, David Caron and Stefanos Nikolaidis, who have been kind and supportive, and always had insightful questions for me. I am fortunate to have received support from the National Science Foundation Graduate Research Fel- lowship Program as well as the USC Annenberg and USC Kunzel Fellowships, which together have funded my research and granted me stability and flexibility during my PhD. My experience would have been immeasurably less pleasant without Lizsl De Leon as my academic advisor and liaison to all things administrative at USC. Her care for me as a student was remarkable and her ability to resolve administrative tasks seemingly instantaneously was a gift to my sanity. ii I am grateful for Maja Matari´ c’s mentorship my first year at USC and for her unwavering support when I decided to make the huge move to several desks away. She was instrumental in my decision to come to USC, and for that I am thankful. It is not an overstatement to say I would have not pursued graduate school at all without Julie Shah, my academic advisor and role model at MIT. She saw potential in me that I did not know was there, and without her gentle but insistent prodding, I would not have sought out research as an undergrad with her bright and talented student, Lindsay Sanneman. My positive experience with Lindsay in Julie’s Interactive Robotics Group was key in leading me to where I am today. I’m so thankful to have learned from both of them. In addition to Julie, there are so many faculty and staff I had the privilege of interacting with and learning from at MIT. To name a few: Marie Stuppard, Joyce Light, Prof. Balakrishnan, Prof. Radovitzky, Prof. Darmofal, Prof. Karaman, Prof. Williams, Prof. How, Prof. Stirling, Prof. Miller, Prof. Spakovszky, and the entire AeroAstro department; Prof. Gibson, Prof. Gabrieli, Prof. Desimone, Prof. Fee, Prof. Hogan, Prof. Tenenbaum, and the Brain and Cognitive Sciences department; Prof. Minicozzi; Prof. Staffilani; and Prof. Kaelbling. My experiences at MIT were an indispensable foundation for everything following those four years. I have also been fortunate to have been a part of several student groups throughout my many years as a student. MIT Women in Aerospace Engineering gave me a sense of community and friendship as well as an opportunity to develop leadership skills, including allowing me the joys of planning social events. With such a positive experience with MIT WAE, it was no surprise that I found comfort in being a part of the committee for the USC PhD Women and Gender Minorities in Computing Club as well. My education has been enhanced by the multiple internships I’ve been able to have. At NASA JPL, I’d like to thank Celina Clewans for her compassionate mentorship, intelligence, and patience, and Rich Rieber for his infectious enthusiasm and willingness to spend seemingly endless time teaching his curious intern. Anne Dorsey at Waymo has been a wonderful mentor, teacher, and friend over the past years, and I iii am lucky to have had such a positive experience even dealing with the realities of remote internships. With her and Jenny Iglesias, I always felt like a valued member of the team, and I’m thankful for their support and constant guidance both in technical and life questions. I am also thankful to all the other coworkers and mentors that I’ve had the chance to interact with at both JPL and Waymo. You all made these experiences some of the highlights of the past five years. If my future mentors are welcoming of even 1% of the amount of questions that I asked Mr. Skerbitz in my high school Differential Equations & Linear Algebra class, I will be unbelievably lucky. Thank you for not just tolerating but encouraging my truly incessant questions, even at 7:00am. When asking for help feels daunting, I lean on the belief that it is not too much or too burdensome to ask, and my sense of curiosity is due in part to Mr. Skerbitz’ extraordinary teaching. I’m thankful for Mr. Kilkelly, another high school math teacher, whose joy in the aha moments of understanding made me eager to keep learning and gave me determination to keep trying, even when I struggled. I’m also grateful for all my other teachers and mentors at Wayzata and St. Louis Park – Ms. Hagen, Mr. French, Mr. Bisanz, Ms. Speers, Mr. Gulsvig, Ms. Gohman, Ms. Brown, Mr. Cole, Mr. Ellingson, Mr. Heebink, Mr. Bailer, Mary Kay Williams, Mr. Vrudny, Mrs. Westman, Mrs. Anderson, Mrs. Blaeser, Sr. Franco, and Sr. Bleske are just a tiny few – for giving me the foundations of my education and fostering a safe and nourishing environment for me to grow up in. My music teachers throughout the years were instrumental in encouraging creativity and inspiring me to see and hear beauty in every part of life. To Mr. Gitch and Ms. P-O, thank you. Pottery and ceramics have been a constant source of happiness and restoration for me. Thank you to Mr. Braun for introducing me to the art, and to Jay, Darrell, Julia, Tom, and Ki for their gentle direction and helpful instruction which have allowed me to experiment and continue building skills. I’ve been shaped by my varied experiences abroad, which would not have been possible without the instruction and inspiration from all my many language teachers over the years, as well as my high school iv advisor Mr. Nagel’s trust in me to complete a semester of courses abroad in Germany, and the support and many opportunities abroad that the MIT MISTI office offered me. To Mme. Magallanes, Mme. Janssen, M. Tuura, Mme. Levet, Mme. Showrai, Frau Gut, Frau Jaeger, Herr Suß, Frau Crocker, Nilma Dominique, Ikeda Sensei, and Matsumoto Sensei: Merci, danke, obrigada, arigat¯ o gozaimasu for opening countless doors for me in every corner of the world and for sharing your love of languages with me. I am so fortunate to have many homes away from home. Thank you to Vivi and Astrid, Ann, Elli, and everyone else who has welcomed me when I have been in a new place. I am honored to have been a part of the Robotic Embedded Systems Lab at USC. My labmates in RESL have been supportive, engaging, and fun; they have challenged me and pushed me to be better and to learn new things; they have been excellent companions and collaborators. I am so grateful to have had the chance to work with such talented roboticists and good people. The work presented in this thesis would not have been possible without my collaborators. Peter Englert helped me navigate my first end-to-end research project and was a skillful mentor, Ragesh Ramachandran provided indispensable technical knowledge, and Giovanni Sutanto brought new, interesting ideas and was excited to collaborate. My day-to-day research would have been undoubtedly less enjoyable without Chris Denniston to work with; our meetings were always rife with informative and interesting discussions, and I’m grateful that he convinced me early on that working on environmental robotics was very cool and fun. To Micaela, my oldest friend – our lasting friendship has been a great happiness to me, and knowing you are always there for me is a comfort I feel privileged to have. Paige, your tremendous compassion and fantastic sense of humor have pulled me through some of my toughest challenges. You are a gift. To Kegan, thank you for your optimism and joy with the small things in life. I am lucky to call someone so kind and intelligent a close friend. To David, you made difficult parts of the PhD not just bearable but enjoyable, and I am grateful for your technical help and discussions, and your patience, support, and care over the years. Rachel, you are a star, and you inspire me every day. It’s not often someone is talented in so many ways v and still a sweet, caring, generous, and funny person besides the fact. My dear friend Justin, you never fail to make me laugh, and I’m so thankful for our friendship. My life is better with you in it. Thank you for making me feel at home, wherever in the world we are. Lilly, I couldn’t have done half the things I’ve done without your constant, unconditional support, although you would say that of course I could have. Thank you for always understanding me even and especially when you’re the only one in the world who does. Finally, nothing would have been possible without my family. My parents have shown me love, encour- agement, courage, discipline, balance, joy, and strength. They’ve believed in me from the start, and I feel incredibly fortunate to have them and their support in my life. To my siblings, Wisi, Atzín, Agie, and Eli – even when you’re the worst, you’re still the best. Thank you for bringing Gabbie, Angie, and CJ into my life, and Luca, Félix, and Mateo. There are so many more people who have impacted me positively, both directly and indirectly, and I am so grateful to all of them. Thank you. vi Table of Contents Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Chapter 2: Background on Constrained Motion Planning . . . . . . . . . . . . . . . . . . . . . . . 6 2.1 Sampling-Based Motion Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2 Constrained Sampling-Based Motion Planning . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.3 Planning with Sequential Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.4 Manifold Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.5 Manifold Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.6 Learning from Demonstration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 Chapter 3: PSM ∗ : Planning on Sequenced Manifolds . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 3.2 Problem Formulation and Application Domain . . . . . . . . . . . . . . . . . . . . . . . . 15 3.2.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2.2 Intersection Point Independent Problems . . . . . . . . . . . . . . . . . . . . . . . 19 3.3 Planning on Sequenced Manifolds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.3.1 Inner Loop: Growing a Tree to the Next Manifold . . . . . . . . . . . . . . . . . . . 21 3.3.1.1 SteerPoint(q near ,q rand ,M i ) . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.3.1.2 SteerConstraint(q near ,M i ,M i+1 ) . . . . . . . . . . . . . . . . . . . . . . . 23 3.3.2 Outer Loop: Initializing the Next Subproblem . . . . . . . . . . . . . . . . . . . . . 25 3.3.3 Completeness and Optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.4.1 3D Point on Geometric Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.4.2 Multi-Robot Object Transport Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.4.3 Pick-and-Pour on Panda Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Chapter 4: ECoMaNN: Learning Equality Constraints for Motion Planning on Manifolds . . . . . . 36 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 vii 4.1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.1.2 Motion Planning on Manifolds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.2 Equality Constraint Manifold Neural Network (ECoMaNN) . . . . . . . . . . . . . . . . . 38 4.2.1 Alignment of Local Tangent and Normal Spaces . . . . . . . . . . . . . . . . . . . 40 4.2.2 Data Augmentation with Off-Manifold Data . . . . . . . . . . . . . . . . . . . . . . 41 4.2.2.1 Training Losses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 4.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 4.3.1 Accuracy and Precision of Learned Manifolds . . . . . . . . . . . . . . . . . . . . . 44 4.3.2 Ablation Study of ECoMaNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.3.3 Learning ECoMaNN on Noisy Data . . . . . . . . . . . . . . . . . . . . . . . . . . 47 4.3.4 Relationship Between the Number of Augmentation Levels and the Projection Success Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 4.3.5 Motion Planning on Learned Manifolds . . . . . . . . . . . . . . . . . . . . . . . . 48 4.4 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Chapter 5: Background on Informative Path Planning in Natural Environments . . . . . . . . . . . . 50 5.1 Informative Path Planning (IPP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 5.2 Objective Functions for IPP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.3 Physical Specimen Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 5.4 Quantiles and Quantile Standard Error Estimation . . . . . . . . . . . . . . . . . . . . . . . 54 5.5 Continuous Non-Convex Gradient-Free Optimization . . . . . . . . . . . . . . . . . . . . . 54 5.6 Multirobot Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 5.6.1 Multirobot IPP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 5.6.2 Bandwidth-Constrained Multirobot Communication . . . . . . . . . . . . . . . . . 56 Chapter 6: Informative Path Planning to Estimate Quantiles for Environmental Analysis . . . . . . . 58 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 6.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.3 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.4 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 6.4.1 Informative Path Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 6.4.2 Location Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.5 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 6.5.1 Informative Path Planning: Objective Functions . . . . . . . . . . . . . . . . . . . . 68 6.5.1.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.5.1.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.5.2 Location Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.5.2.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.5.2.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 6.5.3 Field trial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 6.5.3.1 Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 6.5.3.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Chapter 7: Multirobot Quantile Estimation in Natural Environments . . . . . . . . . . . . . . . . . 78 7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 7.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 7.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 viii 7.3.1 Initial Location Spread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 7.3.2 Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 7.3.3 Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 7.4 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 7.4.1 Initial Location Spread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 7.4.2 Planning Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 7.4.3 Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 7.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 7.5.1 Initial Location Spread . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 7.5.2 Planning Budget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 7.5.3 Team Size and Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 7.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 7.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Chapter 8: Reducing Network Load via Message Utility Estimation for Decentralized Multirobot Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 8.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 8.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 8.3.1 Summary of Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 8.3.1.1 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 8.3.1.2 Exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 8.3.1.3 Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8.3.2 Communication Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 8.3.3 Modeling Others and Updating Models . . . . . . . . . . . . . . . . . . . . . . . . 102 8.3.4 Computing Message Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 8.3.5 Deciding to Communicate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 8.4 Experiments and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 8.4.1 Utility Methods Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 8.4.2 Oracle Handshaking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 8.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 Chapter 9: Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Appendices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Appendix A: PSM ∗ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 PSM ∗ (Single Tree) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 RRT ∗ Extension Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Probabilistic Completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Asymptotic Optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Appendix B: ECoMaNN Orthogonal Subspace Alignment . . . . . . . . . . . . . . . . . . . . . 141 ix List of Tables 3.1 PSM ∗ algorithm parameters and their description. . . . . . . . . . . . . . . . . . . . . . . . 20 3.2 Parameters of the 3D Point on Geometric Constraints and Multi-Robot Object Transport experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3.3 Results of the 3D point and robot transport problems. We report the mean and one unit standard deviation over 10 runs with different random seeds. . . . . . . . . . . . . . . . . . 35 4.1 Accuracy and precision of learned manifolds averaged across 3 instances. “Train" indicates results on the on-manifold training set; “test" indicates N= 1000 projected (ECoMaNN) or sampled (V AE) points. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4.2 Percentage of projection success rate for a variety of ablation on ECoMaNN components. . . 47 5.1 Informative Path Planning as a POMDP. After [147, 24]. . . . . . . . . . . . . . . . . . . . 52 7.1 Parameters studied in our experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 8.1 Number of attempted and successful transmitted messages per utility method. Restricting transmissions with Action results in more than 40% reduction in network load (attempted) while improving mean final quantile estimation error by 1 .84% and only resulting in a less than 25% decrease in successful transmissions. Results reported from 60 experiments total of length 40 steps. All experiments use N = 4, so the most messages possibly sent total is 40× (N− 1)= 120. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 x List of Figures 3.1 3D Point on Geometric Constraints – The surfaces visualize the level sets of the three constraints. The task is to move from the start point (red dot) to the goal point (blue dot) while fulfilling the constraints. The line shows a solution found by PSM ∗ that fulfills these constraints and the magenta points are the found intersection vertices. . . . . . . . . . . . . 13 3.2 Initialization of a new subtree (line 15 – line 17 of Algorithm 1). The reached goal nodes at the intersection M 1 ∩ M 2 are used to initialize the next tree where q root is a synthetic root node that maintains the tree structure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.3 Samples of found paths on the 3D point on with obstacles problem (Section 3.4.1). . . . . . 30 3.4 Path costs over variations of the parameterρ (left) and the number of samples m (right) on the geometric constraints w/o obstacles problem (Section 3.4.1). The graphs show the mean and unit standard deviation over 10 trials. Figure (a) shows the costs increase for higher values ofρ, meaning fewer intersection points are considered during planning. In (b), the performance of all methods improves with larger m values where PSM ∗ and PSM ∗ (Single Tree) converge to similar path costs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.5 Start states of the object transport tasks where the goal is to place the red and blue object to the green target locations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.6 Snapshots of the resulting motion that PSM ∗ found for Task C. . . . . . . . . . . . . . . . . 33 3.7 Snapshots of the pouring task motion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 4.1 A visualization of data augmentation along the 1D normal space of a point q in 3D space. Here, purple points are the dataset, pink points are the KNN of q, and the dark red point is ˇ q. q is at the axes origin, and the green plane is the approximated tangent space at that point. 39 4.2 Visualization for Section 4.3.1, Plane dataset. Red points are the training dataset and blue points are samples generated from the learned manifolds. . . . . . . . . . . . . . . . . . . . 43 4.3 Trained ECoMaNN’s level set contour plot and the normal space eigenvector field, after training on the sphere constraint dataset (left) and plane constraint dataset (right). . . . . . . 45 xi 4.4 (a): ECoMaNN success rate on the Orient dataset over number of training iterations. (b): A path planned on the learned manifold (red) and path on the ground truth manifold (black) are visualized. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 4.5 Visualization of a path that was planned on a learned orientation manifold. . . . . . . . . . . 48 6.1 Full system. First, the parameters for the robotic survey are chosen, such as the area bounds and the quantiles for specimen collection. The robot performs informative path planning using our proposed objective functions, creating an environment model and an estimate of the quantile values. The quantile locations are then selected by minimizing our proposed loss function. These quantile locations correspond to physical locations which, when measured, have the estimated quantile values. After the quantile locations are chosen, humans go to them to collect field specimens which they later analyze in a laboratory setting. This work focuses on the steps shown in the blue shaded boxes. . . . . . . . . . . . 59 6.2 Simulated drone planning experiments with real data. Error between ground truth quantile values and estimated quantile values. Datasets A and B collected using a hyperspectral camera in Clearlake, California. Units are 400nm channel pixel intensity (0− 255). Each dataset and objective function pairing is run three times. . . . . . . . . . . . . . . . . . . . 66 6.3 Experimental dataset distributions. Drone data is measured in pixel intensity; AUV data in µg/L chlorophyll. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 6.4 Simulated AUV planning experiments with real data. Error between ground truth quantile values and estimated quantile values. Datasets C and D collected from a reservoir in California using an underwater robot with a chlorophyll fluorescence sensor. Units are µg/L chlorophyll. Each dataset and objective function pairing is run three times. . . . . . . 68 6.5 Quantile location selection results. RMSE between the ground truth values at the selected locations and the corresponding true quantile values. Units areµg/L chlorophyll for AUV , 400nm channel pixel intensity (0− 255) for Drone. . . . . . . . . . . . . . . . . . . . . . . 70 6.6 Comparison between quantile standard error with POMDP solver and Spiral-STC coverage planner [33] on Drone Environment B estimating deciles, each over 3 trials. . . . . . . . . . 71 6.7 Physical locations (stars) selected by the cross-entropy optimizer for deciles on the drone experiment. [Left] Black lines: true quantile value contours, overlaid on the absolute error between µ(X # ) and GT(X # ). Note that the lowest error tends to follow the quantile contour lines. [Right] Red crosses: locations the robot visited, overlaid on the ground truth image. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 6.8 Physical locations (stars) selected by the cross-entropy optimizer for the upper extrema quantiles with an AUV using a chlorophyll point sensor. Blue/green points are the measured locationsX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 xii 6.9 Visualization of a field trial modeling a crop health task. Left: Crosses are locations where the drone took images. The drone is limited to only visit 20% of the possible locations to take images. The quantiles of interest are the deciles and the locations are chosen by cross-entropy. The 9 stars show locations suggested to collect physical specimen. The measurement of interest is the amount of green in each pixel. Right: The drone used in the field trial, in flight. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 7.1 Some variations of multirobot planning approaches. Top row shows initial spreadα = 0: All robots start in the center. Bottom row shows initial spread α = 0.5: Robots start spread in an area 50% length and width of entire workspace. Left column shows no communication: Robots have no knowledge of the others; equivalent to each robot planning as if it were the only one. Middle column shows stochastic communication: Robots attempt to share observations but messages fail stochastically based on inter-robot distance. Right column shows full communication: Robots share the same environment model; equivalent to centralized planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 7.2 Visualization of the message transmission success probability based on inter-robot distance (Equation (7.8)). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 7.3 Error vs. Initial location spread (α). Top: No communication. Bottom: stochastic communication. Right column shows results further separated by team size. Bars above indicate significance under the one-sided Wilcoxon signed-rank test; ***: p≤ 1e− 3, *: p≤ 5e− 2, ns: no significance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 7.4 Error vs. Budget (number of planning steps B T ). Right shows results further separated by team size. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 7.5 Error vs. Budget Type (Shared and Complete). Here, B T = 15. Left: Shared budget means the combined budget is B T for any N, while complete budget means it is N· B T . Note N= 1 data not included here since budget type has no effect. Right: Results further separated by team size. N= 1 (blue) shown for comparison. . . . . . . . . . . . . . . . . . . . . . . . . 90 7.6 Error vs. Communication type. Left: Aggregated across different N. Note that results for N = 1 are not included, since they are not affected by the variations. Right: Same results, further separated by team size. N= 1 included for comparison. . . . . . . . . . . . . . . . 91 7.7 Example paths of a four-robot team with B T = 10 (left) and B T = 30 (right). . . . . . . . . 93 7.8 Example paths of a single robot with B T = 15 (left) and an 8-robot team with a shared budget of B T = 15 (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 7.9 Example paths of an 8-robot team under partitions (left) and no communication (right). . . . 93 8.1 Overview of problem pipeline. The approach consists of 3 phases: Initialization, Exploration, and Aggregation. The bulk of the work happens during the decentralized exploration phase, where robots repeatedly choose the next location to visit, measure the environment, optionally broadcast the measurement in a message, and update their models until their budget B T is exhausted. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 xiii 8.2 Tradeoff in network load vs. task performance for different utility methods, compared to always sending messages. Top: Percent decrease in messages sent (network load). Higher is better. Bottom: Percent increase in final quantile estimation error. Lower is better. . . . . 105 8.3 Cumulative network load, as measured by messages sent, over time. Each step on the x-axis represents a robot taking a planning step. . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 8.4 Amount of attempted and successful transmissions over time for different utility methods. Each step on the x-axis represents a robot taking a planning step. . . . . . . . . . . . . . . 107 8.5 Final estimation error for different utility methods. . . . . . . . . . . . . . . . . . . . . . . 110 8.6 Example paths using two different utility methods with Q =(0.9,0.95,0.99). Top: Action. Bottom: Reward. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 8.7 Right: the sender receives an oracle handshake when a message is successfully transmitted. Bar indicates significance under one-sided Wilcoxon signed-rank test; *: p≤ 5e− 2. . . . . 112 xiv Abstract Autonomous robots have real-world applications in diverse fields, such as mobile manipulation and en- vironmental exploration, and many such tasks benefit from a hands-off approach in terms of human user involvement over a long task horizon. However, the level of autonomy achievable by a deployment is lim- ited in part by the problem definition or task specification required by the system. Task specifications often require technical, low-level information that is unintuitive to describe and may result in generic solutions, burdening the user technically both before and after task completion. In this thesis, we aim to advance task specification abstraction toward the goal of increasing robot autonomy in real-world scenarios. We do so by tackling problems that address several different angles of this goal. First, we develop a way for the automatic discovery of optimal transition points between subtasks in the context of constrained mobile manipulation, removing the need for the human to hand-specify these in the task specification. We further propose a way to automatically describe constraints on robot motion by using demonstrated data as opposed to manually-defined constraints. Then, within the context of environmental exploration, we propose a flex- ible task specification framework, requiring just a set of quantiles of interest from the user that allows the robot to directly suggest locations in the environment for the user to study. We next systematically study the effect of including a robot team in the task specification and show that multirobot teams have the ability to improve performance under certain specification conditions, including enabling inter-robot communication. Finally, we propose methods for a communication protocol that autonomously selects useful but limited information to share with the other robots. xv Chapter 1 Introduction Many robotic tasks benefit from a hands-off approach in terms of human user involvement over a long task horizon. However, robot systems continue to face barriers to deployment for autonomous real-world, widespread use. One of these barriers stems from the problem descriptions, or task specifications, that robots require from their users. These task specifications often require a high degree of specific, technical input, which is not necessarily intuitive to describe, or they suffer from a lack of customizability or tailoring to the user’s end goal, producing outputs that require further post-processing by the user. In contrast, we would like to be able to specify a robot task in an interpretable way, that both abstracts the specification away from technical details and flexibly targets solutions that the user cares about. In this thesis, we propose solutions to five problems, which advance toward this goal. 1. In long-horizon, sequenced tasks, transitions between subtasks can have a large impact on the quality of solution, but humans are not well-equipped to specify the optimal transitions to use. How can we remove the need for hand-specified transition points? 2. Constraints on robot motions during task execution can be difficult to specify analytically but more accessible to show via demonstration. How can we automatically learn a representation of a motion constraint from data? 1 3. In long-horizon environment exploration tasks, task specifications typically target finding a predefined type of location, but different application domains require different types of interest. How can we describe these distinct tasks in a unified high-level, interpretable task specification that directly leads to automatic discovery of those locations? 4. Different users and domains will also have different resources available to them, including number of robots and inter-robot communication infrastructure, but the effect of task specifications with these conditions on performance remains understudied in a systematic way. How can different robot team setups be effectively used during environment exploration? 5. Inter-robot communication aids collaborative task performance, but real robot networks are resource- constrained and cannot support arbitrary message loads, and not every message is equally informative. How can we autonomously decide which information is most valuable to send to other robots? To ground these problems, we propose solutions in two application domains. We consider the first two problems in the context of constrained manipulation and mobility applications and provide necessary background for this context in Chapter 2. We address (1) in Chapter 3, where we tackle the problem of motion planning for sequential tasks, where each subtask therein is defined by the constraints on the motion allowed during it. Our solution results in the automatic discovery of optimal transition points between subtasks, removing the need for the human to hand-specify these in the task specification. In particular, we address the problem of planning robot motions in constrained configuration spaces where the constraints change throughout the motion. The problem is formulated as a fixed sequence of intersecting manifolds, which the robot needs to traverse in order to solve the task. We propose a class of sequential motion planning problems that fulfill a particular property of the change in the free configuration space when transitioning between manifolds, and for this problem class, we develop the algorithm Planning on Sequenced Manifolds (PSM ∗ ) which searches for optimal 2 intersection points between manifolds by using rapidly exploring random trees (RRT ∗ ) in an inner loop with a novel steering strategy. We provide a theoretical analysis regarding PSM ∗ s probabilistic completeness and asymptotic optimality. We also evaluate its planning performance on multi-robot object transportation tasks and compare it to other methods. We address (2) in Chapter 4, where we propose a way to automatically describe constraints on robot motion by using demonstrated data as opposed to manually-defined constraints. Specifically, in this work, we consider the problem of learning representations of motion constraints from demonstrations with a deep neural network, which we call Equality Constraint Manifold Neural Network (ECoMaNN). To do so, we propose learning a level-set function of the constraint by aligning subspaces in the network with subspaces of the data such that it can be integrated into a constrained sampling-based motion planner, such as PSM ∗ . We combine both learned constraints and analytically described constraints into the planner and use a projection- based strategy to find valid points. We evaluate ECoMaNN on its representation capabilities of constraint manifolds, the impact of its individual loss terms, and the motions produced with it. The last three problems we consider in the context of robotics in environmental sensing and monitoring, inspired by our collaboration with a team of biologists using robots to help explore and monitor harmful algae bloom growth in aquatic environments. Chapter 5 provides relevant background for this context. We address (3) in Chapter 6, where we propose a flexible, abstract task specification, requiring just a set of quantiles of interest from the user that allows the robot to directly suggest locations in the environment to the user that will contain information that they care about. To this end, we propose to choose locations for scientific analysis by using a robot to perform an informative path planning survey, in contrast to relying on expert heuristics to choose them. The survey results in a list of locations that correspond to the quantile values of the phenomenon of interest. We develop a robot planner using novel objective functions to improve the estimates of the quantile values over time and an approach to find locations which correspond to the quantile values. We test our approach in four different environments using previously collected aquatic 3 data and compare it to a baseline approach which attempts to maximize spatial coverage. Additionally, we validate our work in a real field trial. We address (4) in Chapter 7, where we systematically study the effect of including a robot team in the task specification for the quantile estimation task introduced in Chapter 6 and show that multirobot teams have the ability to improve accuracy under certain specification conditions, one of which is when they are communication-enabled. This work aims to understand effects of different multirobot setups, since a multi- robot team can be difficult to practically bring together and coordinate, especially when robot communica- tion is involved. To this end, we present a study across several axes of the impact of using multiple robots to estimate quantiles of a distribution of interest using an informative path planning formulation. We measure quantile estimation accuracy with increasing team size to understand what benefits result from a multirobot approach in a drone exploration task of analyzing the algae concentration in lakes. We additionally perform an analysis on several parameters, including the spread of robot initial positions, the planning budget, and inter-robot communication, and find that while using more robots generally results in lower estimation error, this benefit is achieved under certain conditions. We present our findings in the context of real field robotic applications and discuss the implications of the results. We address (5) in Chapter 8, where we propose methods for a communication protocol which au- tonomously selects useful but limited information to share with the other robots. These methods deter- mine the utility of sharing information across a resource-constrained communication network, in which case naïve, constant information-sharing is infeasible, and we show that limiting the load on the network is possible without sacrificing performance on the estimation task. In particular, we propose online, locally computable metrics for determining the utility of transmitting a given message to the other team members and a decision-theoretic approach that chooses to transmit only the most useful messages, using a decentral- ized and independent framework for maintaining beliefs of other teammates. We validate our approach in simulation on a real-world aquatic dataset, and show that restricting communication via a utility estimation 4 method based on the expected impact of a message on future teammate behavior results in a decrease in network load while simultaneously maintaining good or improved task performance. Finally, we conclude the thesis and summarize the contributions in Chapter 9. 5 Chapter 2 Background on Constrained Motion Planning In this and the next two chapters of this thesis, we place ourselves in the context of constrained mobile manipulation tasks, and in particular, sequenced tasks. These sequenced tasks are composed of smaller individual subtasks, which can together be as short as a picking task, or as complex as a multirobot, multi- handover object manipulation task. Viewed through this lens, such problems are long-horizon in that their subtasks and associated motion constraints can be arbitrarily numerous. As we will discuss, task specifi- cations for these problems have typically required technical descriptions of the subtasks, their constraints, and how to transfer between them. This chapter gives relevant background for this application domain, laying the foundation for Chapters 3 and 4 which address aspects of task abstraction toward increased robot autonomy in these types of problems. In these chapters, we will consider offline planning in observable environments. 2.1 Sampling-Based Motion Planning Sampling-based motion planning (SBMP) is a broad field which tackles the problem of motion planning by using randomized sampling techniques to build a tree or graph of configurations (also called samples), which can then be used to plan paths between configurations. A PRM (probabilistic roadmap) is a path planner that builds a graph in the free configuration space that can be used for multiple queries [3, 62, 97]. 6 Kavraki et al. [62] describe the method as a two-step procedure. First, a roadmap is built by sampling collision-free nodes and edges. Second, a path is found from a start to a goal state by using a graph search algorithm. This technique is multi-query, as it can be reused for many planning problems with the same system. Tree methods such as RRTs (rapidly-exploring random trees) are generally single-query, taking a specific start state from which a tree of feasible states is grown toward a specific goal state or region [77]. Many extensions to RRT exist, such as bidirectional trees and goal biasing [75, 76]. Optimal variants RRT ∗ and PRM ∗ find paths that minimize a cost function and guarantee asymptotic optimality by using a rewiring procedure on the edges in the graphs [61]. All these techniques consider the problem of planning without motion constraints; that is, they plan in the free configuration space. 2.2 Constrained Sampling-Based Motion Planning A more challenging and realistic motion planning task is that of constrained SBMP, where there are motion constraints beyond just obstacle avoidance which lead to a free configuration space manifold C M ∈R l of lower dimension than the ambient configuration space C ∈R n . Many practical tasks require planning with constraints. Constrained SBMP algorithms [135, 6, 54, 65, 68, 138, 18] extend SBMP to these types of constrained configuration spaces (see [69] for a review). Randomized sampling in a n-dimensional space will produce samples on an l-dimensional space with probability 0 where l< n; these spaces usually cannot be sampled directly. Thus, a large focus of these methods is how to generate valid samples and steer the robot while fulfilling the constraints. Many SBMP algorithms derive from rapidly-exploring random trees (RRT), probabilistic roadmaps (PRM), or their optimal counterparts RRT ⋆ and PRM ⋆ . One family of approaches are projection-based strategies [6, 135, 138, 59] that first sample a configu- ration from the ambient configuration space and then project it using an iterative gradient descent strategy to a nearby configuration that satisfies the constraint. Berenson et al. [6] proposed CBiRRT (constrained bidirectional RRT) that uses projections to find configurations that fulfill constraints. The constraints are 7 described by task space regions, which are a representation of pose constraints. Their method can also be used for multiple constraints over a single path. However, their approach requires each constraint’s active domain to be defined prior to planning, or configurations are simply projected to the nearest manifold rather than respecting a sequential order. An alternative is to approximate the constraint surface by a set of local models and use this approxima- tion throughout the planning problem for sampling or steering operations [54, 55, 65, 139, 135, 11]. For example, AtlasRRT [55] builds an approximation of the constraint consisting of local charts defined in the tangent space of the manifold. This representation is used to generate samples that are close to the constraint. Similarly, Tangent Bundle RRT [65, 139] builds a bidirectional RRT by sampling a point on a tangent plane, extending this point to produce a new point, and if it exceeds a certain distance threshold from the center of the plane, projecting it on the manifold to create a new tangent plane. Kingston et al. [68] presented the implicit manifold configuration space (IMACS) framework that de- couples two parts of a geometrically constrained motion planning problem: the motion planning algorithm and the method for constraint adherence. With this approach, IMACS acts as a representative layer between the configuration space and the planner. Many of the previously mentioned techniques fit into this frame- work. They present examples with both projection-based and approximation-based methods for constraint adherence in combination with various motion planning algorithms. Previous research has also investigated incorporating learned constraints or manifolds into planning frameworks. These include performing planning in learned latent spaces [51], learning a better sampling distribution in order to take advantage of the structure of valid configurations rather than blindly sample uniformly in the search space [50, 85], and attempting to approximate the manifold (both explicitly and implicitly) of valid points with graphs in order to plan on them more effectively [105, 38, 156]. 8 2.3 Planning with Sequential Tasks One approach to plan sequential motions is task and motion planning (TAMP), which requires semantic reasoning on selecting and ordering actions to complete a higher-level task [58, 20, 73, 148, 19, 5, 12, 70, 104]. Broadly speaking, TAMP is more general and difficult to solve compared to SBMP due to scaling and a more complicated problem definition. Though we do not address TAMP in this work, it is an interesting future direction that shares many characteristics with SBMP. [37] proposed the multi-modal motion planning algorithm Random-MMP, which plans motions over multiple mode switches that describe changes in the planning domain (e.g., contacts). Their planner builds a tree in a hybrid state using an SBMP that consists of the continuous robot configuration and a discrete mode, which changes based on the domain. Other previous work [150, 70, 37, 128, 127, 94, 122] addressed this problem in related ways. They generally assume sampling of the modes and their boundaries, and build graphs that connect these to each other. Finally, trajectory optimization [137, 123, 146, 111, 9] is an approach to solve sequential motion plan- ning problems where an optimization problem over a trajectory is defined that minimizes costs subject to constraints. However, trajectory optimization often depends on having a good initial trajectory and can suf- fer from poor local minima. The time points in the trajectory at which costs and constraints are active also need to be specified precisely, which can be challenging since it is difficult to know in advance how long a specific part of a task takes. 2.4 Manifold Theory We include a brief background on manifold theory (see [10] for a rigorous treatment). An important idea in differential geometry is the concept of a manifold – a surface which can be well-approximated locally using an open set of a Euclidean space. In general, manifolds are represented using a collection of local regions called charts and a continuous map associated with each chart such that the charts can be continuously 9 deformed to an open subset of a Euclidean space. An alternate representation of manifolds, which is useful from a computational perspective, is to express them as zero level sets of continuous functions defined on a Euclidean space. Since the latter representation is a direct result of the implicit function theorem, it is referred to as implicit representation of the manifold. For example, a unit sphere embedded inR 3 can be represented implicitly as{x∈R 3 |∥x∥− 1= 0}. An implicit manifold is said to be smooth if the implicit function associated with it is smooth. The set of all tangent vectors at a point on a manifold is a vector space called the tangent space of the manifold. The null space of the Jacobian of the implicit function at a point is isomorphic to the tangent space of the corresponding manifold at that point. Since the tangent spaces of a manifold are vector spaces, we can equip them with an inner product structure which enables the computation of the length of curves traced on the manifold. A manifold endowed with an inner product structure is called a Riemannian manifold [78]. In this thesis, we will only consider smooth Riemannian manifolds. 2.5 Manifold Learning Manifold learning is applicable to many fields and thus has garnered a wide variety of methods over the years. Linear methods include PCA and LDA [27], and while they are simple, they lack the complexity to describe nonlinearities. Nonlinear methods include multidimensional scaling (MDS) [74], locally linear embedding (LLE) [119], Isomap [143], and local tangent space alignment (LTSA) [159]. These approaches use techniques such as eigenvalue decomposition, nearest neighbor reconstructions, and local-structure- preserving graphs to visualize and represent manifolds. MDS uses eigenvalue decomposition to produce data that preserve distances between points. LLE preserves local structure by reconstructing each point with the same set of nearest neighbors in both the ambient and the embedding spaces. Isomap approximates geodesic distances on a manifold by constructing a graph from the data, and then computing all shortest paths. In LTSA, the local tangent space information of each point is aligned to create a global representation 10 of the manifold. We refer the reader to [83] for a thorough description of these and other methods. Recent work in manifold learning additionally takes advantage of (deep) neural architectures. Some approaches use autoencoder-like models [42, 15] or deep neural networks [95] to learn manifolds, e.g. of human motion. Others use classical methods combined with neural networks, for example as a loss function for control [141] or as structure for the network [153]. Locally Smooth Manifold Learning (LSML) [26] defines and learns a function which describes the tangent space of the manifold, allowing randomly sampled points to be projected onto it. 2.6 Learning from Demonstration Learning from demonstration (LfD) techniques learn a task representation from data which is usable to generate robot motions that imitate the demonstrated behavior. One approach to LfD is inverse optimal control (IOC), which aims to find a cost function that describes the demonstrated behavior [112, 160, 113, 79]. Recently, IOC has been extended to also extract constraints from demonstrations [107, 31]. There, a cost function as well as equality and inequality constraints are extracted from demonstrations, which are useful to describe behavior like contacts or collision avoidance. A more direct approach to LfD is based on learning parameterized motion representations [121, 100, 102]. They represent the demonstrations in a parameterized form such as Dynamic Movement Primitives [52]. Here, learning is easier because it is often linear regression; however, the ability to generalize to new situations is more limited. The advantage of these approaches lies in dynamic tasks like throwing a ball where a fixed order of states must be followed. A main advantage of using a planner compared to primitives is that planners are better at avoiding collisions between the robot and its environment. Other approaches to LfD include learning of task space [56] and deep learning approaches [32]. We refer the reader to the survey [4] for a broad overview on LfD. 11 Chapter 3 PSM ∗ : Planning on Sequenced Manifolds We begin motivated by the desire to assign the task of selecting subtask transitions to the robot, as opposed to requiring the user to preselect exactly how to transfer between subtasks. Our solution to this problem, presented in this chapter based on the work published in [30], results in autonomous optimal subtask inter- section point selection and motion plans which solve long-horizon problems with an arbitrary number of subtasks. 3.1 Introduction Sampling-based motion planning (SBMP) considers the problem of finding a collision-free path from a start configuration to a goal configuration. Probabilistic roadmaps [62] or rapidly exploring random trees [77] generate plans for such motions while providing theoretical guarantees regarding probabilistic complete- ness. Optimal algorithms like RRT ∗ [61] additionally minimize a cost function and achieve asymptotic optimality. However, many tasks in robotics require the incorporation of additional objectives like subgoals or constraints. SBMP methods are difficult to directly apply to such problems because they require splitting the overall task into multiple planning problems that fit into the SBMP structure. A disadvantage of this approach is that a goal in the configuration space needs to be specified for each subtask, and each selected subgoal will affect the future subtasks. 12 Figure 3.1: 3D Point on Geometric Constraints – The surfaces visualize the level sets of the three constraints. The task is to move from the start point (red dot) to the goal point (blue dot) while fulfilling the constraints. The line shows a solution found by PSM ∗ that fulfills these constraints and the magenta points are the found intersection vertices. We consider a formulation for sequential motion planning where a problem is represented as a fixed sequence of manifolds. We propose Planning on Sequenced Manifolds (PSM ∗ ), an algorithm that searches for an optimal path that starts at an initial configuration, traverses the manifold sequence, and converges when the final manifold is reached. Our solution is to grow a single tree over the manifold sequence. This tree consists of multiple subtrees that originate at the intersections between pairs of manifolds. We propose a novel steering strategy that guides the robot towards these manifold intersections. After an intersection is reached, a new subtree is initialized with the found intersection points and their costs. We use dynamic programming over optimal intersection points which scales well to long-horizon tasks since a new subtree is initialized for every manifold. The algorithm is applicable to a class of problems we call intersection point independent, specified by a property which restricts how the free configuration space changes across subtasks (see Section 3.2). A running example in this chapter is the task of using a robot arm to transport a mug from one table to another while keeping the orientation of the mug upright, a task involving multiple phases and constraints 13 described informally as follows. First, the arm moves to pick up the mug. In this subtask, the arm can move freely in space and only needs to avoid collisions with obstacles. The second subtask is to grasp the mug. The third subtask is for the arm to transfer the mug with the constraint of always keeping the mug upright. The final subtask is to place and release the mug which requires the base of the mug to be near the table. In Section 3.4, we demonstrate the performance of PSM ∗ on similar sequential kinematic planning problems that involve multiple robots and compare it to alternative planning strategies. 3.1.1 Background Our proposed method builds on the methods introduced in Chapter 2 and extends them towards sequential tasks where the active constraints change during the motion. We use a modified version of RRT ∗ in the inner loop of PSM ∗ that can handle goals defined in terms of equality constraints instead of a goal configuration. Restricting the problem class to intersection point independent problems (Section 3.2.2) allows us to grow a single tree over a sequence of manifolds on which rewiring operations can still be performed. We show that PSM ∗ inherits the probabilistic completeness and asymptotic optimality guarantees of RRT ∗ (Section 3.3.3). Our approach is perhaps closest to the CBiRRT algorithm. A main difference is that our method assumes a different problem formulation where the task is given in terms of a fixed sequence of manifolds where the intersections between manifolds describe subgoals that the robot should reach. This formulation allows us to define a more structured steering strategy that guides the robot towards the next subgoals. Another difference is that CBiRRT does not search for optimal paths, while we employ RRT ∗ to minimize path lengths. We compare our method PSM ∗ to CBiRRT in Section 3.4. Our work also shares similarities with others in the task and motion planning (TAMP) problem area, in that we consider planning over multiple manifolds where changes in the configuration space occur due to picking or placing objects. However, our work differs in several key ways. First, TAMP approaches assume the task sequence is unknown in advance, and thus selecting the transition to the next constraint is part of 14 the planning problem. Here we specify our problem formulation over a fixed and known sequence of con- straints, focusing our algorithm on optimizing the transition points between constraints to find an optimal path. Additionally, we do not assume direct sampling of modes or switches is possible; rather, our algorithm is designed to find the mode-switching configurations during exploration toward manifold intersections. Im- portantly, because we build one search tree, these configurations are guaranteed to be connected to the start. This eliminates sampling of configurations that are disconnected from the viable solution space. Further, we differ by applying PSM ∗ in the domain of intersection point independent problems, which we specify for sequential planning problems based on the free space transformation at each manifold intersection (Sec- tion 3.2.2). These problems do not include foliated manifolds and are efficiently solvable by growing a single tree. Finally, in contrast to trajectory optimization, our approach only requires the sequential order of tasks and does not make any assumption of specific time points or durations of subtasks. 3.2 Problem Formulation and Application Domain We consider kinematic motion planning problems in a configuration space C⊆ R k . A configuration q∈ C describes the state of one or multiple robots with k degrees of freedom in total. We represent a manifold M as an equality constraint h M (q)= 0 where h M (q) :R k →R l and l is the intrinsic dimensionality of the constraint (l≤ k). The set of robot configurations that are on a manifold M is given by C M ={q∈ C| h M (q)= 0}. 15 We define a projection operator q proj = Project(q,M) that takes a robot configuration q∈ C and a manifold M as inputs and maps q to a nearby configuration on the manifold q proj ∈ C M . We use an iterative optimization method, similar to [68, 6, 136], that iterates q n+1 = q n − J M (q n ) + h M (q n ) until a fixed point on the manifold is reached, which is checked by the condition ||h M (q proj )||≤ ε. The matrix J M (q) + is the pseudo-inverse of the constraint Jacobian J M (q)= ∂ ∂q h M (q). We are interested in solving tasks that are defined by a sequence of n+1 such manifoldsM ={M 1 ,M 2 ,...,M n+1 } where the number and order of manifolds is given, and an initial configuration q start ∈ C M 1 that is on the first manifold. The goal is to find a path from q start that traverses the manifold sequenceM and reaches a configuration on the goal manifold M n+1 . A path on the i-th manifold is defined as τ i :[0,1]→ C M i and J(τ i ) is a cost function of a path J :T →R ≥ 0 whereT is the set of all non-trivial paths. In the following problem formulation, we consider the scenario where the free configuration space C free changes during the task (e.g., when picking or placing objects). We define an operator ϒ on the 16 free configuration space C free , which describes how C free changes as manifold intersections are traversed.ϒ takes as input the endpoint of a pathτ i− 1 (1) on the previous manifold M i− 1 and its associated C free , which we denote as C free,i− 1 .ϒ outputs an updated free configuration space C free,i =ϒ(C free,i− 1 ,τ i− 1 (1)) that accounts for the geometric changes due to transitioning to a new manifold. Here, we assume these changes only occur at the transition between two manifolds. We define ϒ in this way rather than with the configuration spaces of the robot and the objects in the environment because our formulation does not have a notion of “objects"; any object in the world is either an obstacle, or it becomes part the robot system itself. We assume access to a collision check routine CollisionFree(q a ,q b ,C free,i )→{0,1} that returns 1 if the path between two configurations on the manifold q a ,q b ∈ C M is collision-free, 0 other- wise. Note that here, we use the straight-line path in ambient space between two nearby points. However, other implementations for the collision check are possible, such as a path on the manifold. Returning to our illustrative example, we can now describe one of its constraints more precisely. A simple grasp constraint can be described with h M (q)= x g − f pos,e (q) where f pos,e is the forward kinematics function of the robot end effector point e and x g is the grasp location on the mug. This constraint can be fulfilled for multiple robot configurations q, which correspond to different hand orientations that will affect the free configuration space for the subsequent tasks. For example, a mug grasped from the side will have a different free space during the transport phase than a mug grasped from the top. 17 3.2.1 Problem Formulation We formulate an optimization problem over a set of paths τ =(τ 1 ,...,τ n ) that minimizes the sum of path costs under the constraints of traversingM and of being collision-free. The planning on sequenced mani- folds problem is τ ⋆ = arg minτ n ∑ i=1 J(τ i ) s.t. τ 1 (0)= q start τ i (1)=τ i+1 (0) ∀ i=1,...,n− 1 C free,i+1 =ϒ(C free,i ,τ i (1)) ∀ i=1,...,n τ i (s)∈ C M i ∩C free,i ∀ i=1,...,n ∀ s∈[0,1] τ n (1)∈ C M n+1 ∩C free,n+1 (3.1) The second constraint ensures continuity such that the endpoint of a pathτ i (1) is the start point of the next path τ i+1 (0). The third constraint captures the change in the collision-free space defined by the operator ϒ . The last two constraints ensure that the path is collision free and on the corresponding manifolds. The endpoint of τ n must be on the goal manifold M n+1 , which denotes the successful completion of the task. Note that this problem can be seen as a special case of the multi-modal motion planning problem presented in [37] where the manifolds are provided in their sequential order. There are several advantages to formulating the problem with manifolds in this way. One is that it is not necessary to choose a specific target configuration in C and thus a wider range of goals can be described in the form of manifolds. Another is that it is possible to describe complex sequential tasks in a single planning problem, not requiring the specification of subgoal configurations. The algorithm proposed in Section 3.3 searches for a path that solves this optimization problem for the problem class described in the next section. 18 M 1 ∩ M 2 <latexit sha1_base64="NgBayo2fOdV8wxGPFhhsMOa1KqQ=">AAAC3HicZVHLbtNAFJ2YVwmPtrBESCOqSiys2GOTJtlVsIAuKpVH2kpNiMbj68TKeMaaGUMjyzt2iG238B38Sb+Bj4CJk1YkvZqRztw5956je6Ocp9r4/mXDuXX7zt17G/ebDx4+ery5tf3kWMtCMegzyaU6jagGngrom9RwOM0V0CzicBJN38z/T76A0qkUn8wsh2FGxyJNUkaNTQ0ORwQPGM3x4SgYbe34Lb8OfBOQJdjZf/73D7JxNNpu/B7EkhUZCMM41fqM+LkZllSZlHGomoNCQ07ZlI7hzEJBM9DDsjZd4V2biXEilb3C4Dr7f0VJM61nWWSZGTUTvf43T65IlOeLzs1VXZN0h2Uq8sKAYAvZpODYSDyfB45TBczwmQWUqdQ6x2xCFWXGTq25iwd1aen1tX17IMYcVK68jzPBPAUaqGITj0mRgLL9QXsToDGoloHzuZMYEruZ2liZzcYKQFTlh7evq7IXumQvdEO/WmdFSn69YpHenktC3+0FN2hSUTGGa17b7djTWReNqZpGvLji+ZbikqBb1dt+1Q66pIcXwG8vAQmvt30ctIjfIu/t2g/QIjbQM/QCvUQEddA+eoeOUB8xlKML9BP9cj4735zvzo8F1Wksa56ilXAu/gGAuOae</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="PX96YF1s0/K0PgYyWhG7vQXv65k=">AAACwnicZVJNbxMxEHWWr7IUaM9cLKpKHFZZOxFNc0NwgGMLpK3URJXXO7ux4rVXtpd2tVrEmSv/hn/Cv8HZpIgkI1t6Hr+ZeZqZpJTCOkL+9IIHDx89frL3NHy2Hz5/8fJg/8LqynCYcC21uUqYBSkUTJxwEq5KA6xIJFwmiw/L/8tvYKzQ6qurS5gVLFciE5w57zq7OTgifdIZ3gV0DY7Q2m4Oe7+nqeZVAcpxyay9pqR0s4YZJ7iENpxWFkrGFyyHaw8VK8DOmk5ni4+9J8WZNv4qhzvv/xENK6yti8QzC+bmdvtv6dwo0dytMoebdV12OmuEKisHiq/KZpXETuNlC3AqDHAnaw8YN8Irx3zODOPONyo8xtMutIkn1r9jULkEU5r4S614bMACM3wec60yMD4/2HgOLAXTd3C3VJJC5ofRCWuKOjcAqm0+f3zfNuNhRE+G0ZC026zE6Nt7Fh2fRHRIovFgh6YNUzn8472NRv6MtoumzCwSWd3ziKdEdHDa+mHT7dHugotBn5I+PSdoD71Cr9EbRNEIvUOf0BmaII5S9BP9Cs6D2+D7aimC3no7DtGGBT/+Al923dw=</latexit> <latexit sha1_base64="bKnre0Kw3Ly+17wgbxw6uTot9xY=">AAAC0XicZVJNbxMxEHWWrxIKtFy5WFSVOKyy60Q0za2CA1wqlY+0lboh8npnN1a89sr20q5We+OGuHKF38E/4d/gbFJEkpEtPY/fzDzNTFwIbmwY/ul4d+7eu/9g52H30e7jJ0/39nfPjSo1gzFTQunLmBoQXMLYcivgstBA81jARTx/s/i/+ALacCU/2aqASU4zyVPOqHWu6HRKcMRogU+n/eneQdgLW8PbgKzAAVrZ2XS/8ztKFCtzkJYJaswVCQs7qam2nAloulFpoKBsTjO4clDSHMykbkU3+NB5Epwq7a60uPX+H1HT3Jgqjx0zp3ZmNv8WzrUS9c0yc3e9rk2PJzWXRWlBsmXZtBTYKrzoB064BmZF5QBlmjvlmM2opsy6rnUPcdSG1sHYuHcAMhOgCx18rCQLNBigms0CpmQK2uUHE8yAJqB7Fm4WShJI3WRaYXVeZRpANvWHt6+bejTwydHAH4TNJivW6vqWRUZHPhmE/qi/RVOaygz+8V75Q3eGm0UTquexKG95oaP4pH/cuGmTzdlug/N+j4Q98j5EO+g5eoFeIoKG6AS9Q2dojBgq0A/0E/3yPntfvW/LvfA6qwV5htbM+/4XYofiww==</latexit> <latexit sha1_base64="bKnre0Kw3Ly+17wgbxw6uTot9xY=">AAAC0XicZVJNbxMxEHWWrxIKtFy5WFSVOKyy60Q0za2CA1wqlY+0lboh8npnN1a89sr20q5We+OGuHKF38E/4d/gbFJEkpEtPY/fzDzNTFwIbmwY/ul4d+7eu/9g52H30e7jJ0/39nfPjSo1gzFTQunLmBoQXMLYcivgstBA81jARTx/s/i/+ALacCU/2aqASU4zyVPOqHWu6HRKcMRogU+n/eneQdgLW8PbgKzAAVrZ2XS/8ztKFCtzkJYJaswVCQs7qam2nAloulFpoKBsTjO4clDSHMykbkU3+NB5Epwq7a60uPX+H1HT3Jgqjx0zp3ZmNv8WzrUS9c0yc3e9rk2PJzWXRWlBsmXZtBTYKrzoB064BmZF5QBlmjvlmM2opsy6rnUPcdSG1sHYuHcAMhOgCx18rCQLNBigms0CpmQK2uUHE8yAJqB7Fm4WShJI3WRaYXVeZRpANvWHt6+bejTwydHAH4TNJivW6vqWRUZHPhmE/qi/RVOaygz+8V75Q3eGm0UTquexKG95oaP4pH/cuGmTzdlug/N+j4Q98j5EO+g5eoFeIoKG6AS9Q2dojBgq0A/0E/3yPntfvW/LvfA6qwV5htbM+/4XYofiww==</latexit> <latexit sha1_base64="tNNuYwqrBnIfmIpoFZ/dnf+1EPo=">AAAC3HicZVFNbxMxEHWWrxI+2sKRi0VUicMqayeiaW4VHIBDpfKRtlITIq93dmPF613ZXuhqtTduiGuv8Dv4J/wbnE2KSDKypefxm5mneWEuhbGE/Gl5t27fuXtv5377wcNHj3f39p+cmazQHEY8k5m+CJkBKRSMrLASLnINLA0lnIfz14v/8y+gjcjUJ1vmMElZokQsOLMuNT6ZUjzmLMcn0950r0O6pAm8DegKdNAqTqf7rd/jKONFCspyyYy5pCS3k4ppK7iEuj0uDOSMz1kClw4qloKZVI3oGh+4TITjTLurLG6y/1dULDWmTEPHTJmdmc2/RXJtRHW17Nxen2vjo0klVF5YUHw5Ni4kthle7ANHQgO3snSAcS2ccsxnTDNu3dbaB3jclFbByLh3ACqRoHMdfCwVDzQYYJrPAp6pGLTrDyaYAYtAdy1cLZREEDtnGmFVWiYaQNXVhzev6mrY9+lh3++TepMV6uzrDYsOD33aJ/6wt0XLNFMJ/OO99AfuDDaHRkzPQ1nc8Iij+LR3VDu36aa32+Cs16WkS9+TzvG7le876Bl6jl4gigboGL1Fp2iEOMrRNfqJfnmfvW/ed+/Hkuq1VjVP0Vp4138BOsbkDg==</latexit> <latexit sha1_base64="tNNuYwqrBnIfmIpoFZ/dnf+1EPo=">AAAC3HicZVFNbxMxEHWWrxI+2sKRi0VUicMqayeiaW4VHIBDpfKRtlITIq93dmPF613ZXuhqtTduiGuv8Dv4J/wbnE2KSDKypefxm5mneWEuhbGE/Gl5t27fuXtv5377wcNHj3f39p+cmazQHEY8k5m+CJkBKRSMrLASLnINLA0lnIfz14v/8y+gjcjUJ1vmMElZokQsOLMuNT6ZUjzmLMcn0950r0O6pAm8DegKdNAqTqf7rd/jKONFCspyyYy5pCS3k4ppK7iEuj0uDOSMz1kClw4qloKZVI3oGh+4TITjTLurLG6y/1dULDWmTEPHTJmdmc2/RXJtRHW17Nxen2vjo0klVF5YUHw5Ni4kthle7ANHQgO3snSAcS2ccsxnTDNu3dbaB3jclFbByLh3ACqRoHMdfCwVDzQYYJrPAp6pGLTrDyaYAYtAdy1cLZREEDtnGmFVWiYaQNXVhzev6mrY9+lh3++TepMV6uzrDYsOD33aJ/6wt0XLNFMJ/OO99AfuDDaHRkzPQ1nc8Iij+LR3VDu36aa32+Cs16WkS9+TzvG7le876Bl6jl4gigboGL1Fp2iEOMrRNfqJfnmfvW/ed+/Hkuq1VjVP0Vp4138BOsbkDg==</latexit> <latexit sha1_base64="k0vw0215T61RBafW+o6PI93MoQg=">AAAC3HicZVHJbtswEKXVJam7Je2xCEA0CNCDYIk2Gse3ID10AQKki5MAkWtQ1EgmTFECSSURBN16K3rNtf2O/km/oR/R0nJS1PaABB6Hb2Ye5oW54Nr4/q+Wc+v2nbtr6/fa9x88fPR4Y/PJsc4KxWDIMpGp05BqEFzC0HAj4DRXQNNQwEk4fTX7PzkHpXkmP5kyh1FKE8ljzqixqeBwTHDAaI4Px93xxrbf8ZvAq4Bcg+39rT+/195dbB2NN1s/gyhjRQrSMEG1PiN+bkYVVYYzAXU7KDTklE1pAmcWSpqCHlWN6Brv2EyE40zZKw1usv9XVDTVukxDy0ypmejlv1lyYUR1Oe/cXpxr4r1RxWVeGJBsPjYuBDYZnu0DR1wBM6K0gDLFrXLMJlRRZuzW2js4aEorb6jt2wOZCFC58j6WknkKNFDFJh7LZAzK9gftTYBGoDoGLmdKIoitM42wKi0TBSDr6sPrg7oa9Fyy23N7fr3MClV2ccMig12X9Hx30F2hZYrKBP7xXrp9e/rLQyOqpqEobni+pbiku1dbt8myt6vguNshfoe8t7a/RfNYR8/Qc/QCEdRH++gNOkJDxFCOrtB39MP57Hxxvjrf5lSndV3zFC2Ec/UXZAznOg==</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="r0tjdI3cdsXjoIJfUmCMVZHz2A8=">AAAC3HicZVFNb9NAEN2YrxK+WjhyWRFV4mDFuzZpklsFB+BQqXykrdSEaL0eO1bWa2t3DY0s37ghrr3C7+Cf8G/YOGlF0tGu9Hb2zczTvLAQqTaE/G05t27fuXtv5377wcNHj5/s7j090XmpOIx4LnJ1FjINIpUwMqkRcFYoYFko4DScv1n+n34FpdNcfjaLAiYZS2Qap5wZmxofTSkec1bgo6k/3e2QLmkC3wR0DTpoHcfTvdafcZTzMgNpuGBan1NSmEnFlEm5gLo9LjUUjM9ZAucWSpaBnlSN6Brv20yE41zZKw1usv9XVCzTepGFlpkxM9Pbf8vkxojqYtW5vTnXxINJlcqiNCD5amxcCmxyvNwHjlIF3IiFBYyr1CrHfMYU48Zurb2Px01p5Y20fXsgEwGqUN6nheSeAg1M8ZnHcxmDsv1BezNgEaiugYulkghi60wjrMoWiQKQdfXx7eu6GgYuPQjcgNTbrFDl365YdHjg0oC4Q/8GLVdMJnDN67l9e/rbQyOm5qEor3jEUlzqD+rG7Vc9f0CHeAVIbw1ocO32id+lpEs/kM7h+7XvO+g5eoFeIor66BC9Q8dohDgq0CX6hX47X5zvzg/n54rqtNY1z9BGOJf/AFom5Fc=</latexit> q <latexit sha1_base64="XMvAoPXunJ445FbG0Ipf7vSVACw=">AAAC3nicZVFNb9NAEN2YrxI+2sKRy4qoEgcr9sakSW4VHIBb+UhbqQlhvR47VtZrd3cNtSxLnLghrr3Cz+DOj+DfsLHTiqSjXent2zczTzN+xmOlXfdvy7px89btO1t32/fuP3i4vbP76EiluWQwZilP5YlPFfBYwFjHmsNJJoEmPodjf/Fy+X/8GaSKU/FBFxlMExqJOIwZ1Yb6eDabaDjXpdJU6mq203G7bh34OiAr0DnonP35ihA6nO22fk+ClOUJCM04VeqUuJmelqZYzDhU7UmuIKNsQSM4NVDQBNS0rG1XeM8wAQ5Taa7QuGb/zyhpolSR+EaZUD1Xm39Lcq1Fed5Ubq/31eFwWsYiyzUI1rQNc451ipcTwUEsgWleGECZjI1zzOZUUqbN3Np7eFKnls5YmbcDIuIgM+m8LwRzJCigks0dlooQpKkPypkDDUB2zWCXTgIIzW5qY2VSRBJAVOW7Vy+qcuTZZN+zPbfaVPky/XKpIqN9m3iuPepdk6WSigiudH17YM5gs2lA5cLn+aXONRKb9IbNtp/3e0Mywg1w+ytAvKttH/W6xO2St2btb1ATW+gJeoqeIYIG6AC9RodojBiS6AL9RL+sT9Y367v1o5FarVXOY7QW1sU/pJjpEA==</latexit> <latexit sha1_base64="cH7cYK0f8dqvW9MvFVq4zWFFfKE=">AAAC3nicZVHLbtNAFJ24PEp4tbBkYxFVYmHFnpi8dhUsgF15uK3UhDAeXztWxmN3ZlxqWd6wYIfYdgtfwJo9H8HfMLHTiqRXM9KZM+fee3Svn7FYKsf52zK2bty8dXv7TvvuvfsPHu7sPjqUaS4oeDRlqTj2iQQWc/BUrBgcZwJI4jM48hcvl/9HZyBknPIPqshgmpCIx2FMidLUx9PZRMG5KqUiQlWznY7TdeowrwO8Ap39zumfL96vwcFst/V7EqQ0T4AryoiUJ9jJ1LTUxWLKoGpPcgkZoQsSwYmGnCQgp2VtuzL3NBOYYSr05cqs2f8zSpJIWSS+ViZEzeXm35Jca1GeN5Xb631VOJqWMc9yBZw2bcOcmSo1lxMxg1gAVazQgFARa+cmnRNBqNJza++Zkzq1tD2p3zbwiIHIhP2+4NQWIIEIOrdpykMQuj5Iew4kANHVg106CSDUu6mNlUkRCQBele9evajKsWvhgWu5TrWp8kX6+VKFxwMLu4417l2TpYLwCK50fWuoz3CzaUDEwmf5pc7REgv3Rs22n/d7Izw2G+D0VwC7V9s+7HWx08Vv9drfoCa20RP0FD1DGA3RPnqNDpCHKBLoAv1AP41Pxlfjm/G9kRqtVc5jtBbGxT9s6+pN</latexit> <latexit sha1_base64="cH7cYK0f8dqvW9MvFVq4zWFFfKE=">AAAC3nicZVHLbtNAFJ24PEp4tbBkYxFVYmHFnpi8dhUsgF15uK3UhDAeXztWxmN3ZlxqWd6wYIfYdgtfwJo9H8HfMLHTiqRXM9KZM+fee3Svn7FYKsf52zK2bty8dXv7TvvuvfsPHu7sPjqUaS4oeDRlqTj2iQQWc/BUrBgcZwJI4jM48hcvl/9HZyBknPIPqshgmpCIx2FMidLUx9PZRMG5KqUiQlWznY7TdeowrwO8Ap39zumfL96vwcFst/V7EqQ0T4AryoiUJ9jJ1LTUxWLKoGpPcgkZoQsSwYmGnCQgp2VtuzL3NBOYYSr05cqs2f8zSpJIWSS+ViZEzeXm35Jca1GeN5Xb631VOJqWMc9yBZw2bcOcmSo1lxMxg1gAVazQgFARa+cmnRNBqNJza++Zkzq1tD2p3zbwiIHIhP2+4NQWIIEIOrdpykMQuj5Iew4kANHVg106CSDUu6mNlUkRCQBele9evajKsWvhgWu5TrWp8kX6+VKFxwMLu4417l2TpYLwCK50fWuoz3CzaUDEwmf5pc7REgv3Rs22n/d7Izw2G+D0VwC7V9s+7HWx08Vv9drfoCa20RP0FD1DGA3RPnqNDpCHKBLoAv1AP41Pxlfjm/G9kRqtVc5jtBbGxT9s6+pN</latexit> <latexit sha1_base64="cH7cYK0f8dqvW9MvFVq4zWFFfKE=">AAAC3nicZVHLbtNAFJ24PEp4tbBkYxFVYmHFnpi8dhUsgF15uK3UhDAeXztWxmN3ZlxqWd6wYIfYdgtfwJo9H8HfMLHTiqRXM9KZM+fee3Svn7FYKsf52zK2bty8dXv7TvvuvfsPHu7sPjqUaS4oeDRlqTj2iQQWc/BUrBgcZwJI4jM48hcvl/9HZyBknPIPqshgmpCIx2FMidLUx9PZRMG5KqUiQlWznY7TdeowrwO8Ap39zumfL96vwcFst/V7EqQ0T4AryoiUJ9jJ1LTUxWLKoGpPcgkZoQsSwYmGnCQgp2VtuzL3NBOYYSr05cqs2f8zSpJIWSS+ViZEzeXm35Jca1GeN5Xb631VOJqWMc9yBZw2bcOcmSo1lxMxg1gAVazQgFARa+cmnRNBqNJza++Zkzq1tD2p3zbwiIHIhP2+4NQWIIEIOrdpykMQuj5Iew4kANHVg106CSDUu6mNlUkRCQBele9evajKsWvhgWu5TrWp8kX6+VKFxwMLu4417l2TpYLwCK50fWuoz3CzaUDEwmf5pc7REgv3Rs22n/d7Izw2G+D0VwC7V9s+7HWx08Vv9drfoCa20RP0FD1DGA3RPnqNDpCHKBLoAv1AP41Pxlfjm/G9kRqtVc5jtBbGxT9s6+pN</latexit> <latexit sha1_base64="cH7cYK0f8dqvW9MvFVq4zWFFfKE=">AAAC3nicZVHLbtNAFJ24PEp4tbBkYxFVYmHFnpi8dhUsgF15uK3UhDAeXztWxmN3ZlxqWd6wYIfYdgtfwJo9H8HfMLHTiqRXM9KZM+fee3Svn7FYKsf52zK2bty8dXv7TvvuvfsPHu7sPjqUaS4oeDRlqTj2iQQWc/BUrBgcZwJI4jM48hcvl/9HZyBknPIPqshgmpCIx2FMidLUx9PZRMG5KqUiQlWznY7TdeowrwO8Ap39zumfL96vwcFst/V7EqQ0T4AryoiUJ9jJ1LTUxWLKoGpPcgkZoQsSwYmGnCQgp2VtuzL3NBOYYSr05cqs2f8zSpJIWSS+ViZEzeXm35Jca1GeN5Xb631VOJqWMc9yBZw2bcOcmSo1lxMxg1gAVazQgFARa+cmnRNBqNJza++Zkzq1tD2p3zbwiIHIhP2+4NQWIIEIOrdpykMQuj5Iew4kANHVg106CSDUu6mNlUkRCQBele9evajKsWvhgWu5TrWp8kX6+VKFxwMLu4417l2TpYLwCK50fWuoz3CzaUDEwmf5pc7REgv3Rs22n/d7Izw2G+D0VwC7V9s+7HWx08Vv9drfoCa20RP0FD1DGA3RPnqNDpCHKBLoAv1AP41Pxlfjm/G9kRqtVc5jtBbGxT9s6+pN</latexit> <latexit sha1_base64="MiNMncp/PqoAEfYptbFDVwsFjAE=">AAAC3nicZVFNb9NAEN2YrxK+WjhysYgqcbDi3Zg0ya2CA3ArH2krNSGs12PHynptdtdQy/KVG+LaK/wM/gn/ho3tViQd7Upv376ZeZrxMx4rjfHfjnXj5q3bd3budu/df/Dw0e7e42OV5pLBlKU8lac+VcBjAVMdaw6nmQSa+BxO/NWr9f/JV5AqTsVHXWQwT2gk4jBmVBvq05fFTMO5LpWmUleL3R7u4zrs64C0oIfaOFrsdf7MgpTlCQjNOFXqjOBMz0tTLGYcqu4sV5BRtqIRnBkoaAJqXta2K3vfMIEdptJcoe2a/T+jpIlSReIbZUL1Um3/rcmNFuV5U7m72VeH43kZiyzXIFjTNsy5rVN7PRE7iCUwzQsDKJOxcW6zJZWUaTO37r49q1NLd6rM2wURcZCZdD8UgrkSFFDJli5LRQjS1AflLoEGIPtmsGsnAYRmN7WxMikiCSCq8v3rl1U58Rxy4DkerrZVvky/XarI5MAhHnYmg2uyVFIRwZVu6IzMGW03Dahc+Ty/1GEjcchg3Gz7xXAwJhO7AXjYAuJdbft40Ce4T97h3uHbdu876Cl6hp4jgkboEL1BR2iKGJLoAv1Cv63P1nfrh/WzkVqdNucJ2gjr4h/m5Ob3</latexit> q <latexit sha1_base64="reOk1BPWUjuxijIhs3JFpavkt4g=">AAAC3XicZVFNb9NAEN2YrxI+2sKRi0VUiYMVe2PSJLcKDsCtfKSt1ETRej12rKx3nd011LIsceGGuPYKf4M7P4J/w8ZOK5KOdqW3b9/MG80EGUuU9ry/LevW7Tt37+3cbz94+Ojx7t7+kxMlcklhTAUT8iwgCljCYawTzeAsk0DSgMFpsHi9+j/9DFIlgn/SRQbTlMQ8iRJKtKGmy9lEw4UupRC6mu11vK5Xh30T4DXoHHWWf74ihI5n+63fk1DQPAWuKSNKnWMv09OSSJ1QBlV7kivICF2QGM4N5CQFNS3rriv7wDChHQlpLtd2zf6fUZJUqSINjDIleq62/1bkhkV50VRub/rqaDgtE57lGjhtbKOc2VrYq4HYYSKBalYYQKhMTOc2nRNJqDZjax/Ykzq1dMfKvF3gMQOZSfdjwakrQQGRdO5SwSOQpj4odw4kBNk1c111EkJkVlM3VqZFLAF4VX5486oqR76DD33H96ptVSDFlysVHh062PecUe+GTEjCY7jW9Z2BOYNt05DIRcDyK51nJA7uDZttv+z3hnhkN8DrrwH2r7d90utir4vfm7W/Q03soGfoOXqBMBqgI/QWHaMxomiJLtFP9MuaWd+s79aPRmq11jlP0UZYl/8Ad/PonA==</latexit> <latexit sha1_base64="z4fmbJQ0cWtL2afIr6Nyx3XEojU=">AAAC3XicZVHLbtNAFJ24PEp4tWXJxiKqxMKKPQl57SpYALvycFupiaLx+NqxMp5xZsZQy/IGiR1i2y38AWv2fAR/wyROK5JezUhnzpx7z9W9QcYSpT3vb8PauXX7zt3de837Dx4+ery3f3CiRC4p+FQwIc8CooAlHHydaAZnmQSSBgxOg/mr5f/pJ5AqEfyjLjKYpCTmSZRQog01WUzHGi50KYXQ1XSv5bW9Vdg3AV6D1lFr8eeL/6t/PN1v/B6HguYpcE0ZUeoce5melETqhDKomuNcQUbonMRwbiAnKahJueq6sg8NE9qRkOZyba/Y/zNKkipVpIFRpkTP1PbfktywKC/qys1NXx0NJ2XCs1wDp7VtlDNbC3s5EDtMJFDNCgMIlYnp3KYzIgnVZmzNQ3u8Si1dX5m3CzxmIDPpfig4dSUoIJLOXCp4BNLUB+XOgIQg22auy05CiMxqVo2VaRFLAF6V71+/rMpR18H9rtP1qm1VIMXnKxUe9R3c9ZxR54ZMSMJjuNb1nIE5g23TkMh5wPIrnWckDu4M622/6HWGeGTXwOutAe5eb/uk08ZeG78za3+L6thFT9Ez9BxhNEBH6A06Rj6iaIEu0Q/005paX61v1vdaajXWOU/QRliX/wBARunZ</latexit> <latexit sha1_base64="z4fmbJQ0cWtL2afIr6Nyx3XEojU=">AAAC3XicZVHLbtNAFJ24PEp4tWXJxiKqxMKKPQl57SpYALvycFupiaLx+NqxMp5xZsZQy/IGiR1i2y38AWv2fAR/wyROK5JezUhnzpx7z9W9QcYSpT3vb8PauXX7zt3de837Dx4+ery3f3CiRC4p+FQwIc8CooAlHHydaAZnmQSSBgxOg/mr5f/pJ5AqEfyjLjKYpCTmSZRQog01WUzHGi50KYXQ1XSv5bW9Vdg3AV6D1lFr8eeL/6t/PN1v/B6HguYpcE0ZUeoce5melETqhDKomuNcQUbonMRwbiAnKahJueq6sg8NE9qRkOZyba/Y/zNKkipVpIFRpkTP1PbfktywKC/qys1NXx0NJ2XCs1wDp7VtlDNbC3s5EDtMJFDNCgMIlYnp3KYzIgnVZmzNQ3u8Si1dX5m3CzxmIDPpfig4dSUoIJLOXCp4BNLUB+XOgIQg22auy05CiMxqVo2VaRFLAF6V71+/rMpR18H9rtP1qm1VIMXnKxUe9R3c9ZxR54ZMSMJjuNb1nIE5g23TkMh5wPIrnWckDu4M622/6HWGeGTXwOutAe5eb/uk08ZeG78za3+L6thFT9Ez9BxhNEBH6A06Rj6iaIEu0Q/005paX61v1vdaajXWOU/QRliX/wBARunZ</latexit> <latexit sha1_base64="z4fmbJQ0cWtL2afIr6Nyx3XEojU=">AAAC3XicZVHLbtNAFJ24PEp4tWXJxiKqxMKKPQl57SpYALvycFupiaLx+NqxMp5xZsZQy/IGiR1i2y38AWv2fAR/wyROK5JezUhnzpx7z9W9QcYSpT3vb8PauXX7zt3de837Dx4+ery3f3CiRC4p+FQwIc8CooAlHHydaAZnmQSSBgxOg/mr5f/pJ5AqEfyjLjKYpCTmSZRQog01WUzHGi50KYXQ1XSv5bW9Vdg3AV6D1lFr8eeL/6t/PN1v/B6HguYpcE0ZUeoce5melETqhDKomuNcQUbonMRwbiAnKahJueq6sg8NE9qRkOZyba/Y/zNKkipVpIFRpkTP1PbfktywKC/qys1NXx0NJ2XCs1wDp7VtlDNbC3s5EDtMJFDNCgMIlYnp3KYzIgnVZmzNQ3u8Si1dX5m3CzxmIDPpfig4dSUoIJLOXCp4BNLUB+XOgIQg22auy05CiMxqVo2VaRFLAF6V71+/rMpR18H9rtP1qm1VIMXnKxUe9R3c9ZxR54ZMSMJjuNb1nIE5g23TkMh5wPIrnWckDu4M622/6HWGeGTXwOutAe5eb/uk08ZeG78za3+L6thFT9Ez9BxhNEBH6A06Rj6iaIEu0Q/005paX61v1vdaajXWOU/QRliX/wBARunZ</latexit> <latexit sha1_base64="z4fmbJQ0cWtL2afIr6Nyx3XEojU=">AAAC3XicZVHLbtNAFJ24PEp4tWXJxiKqxMKKPQl57SpYALvycFupiaLx+NqxMp5xZsZQy/IGiR1i2y38AWv2fAR/wyROK5JezUhnzpx7z9W9QcYSpT3vb8PauXX7zt3de837Dx4+ery3f3CiRC4p+FQwIc8CooAlHHydaAZnmQSSBgxOg/mr5f/pJ5AqEfyjLjKYpCTmSZRQog01WUzHGi50KYXQ1XSv5bW9Vdg3AV6D1lFr8eeL/6t/PN1v/B6HguYpcE0ZUeoce5melETqhDKomuNcQUbonMRwbiAnKahJueq6sg8NE9qRkOZyba/Y/zNKkipVpIFRpkTP1PbfktywKC/qys1NXx0NJ2XCs1wDp7VtlDNbC3s5EDtMJFDNCgMIlYnp3KYzIgnVZmzNQ3u8Si1dX5m3CzxmIDPpfig4dSUoIJLOXCp4BNLUB+XOgIQg22auy05CiMxqVo2VaRFLAF6V71+/rMpR18H9rtP1qm1VIMXnKxUe9R3c9ZxR54ZMSMJjuNb1nIE5g23TkMh5wPIrnWckDu4M622/6HWGeGTXwOutAe5eb/uk08ZeG78za3+L6thFT9Ez9BxhNEBH6A06Rj6iaIEu0Q/005paX61v1vdaajXWOU/QRliX/wBARunZ</latexit> <latexit sha1_base64="F7t/f0FgGwZfBcERrc9qhTWZa0E=">AAAC3XicZVHLbtswEKTVV+o+krTHXoQaAXoQLNKuY/sWtIe2t/ThJEBsGBS1kgVTpEJSbQRBx96KXnNtf6N/0r8pbSlB7SxIYDic3VnsBhlPtMH4b8u5c/fe/Qc7D9uPHj95uru3/+xEy1wxmDDJpToLqAaeCJiYxHA4yxTQNOBwGizfrv5Pv4LSiRRfTJHBLKWxSKKEUWOp2cV8auDSlEpKU833OriL1+HeBqQBHdTE8Xy/9WcaSpanIAzjVOtzgjMzK6kyCeNQtae5hoyyJY3h3EJBU9Czct115R5YJnQjqewVxl2z/2eUNNW6SAOrTKlZ6O2/FblhUV7WldubviYazcpEZLkBwWrbKOeuke5qIG6YKGCGFxZQphLbucsWVFFm7NjaB+50nVr6E23fPoiYg8qU/7kQzFeggSq28JkUEShbH7S/ABqC6tq5rjoJIbKrWTdWpkWsAERVfnr3pirHfY8c9r0+rrZVgZLfrlVkfOiRPvbGvVsyqaiI4UY38Ib2DLdNQ6qWAc+vddhKPNIb1dt+PeiNyNitAR40gPRvtn3S6xLcJR9x5+hDs/cd9AK9RK8QQUN0hN6jYzRBDF2gK/QL/Xbmznfnh/OzljqtJuc52gjn6h+6P+aD</latexit> M 2 ∩ M 3 <latexit sha1_base64="t7alCmBpTyrreIV6oLqCwCvs8hQ=">AAAC3HicZVHLbtNAFJ2YVwmPtrBESCOqSiys2GOTJtlVsIAuKpVH2kp1iMbja8fKeGzNjKGR5R07xLZb+A7+pN/AR8DESSuSXs1IZ+6ce8/RvWHBU6Vd97Jl3bp95+69jfvtBw8fPd7c2n5yrPJSMhiynOfyNKQKeCpgqFPN4bSQQLOQw0k4fTP/P/kCUqW5+KRnBYwymog0ThnVJhUcjj0cMFrgw7E/3tpxO24T+CYgS7Cz//zvH2TiaLzd+h1EOSszEJpxqtQZcQs9qqjUKeNQt4NSQUHZlCZwZqCgGahR1Ziu8a7JRDjOpblC4yb7f0VFM6VmWWiYGdUTtf43T65IVOeLzu1VXR33R1UqilKDYAvZuORY53g+DxylEpjmMwMok6lxjtmESsq0mVp7FwdNaeUMlXk7IBIOspDOx5lgjgQFVLKJw3IRgzT9QTkToBHIjobzuZMIYrOZxliVzRIJIOrqw9vXdTXwbbLn275br7NCmX+9YpHBnk181x54N2i5pCKBa17X7pnTWxeNqJyGvLziuYZiE69fN9t+1fX6ZIAXwO0uAfGvt33sdYjbIe/N2g/QIjbQM/QCvUQE9dA+eoeO0BAxVKAL9BP9sj5b36zv1o8F1Wota56ilbAu/gGFm+ag</latexit> <latexit sha1_base64="MneRyM+x4BsitJMXMpMEcJF3b1Y=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0xaZJdBQseUqXySFupCdF4fJ1YGY+tmXFby/KOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dI+f8khp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWSSQYDlvBEnvpUAY8EDHSkOZymEmjsczjxZ6/n/yfnIFWUiM86T2EU04mIwohRbVLDw3EbDxlN8eHYG2/tui23CnwbkAXYPdj5+2f9/cXO0Xi78WsYJCyLQWjGqVJnxE31qKBSR4xD2RxmClLKZnQCZwYKGoMaFZXoEu+ZTIDDRJorNK6y/1cUNFYqj33DjKmeqtW/eXJpRHFZd24uz9Vhb1REIs00CFaPDTOOdYLn+8BBJIFpnhtAmYyMcsymVFKmzdaae3hYlRbOQJm3A2LCQabS+ZQL5khQQCWbOiwRIUjTH5QzBRqAbGm4nCsJIDTOVMKKOJ9IAFEWH9+8Kou+Z5N9z/bccpXly+TimkX6+zbxXLvfvkVLJBUTuOF17K453dWhAZUzn2fXPNdQbNLulZXbLzvtHunjGridBSDejdvH7RZxW+SDsf0dqmMDPUPP0QtEUBcdoLfoCA0QQym6Qj/QT+uL9dX6Zn2vqVZjUfMULYV19Q+IT+eF</latexit> <latexit sha1_base64="MneRyM+x4BsitJMXMpMEcJF3b1Y=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0xaZJdBQseUqXySFupCdF4fJ1YGY+tmXFby/KOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dI+f8khp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWSSQYDlvBEnvpUAY8EDHSkOZymEmjsczjxZ6/n/yfnIFWUiM86T2EU04mIwohRbVLDw3EbDxlN8eHYG2/tui23CnwbkAXYPdj5+2f9/cXO0Xi78WsYJCyLQWjGqVJnxE31qKBSR4xD2RxmClLKZnQCZwYKGoMaFZXoEu+ZTIDDRJorNK6y/1cUNFYqj33DjKmeqtW/eXJpRHFZd24uz9Vhb1REIs00CFaPDTOOdYLn+8BBJIFpnhtAmYyMcsymVFKmzdaae3hYlRbOQJm3A2LCQabS+ZQL5khQQCWbOiwRIUjTH5QzBRqAbGm4nCsJIDTOVMKKOJ9IAFEWH9+8Kou+Z5N9z/bccpXly+TimkX6+zbxXLvfvkVLJBUTuOF17K453dWhAZUzn2fXPNdQbNLulZXbLzvtHunjGridBSDejdvH7RZxW+SDsf0dqmMDPUPP0QtEUBcdoLfoCA0QQym6Qj/QT+uL9dX6Zn2vqVZjUfMULYV19Q+IT+eF</latexit> <latexit sha1_base64="MneRyM+x4BsitJMXMpMEcJF3b1Y=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0xaZJdBQseUqXySFupCdF4fJ1YGY+tmXFby/KOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dI+f8khp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWSSQYDlvBEnvpUAY8EDHSkOZymEmjsczjxZ6/n/yfnIFWUiM86T2EU04mIwohRbVLDw3EbDxlN8eHYG2/tui23CnwbkAXYPdj5+2f9/cXO0Xi78WsYJCyLQWjGqVJnxE31qKBSR4xD2RxmClLKZnQCZwYKGoMaFZXoEu+ZTIDDRJorNK6y/1cUNFYqj33DjKmeqtW/eXJpRHFZd24uz9Vhb1REIs00CFaPDTOOdYLn+8BBJIFpnhtAmYyMcsymVFKmzdaae3hYlRbOQJm3A2LCQabS+ZQL5khQQCWbOiwRIUjTH5QzBRqAbGm4nCsJIDTOVMKKOJ9IAFEWH9+8Kou+Z5N9z/bccpXly+TimkX6+zbxXLvfvkVLJBUTuOF17K453dWhAZUzn2fXPNdQbNLulZXbLzvtHunjGridBSDejdvH7RZxW+SDsf0dqmMDPUPP0QtEUBcdoLfoCA0QQym6Qj/QT+uL9dX6Zn2vqVZjUfMULYV19Q+IT+eF</latexit> <latexit sha1_base64="MneRyM+x4BsitJMXMpMEcJF3b1Y=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0xaZJdBQseUqXySFupCdF4fJ1YGY+tmXFby/KOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dI+f8khp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWSSQYDlvBEnvpUAY8EDHSkOZymEmjsczjxZ6/n/yfnIFWUiM86T2EU04mIwohRbVLDw3EbDxlN8eHYG2/tui23CnwbkAXYPdj5+2f9/cXO0Xi78WsYJCyLQWjGqVJnxE31qKBSR4xD2RxmClLKZnQCZwYKGoMaFZXoEu+ZTIDDRJorNK6y/1cUNFYqj33DjKmeqtW/eXJpRHFZd24uz9Vhb1REIs00CFaPDTOOdYLn+8BBJIFpnhtAmYyMcsymVFKmzdaae3hYlRbOQJm3A2LCQabS+ZQL5khQQCWbOiwRIUjTH5QzBRqAbGm4nCsJIDTOVMKKOJ9IAFEWH9+8Kou+Z5N9z/bccpXly+TimkX6+zbxXLvfvkVLJBUTuOF17K453dWhAZUzn2fXPNdQbNLulZXbLzvtHunjGridBSDejdvH7RZxW+SDsf0dqmMDPUPP0QtEUBcdoLfoCA0QQym6Qj/QT+uL9dX6Zn2vqVZjUfMULYV19Q+IT+eF</latexit> <latexit sha1_base64="jNmz3ZVofQqvBVtogC0x1SEokic=">AAAC3HicZVFNb9NAEN2YrxK+WjhyWRFV4mDFuzFpklsFB+BQqXykrVSHaL0eJ1bWa2t3DY0s37ghrr3C7+Cf8G/YOG5F0tGu9Hb2zczTvDAXiTaE/G05t27fuXtv5377wcNHj5/s7j090VmhOIx5JjJ1FjINIpEwNokRcJYrYGko4DRcvFn9n34FpZNMfjbLHCYpm8kkTjgzNhUcTXs44CzHR1N/utshXVIHvgloAzqoiePpXutPEGW8SEEaLpjW55TkZlIyZRIuoGoHhYac8QWbwbmFkqWgJ2UtusL7NhPhOFP2SoPr7P8VJUu1XqahZabMzPX23yq5MaK8WHdub8418XBSJjIvDEi+HhsXApsMr/aBo0QBN2JpAeMqscoxnzPFuLFba+/joC4tvbG2bw/kTIDKlfdpKbmnQANTfO7xTMagbH/Q3hxYBKpr4GKlJILYOlMLK9PlTAHIqvz49nVVjnyXHviuT6ptVqiyb1csOjpwqU/cUe8GLVNMzuCa13cH9gy2h0ZMLUJRXPGIpbi0N6xqt1/1e0M6wmtA+g2g/rXbJ70uJV36gXQO3ze+76Dn6AV6iSgaoEP0Dh2jMeIoR5foF/rtfHG+Oz+cn2uq02pqnqGNcC7/AV8J5Fk=</latexit> M 1 ∩ M 2 <latexit sha1_base64="NgBayo2fOdV8wxGPFhhsMOa1KqQ=">AAAC3HicZVHLbtNAFJ2YVwmPtrBESCOqSiys2GOTJtlVsIAuKpVH2kpNiMbj68TKeMaaGUMjyzt2iG238B38Sb+Bj4CJk1YkvZqRztw5956je6Ocp9r4/mXDuXX7zt17G/ebDx4+ery5tf3kWMtCMegzyaU6jagGngrom9RwOM0V0CzicBJN38z/T76A0qkUn8wsh2FGxyJNUkaNTQ0ORwQPGM3x4SgYbe34Lb8OfBOQJdjZf/73D7JxNNpu/B7EkhUZCMM41fqM+LkZllSZlHGomoNCQ07ZlI7hzEJBM9DDsjZd4V2biXEilb3C4Dr7f0VJM61nWWSZGTUTvf43T65IlOeLzs1VXZN0h2Uq8sKAYAvZpODYSDyfB45TBczwmQWUqdQ6x2xCFWXGTq25iwd1aen1tX17IMYcVK68jzPBPAUaqGITj0mRgLL9QXsToDGoloHzuZMYEruZ2liZzcYKQFTlh7evq7IXumQvdEO/WmdFSn69YpHenktC3+0FN2hSUTGGa17b7djTWReNqZpGvLji+ZbikqBb1dt+1Q66pIcXwG8vAQmvt30ctIjfIu/t2g/QIjbQM/QCvUQEddA+eoeOUB8xlKML9BP9cj4735zvzo8F1Wksa56ilXAu/gGAuOae</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="PX96YF1s0/K0PgYyWhG7vQXv65k=">AAACwnicZVJNbxMxEHWWr7IUaM9cLKpKHFZZOxFNc0NwgGMLpK3URJXXO7ux4rVXtpd2tVrEmSv/hn/Cv8HZpIgkI1t6Hr+ZeZqZpJTCOkL+9IIHDx89frL3NHy2Hz5/8fJg/8LqynCYcC21uUqYBSkUTJxwEq5KA6xIJFwmiw/L/8tvYKzQ6qurS5gVLFciE5w57zq7OTgifdIZ3gV0DY7Q2m4Oe7+nqeZVAcpxyay9pqR0s4YZJ7iENpxWFkrGFyyHaw8VK8DOmk5ni4+9J8WZNv4qhzvv/xENK6yti8QzC+bmdvtv6dwo0dytMoebdV12OmuEKisHiq/KZpXETuNlC3AqDHAnaw8YN8Irx3zODOPONyo8xtMutIkn1r9jULkEU5r4S614bMACM3wec60yMD4/2HgOLAXTd3C3VJJC5ofRCWuKOjcAqm0+f3zfNuNhRE+G0ZC026zE6Nt7Fh2fRHRIovFgh6YNUzn8472NRv6MtoumzCwSWd3ziKdEdHDa+mHT7dHugotBn5I+PSdoD71Cr9EbRNEIvUOf0BmaII5S9BP9Cs6D2+D7aimC3no7DtGGBT/+Al923dw=</latexit> <latexit sha1_base64="bKnre0Kw3Ly+17wgbxw6uTot9xY=">AAAC0XicZVJNbxMxEHWWrxIKtFy5WFSVOKyy60Q0za2CA1wqlY+0lboh8npnN1a89sr20q5We+OGuHKF38E/4d/gbFJEkpEtPY/fzDzNTFwIbmwY/ul4d+7eu/9g52H30e7jJ0/39nfPjSo1gzFTQunLmBoQXMLYcivgstBA81jARTx/s/i/+ALacCU/2aqASU4zyVPOqHWu6HRKcMRogU+n/eneQdgLW8PbgKzAAVrZ2XS/8ztKFCtzkJYJaswVCQs7qam2nAloulFpoKBsTjO4clDSHMykbkU3+NB5Epwq7a60uPX+H1HT3Jgqjx0zp3ZmNv8WzrUS9c0yc3e9rk2PJzWXRWlBsmXZtBTYKrzoB064BmZF5QBlmjvlmM2opsy6rnUPcdSG1sHYuHcAMhOgCx18rCQLNBigms0CpmQK2uUHE8yAJqB7Fm4WShJI3WRaYXVeZRpANvWHt6+bejTwydHAH4TNJivW6vqWRUZHPhmE/qi/RVOaygz+8V75Q3eGm0UTquexKG95oaP4pH/cuGmTzdlug/N+j4Q98j5EO+g5eoFeIoKG6AS9Q2dojBgq0A/0E/3yPntfvW/LvfA6qwV5htbM+/4XYofiww==</latexit> <latexit sha1_base64="bKnre0Kw3Ly+17wgbxw6uTot9xY=">AAAC0XicZVJNbxMxEHWWrxIKtFy5WFSVOKyy60Q0za2CA1wqlY+0lboh8npnN1a89sr20q5We+OGuHKF38E/4d/gbFJEkpEtPY/fzDzNTFwIbmwY/ul4d+7eu/9g52H30e7jJ0/39nfPjSo1gzFTQunLmBoQXMLYcivgstBA81jARTx/s/i/+ALacCU/2aqASU4zyVPOqHWu6HRKcMRogU+n/eneQdgLW8PbgKzAAVrZ2XS/8ztKFCtzkJYJaswVCQs7qam2nAloulFpoKBsTjO4clDSHMykbkU3+NB5Epwq7a60uPX+H1HT3Jgqjx0zp3ZmNv8WzrUS9c0yc3e9rk2PJzWXRWlBsmXZtBTYKrzoB064BmZF5QBlmjvlmM2opsy6rnUPcdSG1sHYuHcAMhOgCx18rCQLNBigms0CpmQK2uUHE8yAJqB7Fm4WShJI3WRaYXVeZRpANvWHt6+bejTwydHAH4TNJivW6vqWRUZHPhmE/qi/RVOaygz+8V75Q3eGm0UTquexKG95oaP4pH/cuGmTzdlug/N+j4Q98j5EO+g5eoFeIoKG6AS9Q2dojBgq0A/0E/3yPntfvW/LvfA6qwV5htbM+/4XYofiww==</latexit> <latexit sha1_base64="tNNuYwqrBnIfmIpoFZ/dnf+1EPo=">AAAC3HicZVFNbxMxEHWWrxI+2sKRi0VUicMqayeiaW4VHIBDpfKRtlITIq93dmPF613ZXuhqtTduiGuv8Dv4J/wbnE2KSDKypefxm5mneWEuhbGE/Gl5t27fuXtv5377wcNHj3f39p+cmazQHEY8k5m+CJkBKRSMrLASLnINLA0lnIfz14v/8y+gjcjUJ1vmMElZokQsOLMuNT6ZUjzmLMcn0950r0O6pAm8DegKdNAqTqf7rd/jKONFCspyyYy5pCS3k4ppK7iEuj0uDOSMz1kClw4qloKZVI3oGh+4TITjTLurLG6y/1dULDWmTEPHTJmdmc2/RXJtRHW17Nxen2vjo0klVF5YUHw5Ni4kthle7ANHQgO3snSAcS2ccsxnTDNu3dbaB3jclFbByLh3ACqRoHMdfCwVDzQYYJrPAp6pGLTrDyaYAYtAdy1cLZREEDtnGmFVWiYaQNXVhzev6mrY9+lh3++TepMV6uzrDYsOD33aJ/6wt0XLNFMJ/OO99AfuDDaHRkzPQ1nc8Iij+LR3VDu36aa32+Cs16WkS9+TzvG7le876Bl6jl4gigboGL1Fp2iEOMrRNfqJfnmfvW/ed+/Hkuq1VjVP0Vp4138BOsbkDg==</latexit> <latexit sha1_base64="tNNuYwqrBnIfmIpoFZ/dnf+1EPo=">AAAC3HicZVFNbxMxEHWWrxI+2sKRi0VUicMqayeiaW4VHIBDpfKRtlITIq93dmPF613ZXuhqtTduiGuv8Dv4J/wbnE2KSDKypefxm5mneWEuhbGE/Gl5t27fuXtv5377wcNHj3f39p+cmazQHEY8k5m+CJkBKRSMrLASLnINLA0lnIfz14v/8y+gjcjUJ1vmMElZokQsOLMuNT6ZUjzmLMcn0950r0O6pAm8DegKdNAqTqf7rd/jKONFCspyyYy5pCS3k4ppK7iEuj0uDOSMz1kClw4qloKZVI3oGh+4TITjTLurLG6y/1dULDWmTEPHTJmdmc2/RXJtRHW17Nxen2vjo0klVF5YUHw5Ni4kthle7ANHQgO3snSAcS2ccsxnTDNu3dbaB3jclFbByLh3ACqRoHMdfCwVDzQYYJrPAp6pGLTrDyaYAYtAdy1cLZREEDtnGmFVWiYaQNXVhzev6mrY9+lh3++TepMV6uzrDYsOD33aJ/6wt0XLNFMJ/OO99AfuDDaHRkzPQ1nc8Iij+LR3VDu36aa32+Cs16WkS9+TzvG7le876Bl6jl4gigboGL1Fp2iEOMrRNfqJfnmfvW/ed+/Hkuq1VjVP0Vp4138BOsbkDg==</latexit> <latexit sha1_base64="k0vw0215T61RBafW+o6PI93MoQg=">AAAC3HicZVHJbtswEKXVJam7Je2xCEA0CNCDYIk2Gse3ID10AQKki5MAkWtQ1EgmTFECSSURBN16K3rNtf2O/km/oR/R0nJS1PaABB6Hb2Ye5oW54Nr4/q+Wc+v2nbtr6/fa9x88fPR4Y/PJsc4KxWDIMpGp05BqEFzC0HAj4DRXQNNQwEk4fTX7PzkHpXkmP5kyh1FKE8ljzqixqeBwTHDAaI4Px93xxrbf8ZvAq4Bcg+39rT+/195dbB2NN1s/gyhjRQrSMEG1PiN+bkYVVYYzAXU7KDTklE1pAmcWSpqCHlWN6Brv2EyE40zZKw1usv9XVDTVukxDy0ypmejlv1lyYUR1Oe/cXpxr4r1RxWVeGJBsPjYuBDYZnu0DR1wBM6K0gDLFrXLMJlRRZuzW2js4aEorb6jt2wOZCFC58j6WknkKNFDFJh7LZAzK9gftTYBGoDoGLmdKIoitM42wKi0TBSDr6sPrg7oa9Fyy23N7fr3MClV2ccMig12X9Hx30F2hZYrKBP7xXrp9e/rLQyOqpqEobni+pbiku1dbt8myt6vguNshfoe8t7a/RfNYR8/Qc/QCEdRH++gNOkJDxFCOrtB39MP57Hxxvjrf5lSndV3zFC2Ec/UXZAznOg==</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="ChH6EUuwbI21Fxv+skDjEDbe1sU=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0OaZJdBQseUqXySFupDtF4fJ1YGY+tmXHbyPKOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dE+Q8Vhp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWaSwYDlvJUngZUAY8FDHSsOZxmEmgScDgJpq/n/yfnIFWcis96lsEwoWMRRzGj2qT8wxHBPqMZPhx5o61dt+VWgW8DsgC7Bzt//6y/v9g5Gm03fvlhyvIEhGacKnVG3EwPCyp1zDiUTT9XkFE2pWM4M1DQBNSwqESXeM9kQhyl0lyhcZX9v6KgiVKzJDDMhOqJWv2bJ5dGFJd15+byXB31hkUsslyDYPXYKOdYp3i+DxzGEpjmMwMok7FRjtmESsq02VpzD/tVaeEMlHk7IMYcZCadTzPBHAkKqGQTh6UiAmn6g3ImQEOQLQ2XcyUhRMaZSliRzMYSQJTFxzevyqLftsl+22675SorkOnFNYv0923Sdu2+d4uWSirGcMPr2F1zuqtDQyqnAc+vea6h2MTrlZXbLztej/RxDdzOApD2jdvHXou4LfLB2P4O1bGBnqHn6AUiqIsO0Ft0hAaIoQxdoR/op/XF+mp9s77XVKuxqHmKlsK6+geDbOeD</latexit> <latexit sha1_base64="r0tjdI3cdsXjoIJfUmCMVZHz2A8=">AAAC3HicZVFNb9NAEN2YrxK+WjhyWRFV4mDFuzZpklsFB+BQqXykrdSEaL0eO1bWa2t3DY0s37ghrr3C7+Cf8G/YOGlF0tGu9Hb2zczTvLAQqTaE/G05t27fuXtv5377wcNHj5/s7j090XmpOIx4LnJ1FjINIpUwMqkRcFYoYFko4DScv1n+n34FpdNcfjaLAiYZS2Qap5wZmxofTSkec1bgo6k/3e2QLmkC3wR0DTpoHcfTvdafcZTzMgNpuGBan1NSmEnFlEm5gLo9LjUUjM9ZAucWSpaBnlSN6Brv20yE41zZKw1usv9XVCzTepGFlpkxM9Pbf8vkxojqYtW5vTnXxINJlcqiNCD5amxcCmxyvNwHjlIF3IiFBYyr1CrHfMYU48Zurb2Px01p5Y20fXsgEwGqUN6nheSeAg1M8ZnHcxmDsv1BezNgEaiugYulkghi60wjrMoWiQKQdfXx7eu6GgYuPQjcgNTbrFDl365YdHjg0oC4Q/8GLVdMJnDN67l9e/rbQyOm5qEor3jEUlzqD+rG7Vc9f0CHeAVIbw1ocO32id+lpEs/kM7h+7XvO+g5eoFeIor66BC9Q8dohDgq0CX6hX47X5zvzg/n54rqtNY1z9BGOJf/AFom5Fc=</latexit> M 2 ∩ M 3 <latexit sha1_base64="t7alCmBpTyrreIV6oLqCwCvs8hQ=">AAAC3HicZVHLbtNAFJ2YVwmPtrBESCOqSiys2GOTJtlVsIAuKpVH2kp1iMbja8fKeGzNjKGR5R07xLZb+A7+pN/AR8DESSuSXs1IZ+6ce8/RvWHBU6Vd97Jl3bp95+69jfvtBw8fPd7c2n5yrPJSMhiynOfyNKQKeCpgqFPN4bSQQLOQw0k4fTP/P/kCUqW5+KRnBYwymog0ThnVJhUcjj0cMFrgw7E/3tpxO24T+CYgS7Cz//zvH2TiaLzd+h1EOSszEJpxqtQZcQs9qqjUKeNQt4NSQUHZlCZwZqCgGahR1Ziu8a7JRDjOpblC4yb7f0VFM6VmWWiYGdUTtf43T65IVOeLzu1VXR33R1UqilKDYAvZuORY53g+DxylEpjmMwMok6lxjtmESsq0mVp7FwdNaeUMlXk7IBIOspDOx5lgjgQFVLKJw3IRgzT9QTkToBHIjobzuZMIYrOZxliVzRIJIOrqw9vXdTXwbbLn275br7NCmX+9YpHBnk181x54N2i5pCKBa17X7pnTWxeNqJyGvLziuYZiE69fN9t+1fX6ZIAXwO0uAfGvt33sdYjbIe/N2g/QIjbQM/QCvUQE9dA+eoeO0BAxVKAL9BP9sj5b36zv1o8F1Wota56ilbAu/gGFm+ag</latexit> <latexit sha1_base64="MneRyM+x4BsitJMXMpMEcJF3b1Y=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0xaZJdBQseUqXySFupCdF4fJ1YGY+tmXFby/KOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dI+f8khp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWSSQYDlvBEnvpUAY8EDHSkOZymEmjsczjxZ6/n/yfnIFWUiM86T2EU04mIwohRbVLDw3EbDxlN8eHYG2/tui23CnwbkAXYPdj5+2f9/cXO0Xi78WsYJCyLQWjGqVJnxE31qKBSR4xD2RxmClLKZnQCZwYKGoMaFZXoEu+ZTIDDRJorNK6y/1cUNFYqj33DjKmeqtW/eXJpRHFZd24uz9Vhb1REIs00CFaPDTOOdYLn+8BBJIFpnhtAmYyMcsymVFKmzdaae3hYlRbOQJm3A2LCQabS+ZQL5khQQCWbOiwRIUjTH5QzBRqAbGm4nCsJIDTOVMKKOJ9IAFEWH9+8Kou+Z5N9z/bccpXly+TimkX6+zbxXLvfvkVLJBUTuOF17K453dWhAZUzn2fXPNdQbNLulZXbLzvtHunjGridBSDejdvH7RZxW+SDsf0dqmMDPUPP0QtEUBcdoLfoCA0QQym6Qj/QT+uL9dX6Zn2vqVZjUfMULYV19Q+IT+eF</latexit> <latexit sha1_base64="MneRyM+x4BsitJMXMpMEcJF3b1Y=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0xaZJdBQseUqXySFupCdF4fJ1YGY+tmXFby/KOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dI+f8khp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWSSQYDlvBEnvpUAY8EDHSkOZymEmjsczjxZ6/n/yfnIFWUiM86T2EU04mIwohRbVLDw3EbDxlN8eHYG2/tui23CnwbkAXYPdj5+2f9/cXO0Xi78WsYJCyLQWjGqVJnxE31qKBSR4xD2RxmClLKZnQCZwYKGoMaFZXoEu+ZTIDDRJorNK6y/1cUNFYqj33DjKmeqtW/eXJpRHFZd24uz9Vhb1REIs00CFaPDTOOdYLn+8BBJIFpnhtAmYyMcsymVFKmzdaae3hYlRbOQJm3A2LCQabS+ZQL5khQQCWbOiwRIUjTH5QzBRqAbGm4nCsJIDTOVMKKOJ9IAFEWH9+8Kou+Z5N9z/bccpXly+TimkX6+zbxXLvfvkVLJBUTuOF17K453dWhAZUzn2fXPNdQbNLulZXbLzvtHunjGridBSDejdvH7RZxW+SDsf0dqmMDPUPP0QtEUBcdoLfoCA0QQym6Qj/QT+uL9dX6Zn2vqVZjUfMULYV19Q+IT+eF</latexit> <latexit sha1_base64="MneRyM+x4BsitJMXMpMEcJF3b1Y=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0xaZJdBQseUqXySFupCdF4fJ1YGY+tmXFby/KOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dI+f8khp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWSSQYDlvBEnvpUAY8EDHSkOZymEmjsczjxZ6/n/yfnIFWUiM86T2EU04mIwohRbVLDw3EbDxlN8eHYG2/tui23CnwbkAXYPdj5+2f9/cXO0Xi78WsYJCyLQWjGqVJnxE31qKBSR4xD2RxmClLKZnQCZwYKGoMaFZXoEu+ZTIDDRJorNK6y/1cUNFYqj33DjKmeqtW/eXJpRHFZd24uz9Vhb1REIs00CFaPDTOOdYLn+8BBJIFpnhtAmYyMcsymVFKmzdaae3hYlRbOQJm3A2LCQabS+ZQL5khQQCWbOiwRIUjTH5QzBRqAbGm4nCsJIDTOVMKKOJ9IAFEWH9+8Kou+Z5N9z/bccpXly+TimkX6+zbxXLvfvkVLJBUTuOF17K453dWhAZUzn2fXPNdQbNLulZXbLzvtHunjGridBSDejdvH7RZxW+SDsf0dqmMDPUPP0QtEUBcdoLfoCA0QQym6Qj/QT+uL9dX6Zn2vqVZjUfMULYV19Q+IT+eF</latexit> <latexit sha1_base64="MneRyM+x4BsitJMXMpMEcJF3b1Y=">AAAC3HicZVHLbtNAFJ24QEt4tIUlqjSiqsTCij0xaZJdBQseUqXySFupCdF4fJ1YGY+tmXFby/KOHWLbLXwHf8I38BEwsdOKpFcz0pk75957dI+f8khp1/3dsNbu3L23vnG/+eDho8ebW9tPjlWSSQYDlvBEnvpUAY8EDHSkOZymEmjsczjxZ6/n/yfnIFWUiM86T2EU04mIwohRbVLDw3EbDxlN8eHYG2/tui23CnwbkAXYPdj5+2f9/cXO0Xi78WsYJCyLQWjGqVJnxE31qKBSR4xD2RxmClLKZnQCZwYKGoMaFZXoEu+ZTIDDRJorNK6y/1cUNFYqj33DjKmeqtW/eXJpRHFZd24uz9Vhb1REIs00CFaPDTOOdYLn+8BBJIFpnhtAmYyMcsymVFKmzdaae3hYlRbOQJm3A2LCQabS+ZQL5khQQCWbOiwRIUjTH5QzBRqAbGm4nCsJIDTOVMKKOJ9IAFEWH9+8Kou+Z5N9z/bccpXly+TimkX6+zbxXLvfvkVLJBUTuOF17K453dWhAZUzn2fXPNdQbNLulZXbLzvtHunjGridBSDejdvH7RZxW+SDsf0dqmMDPUPP0QtEUBcdoLfoCA0QQym6Qj/QT+uL9dX6Zn2vqVZjUfMULYV19Q+IT+eF</latexit> <latexit sha1_base64="jNmz3ZVofQqvBVtogC0x1SEokic=">AAAC3HicZVFNb9NAEN2YrxK+WjhyWRFV4mDFuzFpklsFB+BQqXykrVSHaL0eJ1bWa2t3DY0s37ghrr3C7+Cf8G/YOG5F0tGu9Hb2zczTvDAXiTaE/G05t27fuXtv5377wcNHj5/s7j090VmhOIx5JjJ1FjINIpEwNokRcJYrYGko4DRcvFn9n34FpZNMfjbLHCYpm8kkTjgzNhUcTXs44CzHR1N/utshXVIHvgloAzqoiePpXutPEGW8SEEaLpjW55TkZlIyZRIuoGoHhYac8QWbwbmFkqWgJ2UtusL7NhPhOFP2SoPr7P8VJUu1XqahZabMzPX23yq5MaK8WHdub8418XBSJjIvDEi+HhsXApsMr/aBo0QBN2JpAeMqscoxnzPFuLFba+/joC4tvbG2bw/kTIDKlfdpKbmnQANTfO7xTMagbH/Q3hxYBKpr4GKlJILYOlMLK9PlTAHIqvz49nVVjnyXHviuT6ptVqiyb1csOjpwqU/cUe8GLVNMzuCa13cH9gy2h0ZMLUJRXPGIpbi0N6xqt1/1e0M6wmtA+g2g/rXbJ70uJV36gXQO3ze+76Dn6AV6iSgaoEP0Dh2jMeIoR5foF/rtfHG+Oz+cn2uq02pqnqGNcC7/AV8J5Fk=</latexit> M 1<latexit sha1_base64="70lj9XpjgrBQBVRLxo++rPfF3mc=">AAAC03icZVHLbtNAFJ2YR0t4tIUlqmRRVWJhxTN2Eye7ChY8JKRCSVupiaLx+NqxMh5bM2PayPIGse0W/oU/4Rv4CBgnLZBwNSOdOXPuQ/eEBU+VxvhHy7p1+87djc177fsPHj7a2t55fKLyUjIYspzn8iykCngqYKhTzeGskECzkMNpOHvZ/J9+AqnSXHzU8wLGGU1EGqeMakMdv5uQyfYe7vQGPa87sHEHY9zziQEE94PAt0nDmNg73P31c+Ptxe7RZKf1fRTlrMxAaMapUucEF3pcUalTxqFuj0oFBWUzmsC5gYJmoMbVYtba3jdMZMe5NFdoe8H+m1HRTKl5FhplRvVUrf815EqL6nJZub3aV8f9cZWKotQg2LJtXHJb53azBjtKJTDN5wZQJlMzuc2mVFKmzbLa+/ZokVq5Q2XeLoiEgyykezwXzJWggEo2dVkuYpCmPih3CjQC2dFw2UwSQWwMWQxWZfNEAoi6+vDqRV0NfIf0fMfH9boqlPnFjYoMeg7xsTPw/pPlkooE/ui6TmBOsN40onIW8vJGh43EIV6/Xrh90PX6pHG7Abh7Dchft0+8DjnoBO+N7W/QMjbRU/QMPUcEBegQvUZHaIgYStAV+oq+WUOrsj5bX5ZSq3Wd8wSthHX1G0tR5M4=</latexit>M 2<latexit sha1_base64="Iv7RQV/GBtlGckJMVy627R1ckcA=">AAAC03icZVHLbtQwFPWER8vwaAtLVMmiqsQimsSZ966CBQ8JqVCmrdQZjRznJhON40S2QzuKskFsu4V/4U/4Bj4CPMm0YoYrWzo+99x7rnz9jMdKu+6vhnXn7r37W9sPmg8fPX6ys7v39FSluWQwYilP5blPFfBYwEjHmsN5JoEmPoczf/56mT/7AlLFqfisFxlMEhqJOIwZ1YY6+TD1prsHbsutArstj3R7g54BnZ7X73QxWaUOjvb//N56f7l/PN1r/BwHKcsTEJpxqtQFcTM9KajUMeNQNse5goyyOY3gwkBBE1CTopq1xIeGCXCYSnOFxhX7b0VBE6UWiW+UCdUztZlbkmsWxVXdubnuq8PBpIhFlmsQrLYNc451ipffgINYAtN8YQBlMjaTYzajkjJtPqt5iMdVaeGMlHk7ICIOMpPOyUIwR4ICKtnMYakIQZr+oJwZ0ABkS8PVcpIAQrOQarAiWUQSQJTFpzevymLYtkmvbbfdclPly/TyRkWGPZu0XXvo/SdLJRUR3Oq6dt+c/qZpQOXc5/mNzjUSm3iDstp2p+sNyBDXwO2uAGnfbvvUa5FOq//RrP0dqmMbPUcv0EtEUB8dobfoGI0QQxG6Rt/RD2tkFdZX61sttRqrmmdoLazrvyRN5L4=</latexit>M 2<latexit sha1_base64="Iv7RQV/GBtlGckJMVy627R1ckcA=">AAAC03icZVHLbtQwFPWER8vwaAtLVMmiqsQimsSZ966CBQ8JqVCmrdQZjRznJhON40S2QzuKskFsu4V/4U/4Bj4CPMm0YoYrWzo+99x7rnz9jMdKu+6vhnXn7r37W9sPmg8fPX6ys7v39FSluWQwYilP5blPFfBYwEjHmsN5JoEmPoczf/56mT/7AlLFqfisFxlMEhqJOIwZ1YY6+TD1prsHbsutArstj3R7g54BnZ7X73QxWaUOjvb//N56f7l/PN1r/BwHKcsTEJpxqtQFcTM9KajUMeNQNse5goyyOY3gwkBBE1CTopq1xIeGCXCYSnOFxhX7b0VBE6UWiW+UCdUztZlbkmsWxVXdubnuq8PBpIhFlmsQrLYNc451ipffgINYAtN8YQBlMjaTYzajkjJtPqt5iMdVaeGMlHk7ICIOMpPOyUIwR4ICKtnMYakIQZr+oJwZ0ABkS8PVcpIAQrOQarAiWUQSQJTFpzevymLYtkmvbbfdclPly/TyRkWGPZu0XXvo/SdLJRUR3Oq6dt+c/qZpQOXc5/mNzjUSm3iDstp2p+sNyBDXwO2uAGnfbvvUa5FOq//RrP0dqmMbPUcv0EtEUB8dobfoGI0QQxG6Rt/RD2tkFdZX61sttRqrmmdoLazrvyRN5L4=</latexit>M 3<latexit sha1_base64="EJtZgXfJRLWfBxvwUlF7aG833i8=">AAAC0nicZVHLbtNAFJ2YR0t4tIUlqmRRVWJhxR67jpNdBQseElKhTVupiaLx+NqxMh5bM2PaYHmB2MIWvoVP4Rv4CJg4aUXC1Yx05s659x7dExYslcpxfrWMW7fv3N3YvNe+/+Dho63tncenMi8FhQHNWS7OQyKBpRwGKlUMzgsBJAsZnIXTl/P/s48gZJrzEzUrYJSRhKdxSonSqeN3Y2+8ved0sBe4vm86Ha/b67qBBk4TJl6CvcPdP7833l7uHo13Wj+HUU7LDLiijEh5gZ1CjSoiVEoZ1O1hKaEgdEoSuNCQkwzkqGq01ua+zkRmnAt9uTKb7L8VFcmknGWhZmZETeT63zy5MqK6WnRur85VcW9UpbwoFXC6GBuXzFS5OV+DGaUCqGIzDQgVqVZu0gkRhCq9rPb+sKms7IHUTxt4wkAUwj6ecWoLkEAEndg05zEI3R6kPQESgegouJoLiSDWfjS6qmyWCABeVx9evairvmfhrmd5Tr3OCkV+ec3C/a6FPcfqu//RckF4Ajc83wr0CdaHRkRMQ1Ze8xxNsbDbqxuzD3y3h/vmAjj+EmDvxuxTt4MPOsF77fobtIhN9BQ9Q88RRgE6RK/RERogihL0DX1HP4wT45Px2fiyoBqtZc0TtBLG179y4eSY</latexit>M 3<latexit sha1_base64="EJtZgXfJRLWfBxvwUlF7aG833i8=">AAAC0nicZVHLbtNAFJ2YR0t4tIUlqmRRVWJhxR67jpNdBQseElKhTVupiaLx+NqxMh5bM2PaYHmB2MIWvoVP4Rv4CJg4aUXC1Yx05s659x7dExYslcpxfrWMW7fv3N3YvNe+/+Dho63tncenMi8FhQHNWS7OQyKBpRwGKlUMzgsBJAsZnIXTl/P/s48gZJrzEzUrYJSRhKdxSonSqeN3Y2+8ved0sBe4vm86Ha/b67qBBk4TJl6CvcPdP7833l7uHo13Wj+HUU7LDLiijEh5gZ1CjSoiVEoZ1O1hKaEgdEoSuNCQkwzkqGq01ua+zkRmnAt9uTKb7L8VFcmknGWhZmZETeT63zy5MqK6WnRur85VcW9UpbwoFXC6GBuXzFS5OV+DGaUCqGIzDQgVqVZu0gkRhCq9rPb+sKms7IHUTxt4wkAUwj6ecWoLkEAEndg05zEI3R6kPQESgegouJoLiSDWfjS6qmyWCABeVx9evairvmfhrmd5Tr3OCkV+ec3C/a6FPcfqu//RckF4Ajc83wr0CdaHRkRMQ1Ze8xxNsbDbqxuzD3y3h/vmAjj+EmDvxuxTt4MPOsF77fobtIhN9BQ9Q88RRgE6RK/RERogihL0DX1HP4wT45Px2fiyoBqtZc0TtBLG179y4eSY</latexit>Figure 3.2: Initialization of a new subtree (line 15 – line 17 of Algorithm 1). The reached goal nodes at the intersection M 1 ∩ M 2 are used to initialize the next tree where q root is a synthetic root node that maintains the tree structure. 3.2.2 Intersection Point Independent Problems We now use the problem formulation in Equation (3.1) to describe robotic manipulation planning problems in which the manifolds describe subtasks for the robot to complete (e.g., picking up objects). The solution is an end-to-end path across multiple manifolds. For a certain class of sequential manifold planning problems, the following property holds: For each manifold intersection M i ∩ M i+1 , the free space output byϒ is set-equivalent for every possible pathτ i . In other words, the precise action taken to move the configuration from one constraint to the next does not affect the feasible planning space for the subsequent subtask. When this property holds for all intersections, we call the problem intersection point independent. The condition for this class of problems is ∀i∈[0,n]∀τ i ,τ ′ i ∈T :τ(1),τ ′ (1)∈ C M i ∩C M i+1 ⇒ϒ(C free,i ,τ(1))≡ ϒ(C free,i ,τ ′ (1)) (3.2) where ≡ denotes set-equivalence. In this work, we focus on the intersection point independent class of motion planning problems, which encompass a wide range of common problems. For grasping constraints, a notion of object symmetry about the grasp locations results in intersection point independent problems. If the object to be grasped is a cylindrical can, for example, allowing grasps to occur at any point around the circumference but at a fixed height would be intersection point independent. Any two grasps with the 19 Table 3.1: PSM ∗ algorithm parameters and their description. Symbol Description α Max step size of an edge in the tree β Probability of SteerConstraint step ρ Min distance between intersection nodes ε Threshold for a point to be on a manifold r Max manifold intersection projection distance same relative orientation of the gripper result in the same free configuration space of the system (robot + can). However, suppose the grasps can occur at any height on the can. Now, a grasp near the top of the can and a grasp in the middle of the can result in different free configuration spaces, and thus this would be an intersection point dependent problem. Focusing on intersection point independent problems allows us to define an efficient algorithm that grows a single tree over a sequence of manifolds. The more general intersection point dependent problem covers a wider class of problems. However, they are more difficult to solve because they require handling foliated manifolds [66] (e.g., every grasp leads to a different manifold). Our illustrative example is one such problem, since the grasp on the handle and the grasp from the top of the mug result in different free spaces. We provide some insights in Section 3.5 on how the proposed algorithm could be used to tackle them. 3.3 Planning on Sequenced Manifolds We now present the algorithm PSM ∗ that solves the problem formulated in Equation (3.1) for tasks that fulfill the intersection point independent property defined in Equation (3.2). The algorithm searches for an optimal solution to the constrained optimization problem, which is a sequence of pathsτ =(τ 1 ,...,τ n ) where eachτ i is a collision-free path on the corresponding manifold M i . The steps of PSM ∗ are outlined in Algorithm 1. The input to the algorithm is a sequence of mani- foldsM , an initial configuration q start ∈ M 1 on the first manifold, and the hyperparameters of the algorithm 20 (α,β,ρ,ε,r,m) (see Table 3.2). V,E are the vertices and edges, respectively, in the tree that the algorithm builds, while C is the robot configuration space and C free,1 is the initial free configuration space. The overall problem is divided into n subproblems, which correspond to the number of manifold inter- sections inM . Each subproblem can be broadly thought of as growing a tree between two consecutive man- ifold intersections. We give a high-level summary here. In the inner loop (line 4 – line 13; see Section 3.3.1), a tree is iteratively grown from a set of initial nodes toward the intersection with the next manifold. First, a new constraint-adhering configuration q new is found by sampling a point, steering, and projecting it (line 5 – line 7). Second, in line 8 – line 13, the point is checked for validity before being added to the tree as well as the set of intersection points (if applicable). After the inner loop completes, the algorithm initializes the next subproblem with the found intersection points in line 15 – line 18 (see Section 3.3.2). It returns the optimal path that traverses all manifolds inM . 3.3.1 Inner Loop: Growing a Tree to the Next Manifold The first phase of the inner loop focuses on producing a new candidate configuration q new which lies on the constraint manifold M i and adding it to the tree. The PSM ∗ _STEER routine in Algorithm 2 computes q new . Instead of targeting a single goal configuration as in general SBMP, we propose two novel steering strategies that steer toward the intersection between manifolds M i ∩ M i+1 . In line 5 – line 6 of Algorithm 1, a new target point q rand in the configuration space is sampled and its nearest neighbor q near in the tree is computed. Next, in Algorithm 2 one of the following two steering strategies is selected to find a direction d in which to extend the tree: 1. SteerPoint(q near ,q rand ,M i ) extends the tree from q near ∈ C M i towards q rand ∈ C while staying on the current manifold M i . 2. SteerConstraint(q near ,M i ,M i+1 ) extends the tree from q near ∈ C M i towards the intersection of the current and next manifold M i ∩ M i+1 . 21 3.3.1.1 SteerPoint(q near ,q rand ,M i ) In this extension step, the robot is at q near ∈ C M i and should step towards the target configuration q rand ∈ C while staying on the manifold M i . We formulate this problem as a constrained optimization problem of finding a curve γ : [0,1]→ C that minimize γ ||γ(1)− q rand || 2 subject to γ(0)= q near , Z 1 0 || ˙ γ(t)|| ds≤ α h M i (γ(s))= 0 ∀ s∈[0,1] (3.3) This problem is hard to solve due to the nonlinear constraints. Since the steering operations are called many times in the inner loop of the algorithm, we choose a simple curve representation and only compute an approximate solution to this problem. We parameterize the curve as a straight lineγ(s)= q near +sα d ||d|| with lengthα where the direction d is chosen as orthogonal projection of q rand − q near onto the tangent space of the manifold at q near . We apply a first-order Taylor expansion of the manifold constraint h M i (q near + d)≈ h(q near )+ J M i (q near )d, which reduces the problem to minimize d 1 2 ||d− (q rand − q near )|| 2 subject to J M i (q near )d= 0 (3.4) The optimal solution of this problem is d=(I− J ⊤ M i (J M i J ⊤ M i ) − 1 J M i )(q rand − q near ) = V ⊥ V ⊤ ⊥ (q rand − q near ) (3.5) where V ⊥ contains the singular vectors that span the right nullspace of J M i [110]. We normalize d later in the algorithm, and thus do not include the constraint||d||≤ α in the reduced optimization problem. The 22 Algorithm 1 PSM ∗ (M,q start ,α,β,ε,ρ,r,m) 1: V 1 ={q start }; E 1 = / 0; n= len(M)− 1 2: for i= 1 to n do 3: V goal = / 0 4: for k= 1 to m do 5: q rand ← Sample(C) 6: q near ← Nearest(V i ,q rand ) 7: q new ← PSM ∗ _STEER(α,β,r,q near ,M i ,M i+1 ) 8: if||h M i (q new )||>ε then 9: continue 10: if RRT ∗ _EXTEND(V i ,E i ,q near ,q new ,C free,i ) then 11: if||h M i+1 (q new )||<ε and 12: ||Nearest(V goal ,q new )− q new ||≥ ρ then 13: V goal ← V goal ∪ q new 14: // initialize next tree with the intersection nodes 15: q root = null,V i+1 ={q root };E i+1 = / 0 16: for q∈ V goal do 17: V i+1 ← V i+1 ∪{q};E i+1 ← E i+1 ∪{(q root ,q)} 18: C free,i+1 ← ϒ(C free,i ,V goal [0]) 19: return OptimalPath(V 1:n ,E 1:n ,q start ,M n+1 ) new configuration q near +α d ||d|| will be on the tangent space of the manifold at configuration q near , so that only few projection steps will be necessary before it can be added to the tree. Note that projection may fail to converge, and if this occurs the sample is discarded. 3.3.1.2 SteerConstraint(q near ,M i ,M i+1 ) This steering step extends the tree from q near ∈ C M i towards the intersection of the current and next manifold M i ∩ M i+1 , which can be expressed as the optimization problem minimize γ ||h M i+1 (γ(1))|| 2 subject to γ(0)= q near , Z 1 0 || ˙ γ(t)|| ds≤ α h M i (γ(s))= 0 ∀ s∈[0,1] (3.6) 23 Algorithm 2 PSM ∗ _STEER(α,β,r,q near ,M i ,M i+1 ) 1: if Sample(U(0,1))<β then 2: d← SteerConstraint(q near ,M i ,M i+1 ) 3: else 4: d← SteerPoint(q near ,q rand ,M i ) 5: q new ← q near +α d ||d|| 6: if||h M i+1 (q new )||< Sample(U(0,r)) then 7: q new ← Project(q new ,M i ∩ M i+1 ) 8: else 9: q new ← Project(q new ,M i ) 10: return q new The difference from Equation (3.3) is that the loss is now specified in terms of the distance to the next manifold h M i+1 (γ(1)). This cost pulls the robot towards the manifold intersection. Again, we approximate the curve with a lineγ(s) and apply a first-order Taylor expansion to the nonlinear terms, which results in the simplified problem minimize d 1 2 ||h M i+1 (q near )+ J M i+1 (q near )d|| 2 subject to J M i (q near )d= 0. (3.7) A solution d can be obtained by solving the linear system J ⊤ M i+1 J M i+1 J ⊤ M i+1 J M i 0 d λ = − J ⊤ M i+1 h M i+1 0 (3.8) whereλ are the Lagrange variables. The solution is in the same direction as the steepest descent direction of the loss projected onto the tangent space of M i . Similar to the goal bias in RRT, a parameter β ∈[0,1] specifies the probability of selecting the SteerConstraint step rather than SteerPoint. After the steering strategy is determined and d is computed, q new is projected either on M i or on the intersection manifold M i ∩ M i+1 depending on the distance to the intersection of the manifolds measured by ||h M i+1 (q new )|| (line 6 of Algorithm 2). The threshold for this condition is sampled uniformly between 0 and 24 r, where the parameter r∈R >0 describes the closeness required by a point around M i+1 to be projected onto M i ∩ M i+1 . This randomization is necessary in order to achieve probabilistic completeness that is discussed in Section 3.3.3, which also gives a formal definition of r. At this point, q new has been produced and the algorithm attempts to add it to the tree using the extend routine from RRT ∗ [61], which we outline in Chapter 9. The final step in each inner iteration of Algorithm 1 is to check two necessary conditions to determine if the point should also be added to the set of intersection nodes V goal : 1. The point q new has to be on the next manifold h M i+1 (line 11); 2. V goal does not already contain the point q new or a point in its vicinity (line 12). For the second condition, we introduce the parameter ρ∈R ≥ 0 that is the minimum distance between two intersection points. As we will discuss in the theoretical analysis, ρ must be 0 in order to achieve probabilistic completeness and asymptotic optimality. In practice, we usually selectρ > 0, which results in better performance since it avoids having a large number of nearby and duplicate intersection points. 3.3.2 Outer Loop: Initializing the Next Subproblem After the inner loop of PSM ∗ completes, a new tree is initialized in line 15 – line 17 with all the intersection nodes in V goal and their costs so far. Initializing a new tree only with the intersection nodes has the advantage that subsequent planning steps focus on these relevant nodes, which results in a lower computation time and more goal-directed growing of the tree towards the next manifold. To keep the tree structure, we add a synthetic root node q root as parent node for all intersection nodes (Figure 3.2). In line 18, the free configuration space is updated based on the reached intersection node. Convergence of the algorithm can be defined in various ways. We typically set an upper limit to the number of nodes or provide a time limit for the inner loop. After reaching the convergence criteria, the algorithm returns the path with the lowest cost that reached the goal manifold M n+1 . 25 3.3.3 Completeness and Optimality Under the assumption thatρ is 0, PSM ∗ is provably probabilistically complete and asymptotically optimal. Here, we outline our approach to proving these properties. Probabilistic completeness is proved in two steps. First we prove probabilistic completeness of PSM ∗ on a single manifold. We claim subsequently that almost surely the tree grown on a manifold can be expanded onto the next manifold as the number of samples tends to infinity. The calculations related to the first step are based on [72, Theorem 1]. Theorem 1 in Chapter 9 proves the subsequent claim. Proving the asymptotic optimality of PSM ∗ relies on three lemmas (Lemma 2, 3 and 4 in Chapter 9). Lemma 2 shows that for any weak clearance path [61] there exist a sequence of paths with strong clearance [61] that converges to a path with weak clearance. Our analysis assumes that the optimal path has weak clearance. Hence, there exist a sequence of strong clearance paths that converges to the optimal path. Using Lemma 3 and Lemma 4, we prove that with an appropriate choice of the parameterρ, the PSM ∗ tree grown on the sequenced manifolds contains the paths which are arbitrarily close to any strong clearance path in the sequence of paths that converges to the optimal path. Finally, using continuity of the cost function, we prove that PSM ∗ is asymptotically optimal. Detailed proofs of the probabilistic completeness and asymptotic optimality can be found in Chapter 9 and Chapter 9. 3.4 Evaluation In the following experiments, we solve kinematic motion planning problems where the cost function mea- sures path length. We compare the following methods with each other: • PSM ∗ : The method proposed here – Algorithm 1. 26 • PSM ∗ (Greedy): Algorithm 1 with the modification that only the node with the lowest cost in V goal is selected to initialize the next tree (line 15 – line 17 of Algorithm 1). • PSM ∗ (Single Tree): This method grows a single tree over the manifold sequence without splitting it into individual subtrees. The algorithm usesρ = 0. It is explained in detail in Algorithm 3 in Chapter 9. • RRT ∗ +IK: This method consists of a two-step procedure that is applied to every manifold in the sequence. First, a goal point on the manifold intersection is generated via inverse kinematics by randomly sampling a point in C and projecting it onto the manifold intersection. Next, RRT ∗ is applied to compute a path on the current manifold towards this node [68]. This procedure is repeated until a point on the goal manifold is reached. • Random MMP: The Randomized Multi-Modal Motion Planning algorithm [37] was originally developed for the more general class of motion planning problems where the sequence of manifolds is unknown. Here, we apply it to the simpler problem where the sequence is given in advance and use RRT ∗ to connect points between mode transitions. • CBiRRT: The Constrained Bidirectional Rapidly-Exploring Random Tree algorithm [7, 6] grows two trees towards each other with the RRT Connect strategy [75]. In each steering step, the sampled config- uration is projected onto the manifold with the lowest level set function value. Since CBiRRT does not optimize an objective function, we apply a short cutting algorithm as a post-processing step. We compare these methods on the following criteria: • Path length – The length of the found path in C space. • Success rate – The number of times a collision-free path to the goal manifold is found for different random seeds. 27 Table 3.2: Parameters of the 3D Point on Geometric Constraints and Multi-Robot Object Transport experi- ments. Parameter name Symbol 3D point (Section 3.4.1) Robot transport (Section 3.4.3) max step size α 1.0 1.0 goal bias probability β 0.1 0.3 constraint threshold ε 0.01 1e-5 duplicate threshold ρ 0.1 0.5 projection distance r 1.5 0.5 number of samples m 1200 2000 • Computation time – Time taken to compute a path. All experiments are run on a 2.2 GHz Quad-Core Intel Core i7. The parameter values α,β,ε,ρ,r,m for the individual experiments were chosen experimentally and are summarized in Table 3.2. 3.4.1 3D Point on Geometric Constraints We demonstrate the planning performance and properties of the individual methods on a simple point that can move in a 3D space. The point needs to traverse three constraints defined by geometric primitives. The configuration space is limited to [− 6,6] in all three dimensions. The initial state is q start =(3.5,3.5,4.45) and the sequence of manifolds is 1. Paraboloid: h M 1 (q)= 0.1q 2 1 + 0.1q 2 2 + 2− q 3 2. Cylinder: h M 2 (q)= 0.25q 2 1 + 0.25q 2 2 − 1.0 3. Paraboloid: h M 3 (q)=− 0.1q 2 1 − 0.1q 2 2 − 2− q 3 28 4. Goal point: h M 4 (q)= q− q goal with the goal configuration q goal =(− 3.5,− 3.5,− 4.45). We evaluate the algorithms on two variants of this problem: an obstacle-free variant (Figure 3.1), and a variant that contains four box obstacles placed at the intersections between the manifolds (Figure 3.3). In Figure 3.1, q start is drawn as red point and q goal as blue point. The intersection nodes V goal are shown as magenta points and a solution path from PSM ∗ is visualized as a line. The results on success rate, path length and computation time are given in Table 3.3. All the methods are consistently able to find a path for all 10 random seeds. PSM ∗ and PSM ∗ (Single Tree) consistently achieve the lowest cost in both scenarios. PSM ∗ (Single Tree) has a higher computation time, because it keeps all nodes in a single tree whereas PSM ∗ splits them into individual trees per manifold. CBiRRT does not optimize an objective function and immediately converges when it finds a feasible path, which achieves the overall lowest computation time. However, the mean cost and standard deviations are higher compared to PSM ∗ . RRT ∗ +IK and Random MMP only optimize over the individual paths, but do not optimize the intersection point selection, which results in higher costs and standard deviations. PSM ∗ (Greedy) is better in terms of computation time since it only takes one intersection point as initial point for the next tree, but achieves an overall lower performance compared to PSM ∗ . The results also show that only PSM ∗ and PSM ∗ (Single Tree) are consistently able to find the intersection regions between the obstacles in which the optimal path lies. The other methods mainly select the intersection regions that are discovered first in the exploration. Figure 3.3 shows a set of found paths on the variant with obstacles. In Figure 3.4a, the path costs J(τ) of PSM ∗ are compared for various values of ρ. This parameter specifies the minimum distance between two intersection points, which influences the number of intersection points created during planning (line 12 in Algorithm 1). As a reference, we visualize the path costs of 29 q 1<latexit sha1_base64="bSV34Q9Y7j5mXTmb4iVWg78+WD4=">AAACznicZVHLbtNAFJ2YVwmPtrBkYxEhsbBiT9y8dhUsgF2guK3URNF4fO2YjMfuzLjUsiwkVmzYwm+w4RP4CP6GiZ1WJL2akc7cOffeo3v8jMVSOc7flnHr9p2793butx88fPR4d2//ybFMc0HBoylLxalPJLCYg6dixeA0E0ASn8GJv3y9+j+5ACHjlH9URQazhEQ8DmNKlE4dnc/xfK/jdJ06zJsAr0HnsHP+56v3azCZ77d+T4OU5glwRRmR8gw7mZqVRKiYMqja01xCRuiSRHCmIScJyFlZa63MFzoTmGEq9OXKrLP/V5QkkbJIfM1MiFrI7b9VcmNEedl0bm/OVeFoVsY8yxVw2owNc2aq1FytwQxiAVSxQgNCRayVm3RBBKFKL6s9rQtL25P6ZQOPGIhM2EcFp7YACUTQhZ2J9JPuIe0FkABEV8HlSkQAofai1lQmRSQAeFV+ePOqKseuhQeu5TrVNssX6ecrFh4PLOw61rh3g5YKwiO45vWtoT7D7aEBEUuf5Vc8R1Ms3BtVtdEH/d4Ij80GOP01wO610ce9Lj7oDt9rx9+hJnbQM/QcvUQYDdEheosmyEMUReg7+oF+GhPjwqiMLw3VaK1rnqKNML79A8o5408=</latexit>q 2<latexit sha1_base64="dydB8GbBotdkhOaWc3yqWni9mRc=">AAACznicZVHLbtNAFJ2YVwmPtrBkYxEhsbDiV/PaVbAAdoHitlITRePxjWMynnFnxqWWZSGxYsMWfoMNn8BH8DdM7LQi6dWMdObOufce3RNmNJHKcf62jFu379y9t3O//eDho8e7e/tPjiXPBYGAcMrFaYgl0IRBoBJF4TQTgNOQwkm4fL36P7kAIRPOPqoig2mKY5bME4KVTh2dz7zZXsfpOnWYN4G7Bp3Dzvmfr8Gv/ni23/o9iTjJU2CKUCzlmetkalpioRJCoWpPcgkZJkscw5mGDKcgp2WttTJf6ExkzrnQlymzzv5fUeJUyiINNTPFaiG3/1bJjRHlZdO5vTlXzYfTMmFZroCRZuw8p6bi5moNZpQIIIoWGmAiEq3cJAssMFF6We1JXVjagdQvG1hMQWTCPioYsQVIwIIs7EzwT7qHtBeAIxBdBZcrERHMtRe1pjItYgHAqvLDm1dVOfItt+9bvlNts0LBP1+x3FHfcn3HGnk3aFxgFsM1r2cN9BlsD42wWIY0v+I5mmK53rCqjT7oeUN3ZDbA6a2B618bfex13YPu4L12/B1qYgc9Q8/RS+SiATpEb9EYBYigGH1HP9BPY2xcGJXxpaEarXXNU7QRxrd/zKHjUA==</latexit>q 3<latexit sha1_base64="j5pE62gGh/jHHJKvEGcrn9sBWgs=">AAACznicZVHLbtNAFJ2YVwmPtrBkYxEhsbBiT9y8dhUsgF2guK3URNF4fO2YjMfuzLjUsiwkVmzYwm+w4RP4CP6GiZ1WJL2akc7cOffeo3v8jMVSOc7flnHr9p2793butx88fPR4d2//ybFMc0HBoylLxalPJLCYg6dixeA0E0ASn8GJv3y9+j+5ACHjlH9URQazhEQ8DmNKlE4dnc/d+V7H6Tp1mDcBXoPOYef8z1fv12Ay32/9ngYpzRPgijIi5Rl2MjUriVAxZVC1p7mEjNAlieBMQ04SkLOy1lqZL3QmMMNU6MuVWWf/ryhJImWR+JqZELWQ23+r5MaI8rLp3N6cq8LRrIx5livgtBkb5sxUqblagxnEAqhihQaEilgrN+mCCEKVXlZ7WheWtif1ywYeMRCZsI8KTm0BEoigCzsT6SfdQ9oLIAGIroLLlYgAQu1FralMikgA8Kr88OZVVY5dCw9cy3WqbZYv0s9XLDweWNh1rHHvBi0VhEdwzetbQ32G20MDIpY+y694jqZYuDeqaqMP+r0RHpsNcPprgN1ro497XXzQHb7Xjr9DTeygZ+g5eokwGqJD9BZNkIcoitB39AP9NCbGhVEZXxqq0VrXPEUbYXz7B88J41E=</latexit>Figure 3.3: Samples of found paths on the 3D point on with obstacles problem (Section 3.4.1). PSM ∗ (Greedy) and PSM ∗ (Single Tree). The results show that the computed paths of PSM ∗ with a smallρ value are very similar to the ones of PSM ∗ (Single Tree) whereas for largerρ values, PSM ∗ converges to the same performance as PSM ∗ (Greedy) since only a single intersection point is considered. Therefore,ρ can be seen as a trade-off between planning faster and achieving lower path costs. Figure 3.4b compares the path costs for different samples m. All methods improve with an increasing amount of samples and PSM ∗ and PSM ∗ (Single Tree) converge to similar cost values. 3.4.2 Multi-Robot Object Transport Tasks In this experiment, we evaluate PSM ∗ on various object transportation tasks involving multiple robots. The overall objective is to transport an object from an initial to a goal location. We consider three variations of this task: • Task A: A single robot arm mounted on a table with k= 6 degrees of freedom. The task is to transport an object from an initial location on the table to a target location. This task is described by n= 3 manifolds. • Task B: This task consists of two robot arms and a mobile base consisting of k= 14 degrees of freedom. The task is defined such that the first robot arm picks the object and places it on the mobile base. Then, 30 the mobile base brings it to the second robot arm that picks it up and places it on the table. This procedure is described with n= 5 manifolds. • Task C: In this task, four robots are used to transport two objects between two tables. Three arms are mounted on the tables and another arm with a tray is mounted on a mobile base. Besides transporting the objects, the orientation of the two objects needs to be kept upright during the whole motion. This task is described with n= 12 manifolds and the configuration space has k= 26 degrees of freedom. The initial states of the three tasks are visualized in Figure 3.5 where the target locations of the objects are shown in green. The geometries of the objects were chosen such that the tasks are intersection point independent (Section 3.2.2). Three types of constraints are used to describe the tasks. Picking up an object is defined with the constraint h M (q)= x g − f pos,e (q) where x g ∈R 3 is the location of the object and f pos,e (q) is the forward kinematics function to a point e∈R 3 on the robot end effector. The handover of an object between two robots is described by h M (q)= f pos,e 1 (q)− f pos,e 2 (q) where f pos,e 1 (q) is the forward kinematics function to the end effector of the first robot and f pos,e 2 (q) is that of the second robot. The orientation constraint is given by an alignment constraint h M (q)= f rot,z (q) ⊤ e z − 1 where f rot,z (q) is a unit vector attached to the robot end effector that should be aligned with the vector e z = (0,0,1) to point upwards. These constraints are sufficient to describe the multi-robot transportation tasks. The parameters of the algorithms are summarized in Table 3.2. The costs of the algorithms are reported in Table 3.3. CBiRRT could not be applied to this task since it requires a goal configuration at which the backward tree originates, which is not available for these tasks. On task A, nearly all methods robustly find solutions. However, when task complexity increases, RRT ∗ +IK, PSM ∗ (Single Tree), and Random MMP have difficulties to solve to the problem. Only PSM ∗ and PSM ∗ (Greedy) were able to find solutions of task C with the given parameters. The cost and computation time of PSM ∗ is lower compared to PSM ∗ (Greedy). A found solution of PSM ∗ for Task C is visualized in Figure 3.6. 31 0 1 2 3 4 5 14.5 15 15.5 16 16.5 17 PSM* PSM* (Greedy) PSM* (Single T ree) (a) 0 500 1000 1500 2000 14.5 15 15.5 16 16.5 17 17.5 PSM* PSM* (Greedy) PSM* (Single T ree) (b) Figure 3.4: Path costs over variations of the parameterρ (left) and the number of samples m (right) on the geometric constraints w/o obstacles problem (Section 3.4.1). The graphs show the mean and unit standard deviation over 10 trials. Figure (a) shows the costs increase for higher values ofρ, meaning fewer intersec- tion points are considered during planning. In (b), the performance of all methods improves with larger m values where PSM ∗ and PSM ∗ (Single Tree) converge to similar path costs. (a) Task A (b) Task B (c) Task C Figure 3.5: Start states of the object transport tasks where the goal is to place the red and blue object to the green target locations. 3.4.3 Pick-and-Pour on Panda Robot In this experiment, we demonstrate PSM ∗ on a real Panda robot arm with seven degrees of freedom. The task description consists of n= 7 manifolds, which describe the individual grasp, transport, and pouring motions. We assume the position of the objects in the scene are known. The parameters are the same as in the robot transport experiments (Table 3.2) and the planning time was 131.65 seconds. Figure 3.7 shows a found path executed on the real robot. 32 Figure 3.6: Snapshots of the resulting motion that PSM ∗ found for Task C. Figure 3.7: Snapshots of the pouring task motion. 3.5 Discussion We proposed the algorithm PSM ∗ to solve sequential motion planning posed as a constrained optimization problem, where the goal is to find a collision-free path that minimizes a cost function and fulfills a given sequence of manifold constraints. The algorithm is applicable to a certain problem class that is specified by the intersection point independent property, which says that the change in free configuration space is independent of the selected intersection point between two manifolds. PSM ∗ uses RRT ∗ ’s extend method with a novel steering strategy that is able to discover intersection points between manifolds. We proved that PSM ∗ is probabilistically complete and asymptotically optimal and demonstrated it on multi-robot trans- portation tasks. Restricting the problem class to intersection point independent planning problems allowed us to develop efficient solution strategies like growing a single tree over a sequence of manifolds. An interesting question for future research is how to extend the strengths of PSM ∗ to a larger problem class; specifically, how PSM ∗ can be used for problems that do not fulfill the intersection point independent property. For such 33 problems, the choice of intersection points influences the future parts of the planning problem, which results in a more complex problem. An interesting aspect is the effect of an intersection point on subsequent manifolds and how solutions for one intersection point can be reused and transferred to other intersection points without replanning from scratch. Further, it may be possible to reduce such problems to the simpler intersection point independent problem class, for example, by morphing object geometries into simpler shapes such that the intersection point independent property is satisfied. PSM ∗ could be applied to the reduced problem and provide a good initial guess for solving the original problem. 34 Table 3.3: Results of the 3D point and robot transport problems. We report the mean and one unit standard deviation over 10 runs with different random seeds. Success Path length Comp. time [s] 3D point w/o obstacles PSM ∗ 10/10 14.47± 0.04 10.64± 0.16 PSM ∗ (Single Tree) 10/10 14.47± 0.05 13.72± 0.18 PSM ∗ (Greedy) 10/10 16.20± 0.05 10.36± 0.10 RRT ∗ +IK 10/10 17.84± 2.23 28.35± 13.50 Random MMP 10/10 17.33± 1.28 34.75± 17.21 CBiRRT 10/10 14.70± 0.71 5.04± 0.30 3D point w/ obstacles PSM ∗ 10/10 15.95± 0.13 13.74± 0.51 PSM ∗ (Single Tree) 10/10 15.87± 0.18 20.42± 0.86 PSM ∗ (Greedy) 10/10 19.69± 0.27 12.89± 0.51 RRT ∗ +IK 10/10 21.56± 3.05 30.21± 7.55 Random MMP 10/10 22.15± 2.20 42.09± 17.22 CBiRRT 10/10 16.66± 1.34 3.34± 0.25 Robot transport A PSM ∗ 10/10 7.76± 0.84 7.22± 0.81 PSM ∗ (Single Tree) 9/10 8.53± 1.19 11.56± 0.41 PSM ∗ (Greedy) 10/10 8.18± 0.91 7.03± 0.09 RRT ∗ +IK 10/10 11.83± 3.23 74.65± 28.86 Random MMP 10/10 11.62± 2.36 85.45± 25.66 Robot transport B PSM ∗ 10/10 14.73± 1.27 14.19± 0.76 PSM ∗ (Single Tree) 0/10 – – PSM ∗ (Greedy) 9/10 15.41± 2.38 14.75± 0.53 RRT ∗ +IK 2/10 45.14± 0.58 89.84± 11.00 Random MMP 10/10 16.96± 4.59 163.26± 37.80 Robot transport C PSM ∗ 10/10 27.07± 2.58 275.83± 19.73 PSM ∗ (Single Tree) 0/10 – – PSM ∗ (Greedy) 10/10 31.75± 2.51 311.81± 9.79 RRT ∗ +IK 0/10 – – Random MMP 0/10 – – 35 Chapter 4 ECoMaNN: Learning Equality Constraints for Motion Planning on Manifolds With Chapter 3, we have a motion planner for constrained motions, where we assumed the user has the knowledge and capability to write the constraint manifolds for the entire task sequence. However, this may not always be the case, or it may be more convenient to give demonstrations of a task such that the constraints can be automatically extracted for future planning. In this chapter, we explore the integration of learning techniques into the motion planner, specifically, neural network models that are able to automatically learn manifold constraints from robot demonstrations instead of requiring hand-specified constraints. This chapter contains work published in [142]. 4.1 Introduction Robots must be able to plan motions that follow various constraints in order for them to be useful in real- world environments. Constraints such as holding an object, maintaining an orientation, or staying within a certain distance of an object of interest are just some examples of possible restrictions on a robot’s motion. In general, two approaches to many robotics problems can be described. One is the traditional approach of using handwritten models to capture environments, physics, and other aspects of the problem mathematically or analytically, and then solving or optimizing these to find a solution. The other, popularized more recently, 36 involves the use of machine learning to replace, enhance, or simplify these hand-built parts. Both have challenges: Acquiring training data for learning can be difficult and expensive, while describing precise models analytically can range from tedious to impossible. Here, we approach the problem from a machine learning perspective and propose a solution to learn constraints from demonstrations. The learned constraints can be used alongside analytical solutions within a motion planning framework. In this work, we propose a new learning-based method for describing motion constraints, called Equal- ity Constraint Manifold Neural Network (ECoMaNN). ECoMaNN learns a function which evaluates a robot configuration on whether or not it meets the constraint, and for configurations near the constraint, on how far away it is. We train ECoMaNN with datasets consisting of configurations that adhere to constraints, and show results for tasks learned from demonstrations of robot tasks. We use a sequential motion plan- ning framework to solve motion planning problems that are both constrained and sequential in nature, and incorporate the learned manifolds into it. We evaluate the constraints learned by ECoMaNN with vari- ous datasets on their representation quality. Further, we investigate the usability of learned constraints in sequential motion planning problems. 4.1.1 Background The work described in this chapter combines ideas from many fields, including constrained sampling-based motion planning (SBMP), manifold learning, and learning from demonstrations, as described in Chapter 2. As opposed to other constrained SBMP methods, our method differs in that ECoMaNN learns an im- plicit description of a constraint manifold via a level set function, and during planning, we assume this representation for each task. We note that our method could be combined with others, e.g. learned sampling distributions, to further improve planning results. 37 Our work is related to many of the manifold learning approaches described in Section 2.5; in particular, the tangent space alignment in LTSA is an idea that ECoMaNN uses extensively. Similar to the ideas pre- sented in this chapter, the work in [96] delineates an approach to solve motion planning problems by learning the solution manifold of an optimization problem. In contrast to others, our work focuses on learning im- plicit functions of equality constraint manifolds, which is a generalization of the learning representations of Signed Distance Fields (SDF) [101, 86], up to a scale, but for higher-dimensional manifolds. Further, our work can be seen as a special case of inverse optimal control (IOC) where the task is only represented in form of constraints. Instead of using the extracted constraints in optimal control methods, we integrate them into sampling-based motion planning methods, which are not parameterized by time and do not suffer from poor initializations. 4.1.2 Motion Planning on Manifolds In this work, we aim to integrate learned constraint manifolds into the PSM ∗ framework from Chapter 3, using the same problem formulation and notation as described in Section 3.2. We employ data-driven learning methods to learn individual manifolds M from demonstrations with the goal to integrate them with analytically defined manifolds into this framework. 4.2 Equality Constraint Manifold Neural Network (ECoMaNN) We propose a novel neural network structure, called Equality Constraint Manifold Neural Network (ECo- MaNN), which is a (global) equality constraint manifold learning representation that enforces the alignment of the (local) tangent spaces and normal spaces with the information extracted from the Local Principal Component Analysis (Local PCA) [60] of the data. ECoMaNN takes a configuration q as input and outputs the prediction of the implicit function h M (q). We train ECoMaNN in a supervised manner, from demon- strations. One of the challenges is that the supervised training dataset is collected only from demonstrations 38 of data points which are on the equality constraint manifoldC M , called the on-manifold dataset. Collecting both the on-manifoldC M and off-manifoldC \M ={q∈C | h M (q)̸= 0} datasets for supervised training would be tedious because the implicit function h M values of points inC \M are typically unknown and hard to label. We show that, though our approach is only given data onC M , it can still learn a useful and sufficient representation of the manifold for use in planning. Figure 4.1: A visualization of data augmentation along the 1D normal space of a point q in 3D space. Here, purple points are the dataset, pink points are the KNN of q, and the dark red point is ˇ q. q is at the axes origin, and the green plane is the approximated tangent space at that point. Our goal is to learn a single global representation of the constraint manifold M in the form of a neural network. Our approach leverages local information on the manifold in the form of the tangent and normal spaces [25]. With regard to h M , the tangent and normal spaces are equivalent to the null and row space, respectively, of the Jacobian matrix J M (´ q)= ∂h M (q) ∂q q=´ q , and valid in a small neighborhood around the point ´ q. Using on-manifold data, the local information of the manifold can be analyzed using Local PCA. For each data point q in the on-manifold dataset, we establish a local neighborhood using K-nearest neighbors (KNN) ˆ K ={ˆ q 1 , ˆ q 2 ,..., ˆ q K }, with K≥ n. After a change of coordinates, q becomes the origin of a new local coordinate frameF L , and the KNN becomes ˜ K ={˜ q 1 , ˜ q 2 ,..., ˜ q K } with ˜ q k = ˆ q k − q for all values of k. Defining the matrix X= ˜ q 1 ˜ q 2 ... ˜ q K ⊤ ∈R K× n , we can compute the covariance matrix S= 1 K− 1 X ⊤ X∈R n× n . The eigendecomposition of S= VΣV ⊤ gives us the Local PCA. The matrix V contains 39 the eigenvectors of S as its columns in decreasing order w.r.t. the corresponding eigenvalues in the diagonal matrixΣ. These eigenvectors form the basis of the coordinate frameF L . This local coordinate frameF L is tightly related to the tangent and normal spaces of the manifold at q. That is, the (n− l) eigenvectors corresponding to the (n− l) biggest eigenvalues of Σ form a basis of the tangent space, while the remaining l eigenvectors form the basis of the normal space. Furthermore, the l smallest eigenvalues of Σ will be close to zero, resulting in the l eigenvectors associated with them forming the basis of the null space of S. On the other hand, the remaining(n− l) eigenvectors form the basis of the row space of S. We follow the same technique as [25] for automatically determining the number of constraints l from data, which is also the number of outputs of ECoMaNN 1 . Suppose the eigenvalues of S are {λ 1 ,λ 2 ,...,λ n } (in decreasing order w.r.t. magnitude). Then the number of constraints can be determined as l= arg max([λ 1 − λ 2 ,λ 2 − λ 3 ,...,λ n− 1 − λ n ]). We now present several methods to define and train ECoMaNN. 4.2.1 Alignment of Local Tangent and Normal Spaces ECoMaNN aims to align the following: 1. the null space of J M and the row space of S (which must be equivalent to the tangent space) 2. the row space of J M and the null space of S (which must be equivalent to the normal space) for the local neighborhood of each point q in the on-manifold dataset. Suppose the eigenvectors of S are{v 1 ,v 2 ,...,v n } and the singular vectors of J M are{e 1 ,e 2 ,...,e n }, where the indices indicate decreas- ing order w.r.t. the eigenvalue/singular value magnitude. The null spaces of S and J M are spanned by {v n− l+1 ,...,v n } and{e l+1 ,...,e n }, respectively. The two conditions above imply that the projection of the null space eigenvectors of J M into the null space of S should be 0, and similarly for the row spaces. Hence, 1 Here we assume that the intrinsic dimensionality of the manifold at each point remains constant. 40 we achieve this by training ECoMaNN to minimize projection errors V N V ⊤ N E N 2 2 and E N E ⊤ N V N 2 2 with V N = v n− l+1 ... v n and E N = e l+1 ... e n . 4.2.2 Data Augmentation with Off-Manifold Data The training dataset is on-manifold, i.e., each point q in the dataset satisfies h M (q)= 0. Through Local PCA on each of these points, we know the data-driven approximation of the normal space of the manifold at q. Hence, we know the directions where the violation of the equality constraint increases, i.e., the same direction as any vector picked from the approximate normal space. Since our future use of the learned constraint manifold on motion planning does not require the acquisition of the near-ground-truth value of h M (q)̸= 0, we can set this off-manifold valuation of h M arbitrarily, as long as it does not interfere with the utility for projecting an off-manifold point onto the manifold. Therefore, we can augment our dataset with off-manifold data to achieve a more robust learning of ECoMaNN. For each point q in the on-manifold dataset, and for each random unit vector u picked from the normal space at q, we can add an off-manifold point ˇ q= q+ iεu with a positive integer i and a small positive scalarε (see Figure 4.1 for a visualization). However, if the closest on-manifold data point to an augmented point ˇ q= q+ iεu is not q, we reject it. This prevents situations like augmenting a point on a sphere beyond the center of the sphere. With this data augmentation, we now define several losses used to train ECoMaNN. 4.2.2.1 Training Losses Loss Based on the Norm of ECoMaNN Output. For the augmented data point ˇ q= q+ iεu, we set the label satisfying∥h M (ˇ q)∥ 2 = iε. During training, we minimize the norm prediction error: L norm =∥(∥h M (ˇ q)∥ 2 − iε)∥ 2 2 for each augmented point ˇ q. 41 Loss for Reflection Pairs. For the augmented data point ˇ q= q+ iεu and its reflection pair q− iεu, we can expect that h M (q+ iεu)=− h M (q− iεu). Therefore, during training we also try to minimize the following pairs loss: L reflection =∥h M (q+ iεu)+ h M (q− iεu)∥ 2 2 Loss for Augmentation Fraction Pairs. Similarly, between the pair ˇ q= q+ iεu and q+ a b iεu, where a and b are positive integers satisfying 0< a b < 1, we can expect that h M (q+iεu) ∥h M (q+iεu)∥ 2 = h M (q+ a b iεu) ∥h M (q+ a b iεu)∥ 2 . Hence, during training we also try to minimize the pairs loss: L fraction = h M (q+ iεu) ∥h M (q+ iεu)∥ 2 − h M (q+ a b iεu) h M (q+ a b iεu) 2 2 2 Loss for Similar Augmentation Pairs. Suppose for nearby on-manifold data points q a and q c , their approximate normal spaces N q a M and N q c M are spanned by eigenvector basesF a N ={v a n− l+1 ,...,v a n } and F c N ={v c n− l+1 ,...,v c n }, respectively 2 . IfF a N andF c N are closely aligned, the random unit vectors u a from F a N and u c fromF c N can be obtained by u a = ∑ n j=n− l+1 w j v a j ∥∑ n j=n− l+1 w j v a j ∥ 2 and u c = ∑ n j=n− l+1 w j v c j ∥∑ n j=n− l+1 w j v c j ∥ 2 , where{w n− l+1 ,...,w n } are random scalar weights from a standard normal distribution common to both the bases ofF a N andF c N . This will ensure that u a and u c are aligned as well, and we can expect that h M (q a + iεu a )= h M (q c + iεu c ). Therefore, during training we also try to minimize the pairs loss: L similar =∥h M (q a + iεu a )− h M (q c + iεu c )∥ 2 2 In general, the alignment ofF a N andF c N is not guaranteed, for example due to the numerical sensitivity of singular value/eigen decomposition. Therefore, we introduce an algorithm for Orthogonal Subspace Alignment (OSA) in Chapter 9 to ensure that this assumption is satisfied. 2 N q M is the normal space at point q on manifold M. 42 (a) Projected samples from ECoMaNN (b) New samples from V AE Figure 4.2: Visualization for Section 4.3.1, Plane dataset. Red points are the training dataset and blue points are samples generated from the learned manifolds. WhileL norm governs only the norm of ECoMaNN’s output, the other three lossesL reflection ,L fraction , andL similar constrain the (vector) outputs of ECoMaNN based on pairwise input data points without explic- itly hand-coding the desired output itself. We avoid the hand-coding of the desired output because this is difficult for high-dimensional manifolds, except when we have prior knowledge about the manifold, such as in the case of Signed Distance Fields (SDF) manifolds. 4.3 Experiments We use the robot simulator MuJoCo [145] to generate four datasets. For each dataset, we define a ground truth constraint ¯ h M , randomly sample points in the configuration (joint) space, and use a constrained motion planner to find robot configurations in C M that produce the on-manifold datasets: • Sphere: 3D point that has to stay on the surface of a sphere. N= 5000, d= 3, l= 1. • 3D Circle: A point that has to stay on a circle in 3D space. N= 1000, d= 3, l= 2. • Plane: Robot arm with 3 rotational DoFs where the end effector has to be on a plane. N= 20000, d= 3, l= 1. 43 • Orient: Robot arm with 6 rotational DoFs that has to keep its orientation upright (e.g., transporting a cup). N= 21153, d= 6, l= 2. 4.3.1 Accuracy and Precision of Learned Manifolds We compare the accuracy and precision of the manifolds learned by ECoMaNN with those learned by a variational autoencoder (V AE) [67]. V AEs are a popular generative model that embeds data points as a distribution in a learned latent space, and as such new latent vectors can be sampled and decoded into new examples which fit the distribution of the training data. We use two metrics: First, the distance µ ¯ h M which measures how far a point is away from the ground-truth manifold ¯ h M and which we evaluate for both the training data points and randomly sampled points, and second, the percent P ¯ h M of randomly sampled points that are on the manifold ¯ h M . We use a distance threshold of 0.1 to determine success when calculating P ¯ h M . For ECoMaNN, randomly sampled points are projected using gradient descent with the learned implicit function until convergence. For the V AE, latent points are sampled fromN (0,1) and decoded into new configurations. We sample 1000 points for each of these comparisons. We report results in Table 4.1 and a visualization of the test phase in Figure 4.2. We also plot the level set and the normal space eigenvector field of the ECoMaNN trained on the sphere and plane constraint dataset in Figure 4.3. In all experiments, we set the value of the augmentation magnitude ε to the square root of the mean eigenvalues of the approximate tangent space, which we found to work well experimentally. With the exceptions of the embedding size and the input size, which are set to the same dimensionality l as the constraint learned by ECoMaNN and the ambient space dimensionality d of the dataset, respectively, the V AE has the same parameters for each dataset: 4 hidden layers with 128, 64, 32, and 16 units in the encoder and the same but reversed in the decoder; the weight of the KL divergence loss β = 0.01; using batch normalization; and trained for 100 epochs. 44 Figure 4.3: Trained ECoMaNN’s level set contour plot and the normal space eigenvector field, after training on the sphere constraint dataset (left) and plane constraint dataset (right). Our results show that for every dataset except Orient, ECoMaNN out performs the V AE in both metrics. ECoMaNN additionally outperforms the V AE with the Orient dataset in the testing phase, which suggests more robustness of the learned model. We find that though the V AE also performs relatively well in most cases, it cannot learn a good representation of the 3D Circle constraint and fails to produce any valid sampled points. ECoMaNN, on the other hand, can learn to represent all four constraints well. Table 4.1: Accuracy and precision of learned manifolds averaged across 3 instances. “Train" indicates results on the on-manifold training set; “test" indicates N= 1000 projected (ECoMaNN) or sampled (V AE) points. Method Dataset ECoMaNN V AE µ ¯ h M (train) µ ¯ h M (test) P ¯ h M µ ¯ h M (train) µ ¯ h M (test) P ¯ h M Sphere 0.024± 0.009 0.023± 0.009 100.0± 0.0 0.105± 0.088 0.161± 0.165 46.867± 18.008 3D Circle 0.029± 0.011 0.030± 0.011 78.0± 22.0 0.894± 0.074 0.902± 0.069 0.0± 0.0 Plane 0.020± 0.005 0.020± 0.005 88.5± 10.5 0.053± 0.075 0.112± 0.216 77.733± 7.721 Orient 0.090± 0.009 0.090± 0.009 73.5± 6.5 0.010± 0.037 0.085± 0.237 85.9± 1.068 45 0 5 10 15 20 Training iterations 0 20 40 60 80 100 Projection success [%] (a) (b) Figure 4.4: (a): ECoMaNN success rate on the Orient dataset over number of training iterations. (b): A path planned on the learned manifold (red) and path on the ground truth manifold (black) are visualized. 4.3.2 Ablation Study of ECoMaNN In the ablation study, we compare P ¯ h M across 4 different ECoMaNN setups: 1. no ablation; 2. without data augmentation; 3. without pairs losses during training; 4. and without orthogonal subspace alignment (OSA) during data augmentation. Results are reported in Table 4.2. For this ablation test we use ECoMaNN with 4 hidden layers of size 36, 24, 18, and 10. The data suggest that all three parts of the training process are essential for a high success rate during projection. Of the three features tested, data augmentation appears to have the most impact. This makes sense because without augmented data to train on, any configuration that does not already lie on the manifold will have undefined output when evaluated with ECoMaNN. 46 Table 4.2: Percentage of projection success rate for a variety of ablation on ECoMaNN components. Ablation Type Sphere 3D Circle Plane No Ablation (98.67± 1.89) % (100.00± 0.00) % (92.33± 10.84) % w/o Data Augmentation ( 9.00± 2.45) % ( 16.67± 4.64) % ( 3.67± 3.30) % w/o Pairs Losses (17.33± 3.40) % ( 65.67± 26.03) % (35.67± 5.56) % w/o Pairs LossL reflection (92.67± 4.03) % ( 9.67± 2.05) % (38.00± 16.87) % w/o Pairs LossL fraction (88.33± 8.34) % ( 99.67± 0.47) % (85.33± 16.68) % w/o Pairs LossL similar (83.00± 5.10) % ( 70.67± 21.00) % (64.33± 2.62) % w/o OSA (64.33± 9.03) % ( 33.33± 17.13) % (61.00± 6.16) % 4.3.3 Learning ECoMaNN on Noisy Data We also evaluate ECoMaNN learning on noisy data. We generate a noisy unit sphere dataset and a noisy 3D unit circle with additive Gaussian noise of zero mean and standard deviation 0.01. After we train ECoMaNN on this noisy sphere and 3D circle datasets, we evaluate the model and obtain (82.00± 12.83) % and (89.33 ± 11.61) %, respectively, as the P ¯ h M metric. 3 4.3.4 Relationship Between the Number of Augmentation Levels and the Projection Success Rate Moreover, we also perform an experiment to study the relationship between the number of augmentation levels – i.e., the maximum value of the positive integer i in the off-manifold points ˇ q= q+ iεu – and the projection success rate. As we vary the maximum value of i at 1, 2, 3, and 7 on the sphere dataset, the projection success rates are (5.00± 2.83) %, (12.33± 9.03) %, (83.67± 19.01) %, and (97.33± 3.77) %, respectively, showing that the projection success rate improves as the number of augmentation levels are increased. 3 The small positive scalarε needs to be chosen sufficiently large as compared to the noise level, so that the data augmentation will not create inconsistent data w.r.t. the noise. 47 Figure 4.5: Visualization of a path that was planned on a learned orientation manifold. 4.3.5 Motion Planning on Learned Manifolds In the final experiment, we integrate ECoMaNN into the sequential motion planning framework described in Section 4.1.2. We mix the learned constraints with analytically defined constraints and evaluate it for two tasks. The first one is a geometric task, visualized in Figure 4.4b, where a point starting on a paraboloid in 3D space must find a path to a goal state on another paraboloid. The paraboloids are connected by a sphere, and the point is constrained to stay on the surfaces at all times. In this case, we give the paraboloids analytically to the planner, and use ECoMaNN to learn the connecting constraint using the Sphere dataset. Figure 4.4b shows the resulting path where the sphere is represented by a learned manifold (red line) and where it is represented by the ground-truth manifold (black line). While the paths do not match exactly, both paths are on the manifold and lead to similar solutions in terms of path lengths. The task was solved in 27.09 s on a 2.2 GHz Intel Core i7 processor. The tree explored 1117 nodes and the found path consists of 24 nodes. The second task is a robot pick-and-place task with the additional constraint that the transported object needs to be oriented upwards throughout the whole motion. For this, we use the Orient dataset to learn the manifold for the transport phase and combine it with other manifolds that describe the pick and place operation. The planning time was 42.97 s, the tree contained 1421 nodes and the optimal path had 22 nodes. Images of the resulting path are shown in Figure 4.5. 48 4.4 Discussion and Conclusion In this chapter, we presented a novel method called Equality Constraint Manifold Neural Network for learn- ing equality constraint manifolds from data. ECoMaNN works by aligning the row and null spaces of the local PCA and network Jacobian, which results in approximate learned normal and tangent spaces of the underlying manifold, suitable for use within a constrained sampling-based motion planner. In addition, we introduced a method for augmenting a purely on-manifold dataset to include off-manifold points and several loss functions for training. This improves the robustness of the learned method while avoiding hand-coding the labels for the augmented points. We also showed that the learned manifolds can be used in a sequential motion planning framework for constrained robot tasks. While our experiments show success in learning a variety of manifolds, there are some limitations to our method. First, ECoMaNN by design can only learn equality constraints. Although many tasks can be specified with such constraints, inequality constraints are also an important part of many robot planning problems. Additionally, because of inherent limitations in learning from data, ECoMaNN does not guar- antee that a randomly sampled point in configuration space will be projected successfully onto the learned manifold. This presents challenges in designing asymptotically complete motion planning algorithms, and is an important area of research. 49 Chapter 5 Background on Informative Path Planning in Natural Environments Until now, we have considered offline planning in fully observable environments. We next consider sit- uations in which these assumptions do not necessarily hold, and the next three chapters address aspects of task abstraction in one such context. Inspired by our collaboration with a team of microbiologists, the robot planning problem in these chapters aims to explore an unknown environment in an online fashion, in this case, natural aquatic environments to study harmful algae blooms. Though in a different domain, we are faced with similar concerns with task specification abstraction; these problems have historically only accepted generic specifications, and expanding these to complex multirobot systems has not been deeply investigated. In Chapters 6, 7 and 8, these aspects of task abstractions are addressed, and this chapter gives the background and motivation for this domain. 5.1 Informative Path Planning (IPP) Informative path planning (IPP) is a common planning framework often used in an online manner and con- sists of alternating between planning and taking an action. It uses knowledge of the environment, typically in the form of an internal model, to inform the next action that is taken, after which new information is gained and incorporated into the model, and the process repeats. IPP uses objective functions to determine which actions are the most informative at each step. These objective functions depend on the application, 50 but some common functions include entropy and mutual information [63, 35]. The robot produces a planned trajectory p at each planning step which maximizes (or minimizes) the objective function f , and these tra- jectories together create the complete planned path P through the environment. Typically there is a planning step limit, or budget, B on the problem which defines the maximum cost c of the robot path [24, 43], so the plan and act steps iterate until that budget is reached. This problem can be described as P ∗ = arg max P∈Φ f(P)| c(P)≤ B (5.1) whereΦ is the space of full trajectories and P ∗ is the optimal trajectory. For planning for taking measurements, Partially Observable Markov Decision Processes (POMDPs) provide a useful formulation. POMDPs are a framework that can determine optimal actions when the en- vironment is not fully observable or there is uncertainty or noise in the environment, resulting in their widespread applicability in planning problems. Formulating IPP as a partially observable Markov decision process (POMDP) has been explored [147]; previous works have used this for finding maxima [89] and to adapt the parameters of a rollout-based POMDP solver online to improve its efficiency [24]. Gaussian Processes (GPs) are non-parametric models with uncertainty quantification which are widely used to represent the belief distribution from observations in POMDP formulations of sequential Bayesian optimization and IPP [147, 89, 24, 63]. They approximate an unknown function from its known outputs by computing the similarity between points from a kernel function, in our case the squared exponential kernel [109], and can incorporate spatial dependency of the distribution into its predictions of unmeasured locations. The GP model produces an estimate of the mean value µ(x) and variance σ 2 (x) at any specific location x. Using observations as measurements from the environment and a GP to represent the belief distribution, sequential Bayesian optimization can be formulated as a Bayesian search game [147]. Measurements from 51 Table 5.1: Informative Path Planning as a POMDP. After [147, 24]. POMDP Informative Path Planning States Robot position g t , Underlying unknown function GT Actions Neighboring search points Observations Robot position g t , Measured location(s)X t = o(g t ) Measured value(s)Y t = GT(X t ) Belief GP conditioned on previously measured locations(X 0:t− 1 ) and values(Y 0:t− 1 ) Rewards f(X t ) the environmentY t = GT(X t ) constitute observations which are partially observable components of the overall ground truth environment GT . To adapt the Bayesian search game formulation to IPP, the belief state is augmented with the state of the robot g t and the actions the planner is allowed to take are restricted to local movements which are feasible for the robot [89]. A complete description of the POMDP for IPP can be seen in Table 5.1. Solving the POMDP allows action selection to take measurements that maximize the objective function. 5.2 Objective Functions for IPP Objective functions are used to specify the mission objective to maximize during planning. We describe some history here and show that there is a gap in the current literature for estimating quantile values. Often, the objective in IPP is to generate good coverage of a distributed phenomenon using an informa- tion theoretic objective such as entropy [63] or mutual information [35]. Other common objectives include finding the highest concentration location [89] or measuring near hotspots [92]. Sequential Bayesian opti- mization objectives are used to locate extreme values or areas of high concentration and have shown success in doing so in simulation and in real field scenarios [89, 8, 131]. These specialized objective functions are typically tuned to locate a single point, often the extrema. There has also been work on extending IPP missions to domains such as underwater inspection by modifying the GP and objective functions [45]. Specialized objective functions have also been developed for the task of modeling continuous and discrete 52 variables, as well as including sensor-specific characteristics such as the increase in uncertainty due to cam- era distance to the subject [106]. Other objective functions have been proposed for level set estimation which seek to classify points into sets above and below a pre-specified concentration, or a fraction of the concentration [34]. 5.3 Physical Specimen Collection Physical specimen collection is the process of collecting portions of the environment for later analysis. The choice of physical specimen collection locations is driven by the spatial heterogeneity of the studied phe- nomena [40], e.g., freshwater algal growth or crop health. We are primarily motivated by the study of algal growth and distribution in freshwater and marine ecosystems. Spatial heterogeneity of algal and cyanobacte- rial blooms is often obvious as conspicuous accumulations at down-wind and down-current locations at the surface of lakes, and vertical heterogeneity can occur due to water column stratification, differential growth at different depths, or active vertical migration by some planktonic species [124, 48]. Such heterogeneity thwarts studies designed to quantify the spatial distribution of algal biomass, and the concentrations of al- gal toxins that may be produced by some algae and cyanobacteria. Nonetheless, accurately characterizing this heterogeneity is fundamental for investigating average conditions (indicated by median quantiles) for investigating trends across many lakes or large geographical areas [41, 88]. Assessing heterogeneity is also essential for assessing the worst case scenarios for exposure of animals and humans to algal or cyanobacte- rial toxins (specifically the highest quantiles). Such information contributes to ecotoxicological studies used to develop thresholds that constitute significant exposure to these toxins [93]. Robotic approaches for quantifying algal biomass across quantiles, often using chlorophyll or chloro- phyll fluorescence as a proxy for algal biomass, have become a mainstay for quantitatively documenting heterogeneity in natural ecosystems [158, 125]. Adaptive surveys for selecting collection locations have 53 typically focused on finding high-concentration areas from which to measure, and perform the specimen collection onboard the robot or by a following robot [21, 87]. 5.4 Quantiles and Quantile Standard Error Estimation Quantile estimation refers to acquiring the value of a given quantile in a distribution. For example, the median algae concentration would be given by the value of the 0.5 quantile. Statistical techniques have been proposed to estimate quantiles and quantile standard error of a distribution. Quantile values can be estimated from measurements using ˜ v= x ⌊h⌋ +(h−⌊ h⌋)(x ⌈h⌉ − x ⌊h⌋ )where h=(n− 1)q+1, n is the number of measurements, and q is the quantile (Equation 7 in [49]), which is the default in the numpy Python package. Standard error of a quantile estimate is a measure of the uncertainty, and is typically used in the construction of confidence intervals of a quantile estimate. Given a probability density function p, the standard error of the qth quantile is p q(1− q)/( √ np( ˜ v)). Typically p is not known, but can be estimated from samples using a density estimator [154]. 5.5 Continuous Non-Convex Gradient-Free Optimization General optimization problems can be solved using continuous, non-convex, gradient-free methods. The cross-entropy (CE) method is a popular approach which works by maintaining an estimate about the dis- tribution of good solutions, and iteratively updating the distribution parameters based on the quality of the solutions sampled at each step, where the quality is determined via some function of the solution config- uration [22]. More precisely, a prior Pr and a posterior Po set of samples are maintained. Po contains configurations sampled from N (⃗ µ,⃗ σ). ⃗ µ and⃗ σ are computed from Pr, which is the bestη% configura- tions from Po. This continues iteratively, minimizing the cross-entropy between the maintained and target distributions [22]. 54 Simulated annealing (SA) [71] is an iterative algorithm inspired by statistical mechanics. SA optimizes an energy function similar to a loss function in other optimization schemes, as configurations with lower energy are preferred. SA begins with a temperature T = T max . At each iteration, T is decreased exponen- tially, and the configuration is slightly perturbed randomly. The perturbed state is either accepted or rejected probabilistically based on its energy and T . The algorithm terminates when T reaches a given threshold. Bayesian optimization (BO) is a method for selecting parameters for difficult optimization problems such as hyperparameter selection [130]. BO is similar to IPP and sequential Bayesian optimization [89] in that these methods build a model using a GP and select points which maximally improve this model. BO uses acquisition functions, such as expected improvement, and iteratively selects the best point to sample from [57, 108]. Other methods include genetic or evolutionary algorithms [132]. These algorithms are inspired by biol- ogy and generally incorporate some method for crossover between two solutions or mutations to encourage exploration. 5.6 Multirobot Studies Multirobot systems research has a rich history; many well-known problems have been extended to more than one robot, such as multirobot SLAM [46], multirobot exploration [47], and multirobot learning [120]. Many previous studies concerning the effect of robot team size on performance have focused on human factors such as mental demand and operator workload [152, 151], while those that have studied autonomous group performance have shown inconclusive results regarding correlation between performance and team size [118], and that multirobot problems tend to follow the Law of Marginal Returns [117, 133], which states that as more resources are added to a problem, smaller returns are generated. This property, also known as submodularity, can be seen both in number of robots as well as measurements taken, and previous work has exploited it to approximately solve challenging planning problems [17]. 55 5.6.1 Multirobot IPP Multirobot informative path planning uses a team of robots to collect information about an underlying spatial distribution. Previous works have used objective functions like entropy or mutual information to explore phenomena such as temperature fields, plankton density, salinity, and chlorophyll. [13] exploits spatial correlation in the concentration to solve the planning problem while also improving the tradeoff between performance and efficiency. [29] addresses the problem of multirobot planning with continuous connectivity constraints. In that work, bipartite graphs are used to determine the next points for each robot to travel to. Though the main experimental focus is on varying the communication radius and the amount of collected information used to make future decisions with, the authors also find a linear relationship between the number of robots and the improvement using an entropy-based objective function. Bipartite graph matching has also been used to iteratively plan paths for each robot to the most informative points found in the environment [82]. A similar problem is addressed in [28] where there is intermittent communication in cluttered environments. There, the region is partitioned into V oronoi cells to better balance the workload between robots at every point when they come into communication range with each other. [64] also uses repeated V oronoi partitioning combined with limited information-sharing between underwater robots. Non- constant connectivity has also been studied in [44] where multirobot search with periodic connectivity is resolved using implicit coordination to solve the problem in a scalable manner. Reinforcement learning has been used to learn a planning policy prioritizing exploring hotspots and robust to robot failures in the environment [99]. 5.6.2 Bandwidth-Constrained Multirobot Communication Work that has considered the problem of deciding what or when to communicate is largely motivated by the same desire to respect bandwidth limitations that real networks must face and to reduce communica- tion costs. However, particular methods differ in how they approach the solution and in what context the 56 problem is situated. Some of the varied contexts in which utility-based decision-making for inter-robot com- munication has been investigated, include months-long data collection to be routed to a base station [98], non-cooperative multi-agent path finding [84], and those with request-reply protocols [84, 157]. In addition, some metrics used to determine utility include message category or intrinsic data features [98], difference in expected reward after incorporating a message [129, 90, 149], and the expected effect of the message on robot actions [84, 157], Previous work using a measure of utility based on expected reward have focused on scenarios where the end goal is to reach some goal locations, assigned to each robot, and the environment is fully or partially un- known [90, 149]. Robot teams in these works are decentralized in terms of planning and reward calculation, and may include forward-simulation of robot teammates during planning. Markov decision processes may be used to underlie robot planning. Communication network models in these works make several simpli- fying assumptions, including that communication transmission is deterministic, so reception is guaranteed, and that the environment is perfectly measurable. 57 Chapter 6 Informative Path Planning to Estimate Quantiles for Environmental Analysis Motivated to be able to describe task specification as a high-level, interpretable scientific goal, this chapter describes a method that addresses this in the context of analysis of natural environments. In this work, based on [115], we move away from generic task specifications that result in users burdened by post-processing the planning survey results, and instead automate physical location selection such that scientists can directly use the output to collect useful specimens. 6.1 Introduction In order to understand biological phenomena, scientists take a small number of physical specimens for later analyses to characterize the biological community and the contextual environmental conditions at the site. A marine biologist may capture water in a container or filter media at a location for later analysis, and an agricultural scientist may capture a small portion of a plant. Scientists later analyze these captured portions of the environment in a detailed laboratory setting. It is often expensive to collect these physical specimens as the scientist must go to the location and use a physical reagent, such as a container or filter media, which are limited in the field. Traditionally, expert heuristics underpin the selection of locations for scientific analysis. These expert heuristics generally attempt to spread out the analysis locations in 58 Desired Quantiles and Workspace Informative Path Planning Location Selection Field Specimen Collection Scientific Analysis Parameters Quantile Locations (Analysis Locations) Collected Specimens Q = [0.25,0.5,0.75] Plan next Step Move to Location Update Model Initialize Propose New Solution Accept/ Reject Environment Model and Quantile Values Measure Environment Update Best Figure 6.1: Full system. First, the parameters for the robotic survey are chosen, such as the area bounds and the quantiles for specimen collection. The robot performs informative path planning using our proposed objective functions, creating an environment model and an estimate of the quantile values. The quantile locations are then selected by minimizing our proposed loss function. These quantile locations correspond to physical locations which, when measured, have the estimated quantile values. After the quantile locations are chosen, humans go to them to collect field specimens which they later analyze in a laboratory setting. This work focuses on the steps shown in the blue shaded boxes. the phenomena of interest in order to take specimens with differing concentrations for characterization of heterogeneity [40]. The heuristics may make use of robotic surveys, but currently most robotic surveys use either pre-programmed paths [40], or autonomy which seeks maxima [89] or to cover the area spatially [63, 35], and do not directly plan for the specimen collection tasks. In contrast, we propose performing an adaptive robotic survey to find locations of interest for scientific analysis. To specify these locations, we propose calculating the quantiles of the distribution of interest so that scientists can capture specimens at varied locations in the environment based on differing values of some phenomenon of interest to them. 59 For instance, if a marine biologist is interested in taking 9 physical water specimens that are spread over a range of chlorophyll concentrations, they may choose to use the deciles of the concentration. If only a small number of locations can be analyzed and the upper extrema values are of interest, they may choose to perform analysis at the locations of the (0.90,0.95,0.99) quantiles. The quantiles of interest are a flexible way to describe the objective of specimen collection that is largely invariant to the exact phenomena being measured. Our goal is to select locations for detailed analysis by scientists in two steps. We use a robot to perform an adaptive survey, and then find locations to suggest for specimen collection based on the measurements it takes. Specifically, we first aim to find the desired quantile values of the measurement distribution by adaptively selecting robot measurement locations that maximize an objective function designed to estimate quantile values, and then we produce suggested locations for specimen collection that are likely to contain these values. Figure 6.1 summarizes the approach, showing how our objective function for informative path planning and our loss function for location selection fit into an overall physical specimen collection pipeline. Our contributions are: • A planner with custom objective functions that adaptively improve the quality of estimated quantile values; • A general loss function, which can be used with any optimizer, for selecting spatial locations for physical specimen collection which have the values estimated for the quantiles; • Quantitative evaluation of our method on point and camera sensors with previously collected aquatic datasets; • A demonstration of our method on a robot in a real-world crop health estimation task. 60 6.2 Background We pose our problem as an IPP problem, where the overall goal is to minimize error in the estimate of a given set of quantiles of a distribution in the environment, and we use a GP to model the environment. The GP environment model estimates the valueµ(x) and varianceσ 2 (x) at a specific location x. Similar to previous work, we use the POMDP formulation, and approximately solve this POMDP to select actions to take measurements for maximizing the objective function. Our works differs from others primarily in the overall goal that the task specification permits. Specif- ically, as discussed in Chapter 5, previous work has typically aimed for coverage of the given area, or to find the highest concentration of the distribution. In contrast, our goal is to directly address the user’s sci- entific goal by instead focusing on improving estimates for any arbitrary specific quantiles instead of broad understanding of the spatial makeup of a distribution. In this work, we use a Gaussian kernel density estimator for quantile standard error when planning. Other estimation methods include the Maritz-Jarrett method [91], and the bootstrap or jackknife methods which involve calculating the quantiles over repeatedly sampled subsets of the data. These last two methods can be slow for large datasets and require many iterations to converge. The final output of this algorithm is to select the analysis locations by minimizing our proposed loss function once the IPP survey is completed (see Section 6.4.2), and for this, we use continuous non-convex gradient-free optimization methods. 6.3 Formulation We use a grid-based representation of the planning space,G # , which defines the set of locations that the robot could visit. For a robot that moves in R d ,G # ⊂ R d . X # is the set of locations the robot could measure. If the robot sensor has finer resolution than G # , e.g. if the robot uses a camera sensor or takes 61 measurements while traversing between grid points, then|X # |>|G # |. We define X 0:t as the locations the robot has measured up to time t andY # andY 0:t as the values at all possible measured locations, and the values the robot has measured up to time t, respectively, such that|X # |=|Y # | and|X|=|Y|. We define the ground truth quantile values as V = quantiles(Y # ,Q) where quantiles is the function described in Section 5.4 which computes the values V of the quantiles Q of a set of measurements, in this caseY # . To define the robot’s estimated quantile values, we compute ˜ V = quantiles(µ(X # ),Q) that is, the quantile values of the predicted values from the robot’s current GP for all locations the robot could sense. This is done to prevent the number of measurements from which the quantile values are estimated from changing as the robot explores (instead of, e.g. usingµ(X)). By doing this, we ensure we always estimate the quantiles across the entire measurable area. During planning, we aim to minimize this error by taking actions which maximize an objective function f that minimizes the error in the quantile value estimate. To suggest locations for the quantile values, we aim to find a set of |Q| locationsQ in the continuous space whose values at those locations are equal to the quantile ground-truth values. A set of locations is defined as Q∈Q # whereQ # ⊂ R d×| Q| andQ # is continuous over the space ofX # . In practice, the robot only has access to ˜ V (not V ) during the selection process, so the problem of finding the estimated quantile spatial locations ˜ Q can be stated as shown in Equation (6.1) with some selection loss function l s (see Equation (6.7)). ˜ Q= arg min Q ′ ⊂ Q # l s ( ˜ V,Q ′ ) (6.1) 6.4 Approach Figure 6.1 illustrates our method. We separate our approach into two steps: the survey, and the suggestion of locations. When performing the survey using IPP (Section 6.4.1), the robot takes measurements of the environment to improve its estimate of the quantiles. After the survey has concluded, location selection (Section 6.4.2) produces locations for scientists to visit to perform specimen collection. 62 6.4.1 Informative Path Planning To plan which locations to measure, the robot uses a POMDP formulation of IPP. In order to generate a policy, we use the partially observable Monte Carlo planner (POMCP) [126]. POMCP uses Monte Carlo tree search to create a policy tree. To expand the tree and estimate rewards, the tree is traversed until a leaf node is reached. From the leaf node, the reward conditioned on that action is estimated by a random policy rollout which is executed until the discounted reward is smaller than some valueε. We modify the rollout reward to be fixed horizon, giving a reward of zero after a certain number of random policy steps. We adopt the t-test heuristic for taking multiple steps from a POMCP plan for IPP to improve performance of the planner with fewer rollouts [24]. We define GP i− 1 = GP(X 0:i− 1 ,Y 0:i− 1 ;θ) as a GP conditioned on the previous locations (X 0:i− 1 ) and measurements (Y 0:i− 1 ) before measuring a proposed value, and GP i = GP(X 0:i− 1 ∪ X i ,Y 0:i− 1 ∪Y i ;θ) as a GP conditioned on the previous and proposed locations (X i ) and measurements (Y i ), whereθ are GP parameters andY i = GP i− 1 (X i ). Because the observations GT(x) for unseen locations are not known during planning, the predicted mean from GP i− 1 is used [89]. Objective Functions for Quantile Estimation We develop two novel objective functions to improve the quality of quantile value estimates. Both compare a measure of the quality of the quantiles estimated by the GP before and after adding a measurement to the GP, and include an exploration term c plan σ 2 (x i ) inspired by the upper confidence bound objective function [89], where c plan is a chosen constant. For both proposed objective functions we use Equation (6.2), whereδ is defined by the objective: f(X i )= δ(X i ) |Q| + ∑ x j ∈X i c plan σ 2 (x j ), (6.2) The first objective function, which we call quantile change, is based on the idea of seeking out values which change the estimate of the quantile values by directly comparing the estimated quantiles before and after adding the measured values to the GP. The idea behind this is that a measurement which changes the 63 estimate of the quantiles indicates that the quantiles are over- or under-estimated. This can be seen in in Equation (6.3): δ qc (X i )=∥quantile(µ GP i− 1 (X # ),Q)− quantile(µ GP i (X # ),Q)∥ 1 (6.3) The second objective function we develop, which we call quantile standard error, is based on the change in the estimate of the standard error for the estimated quantiles. It draws from the same idea that if the uncer- tainty in the quantile estimate changes after observing a measured value, then it will change the estimate of the quantile values, shown in Equation (6.4): δ se (X i )=∥se(µ GP i− 1 (X # ),Q)− se(µ GP i (X # ),Q)∥ 1 (6.4) se is an estimate of the standard error of the quantile estimate for quantiles Q. se uses a Gaussian kernel density estimate. To compute the objective function over a set of measured points, we average the objective function at each point. Baseline Objective Functions We compare against two baselines, one which maximizes spatial cover- age of a phenomena and another which seeks maximal areas. Entropy is a common objective function for IPP when only good spatial coverage of the environment is desired [63, 35, 23]. It provides a good baseline as it is often used when the specific values of the underlying concentration are unknown. Entropy is defined Equation (6.5): f en (X i )= ∑ x j ∈X i 1 2 log(2πeσ 2 (x j )) (6.5) Another objective function we compare against is expected improvement (EI), which is widely used in Bayesian optimization and sequential Bayesian optimization for finding maxima [57, 108]. EI favors 64 actions that offer the best improvement over the current maximal value, with an added exploration termξ to encourage diverse exploration. The EI objective function is defined according to Equation (6.6): f ei (X i )= ∑ x j ∈X i IΦ(Z)+σ(x j )φ(Z) (6.6) where Z= I σ 2 (x j ) , I=µ(x j )− max ( µ(X # ))− ξ , andΦ andφ are the CDF and PDF of the normal distribu- tion, respectively. 6.4.2 Location Selection Our final goal is to produce a set of |Q| locationsQ, at which the concentration values will be equal to V , the values of the quantiles Q. The selection process can be done offline as it does not affect planning. Finding locations that represent Q is difficult because the objective function over arbitrary phenomena in natural environments will likely be non-convex, and in a real-world deployment, the robot will only have an estimate ˜ V of the V it searches for. With the location selection problem formulation as in Equation (6.1), we propose the loss function l s ( ˜ V, ˜ Q)=∥ ˜ V− µ( ˜ Q)∥ 2 + c select σ 2 ( ˜ Q), (6.7) where c select σ 2 ( ˜ Q) is an added penalty for choosing points that the GP of collected measurements is not confident about 1 , and l s can be used in any optimization scheme. During optimization, Equation (6.7) is eval- uated using ˜ V and returns the suggested specimen collection locations ˜ Q. We compare three optimization methods in our experiments to determine which best minimizes our selection loss function (Equation (6.7)). A strength of these types of optimization methods is that the formulation allows for suggesting points that 1 The parameters c plan and c select are distinct. 65 1.25 1.50 1.75 2.00 2.25 2.50 2.75 3.00 RMSE Env: A, Deciles 1.0 1.5 2.0 2.5 3.0 3.5 RMSE Env: A, Quartiles 35 36 37 38 RMSE Env: A, Extrema Method Entropy Quantile Change Quantile Standard Error Expected Improvement 1.2 1.4 1.6 1.8 2.0 2.2 2.4 RMSE Env: B, Deciles 1.5 2.0 2.5 3.0 3.5 RMSE Env: B, Quartiles 16 17 18 19 20 21 22 23 RMSE Env: B, Extrema Figure 6.2: Simulated drone planning experiments with real data. Error between ground truth quantile values and estimated quantile values. Datasets A and B collected using a hyperspectral camera in Clearlake, California. Units are 400nm channel pixel intensity (0− 255). Each dataset and objective function pairing is run three times. may be spatially far from locations the robot was able to measure if they have values closer to the quantile values of interest. 6.5 Experiments To evaluate our approaches for planning and for location selection, we compare against baselines in two different informative path planning tasks in simulation using four datasets collected in the real world. In the first task, a simulated drone with a virtual camera gathers data from orthomosaics (a single image produced by combining many smaller images, called orthophotos) collected of a lake using a hyperspectral 66 sensor. The orthomosaics, A and B, are taken in the same location but on different days and times. The drone collects many measurements from one location, where each is a pixel in a downsampled image. As a proxy for chlorophyll concentration, we measure the 400nm channel pixel intensity (0− 255). The drone maintains a constant altitude and moves in a 2D plane with a north-fixed yaw, moving in either the x or y direction per step. In the second task, an autonomous underwater vehicle (AUV) explores an environment. Two AUV surveys, C and D, which were taken in the same reservoir but at different times and different areas, were conducted in a 3D lawnmower pattern using a chlorophyll sensor and are interpolated using a GP toX # . At each step, the AUV moves in one x, y, or z direction and takes five evenly spaced measurements when moving between locations. We evaluate each task on their two respective datasets (A/B, C/D) and three different quantiles: deciles (0.1,0.2,...,0.8,0.9), quartiles (0.25,0.5,0.75), and upper extrema (0.9,0.95,0.99). See Figure 6.3 for a summary of the dataset distributions. 0 200 400 600 800 Environment grid point index (sorted) 0 50 100 150 200 250 Environment grid point value Drone Environment A Drone Environment B 0 500 1000 1500 2000 Environment grid point index (sorted) 0 20 40 60 80 Environment grid point value AUV Environment C AUV Environment D Figure 6.3: Experimental dataset distributions. Drone data is measured in pixel intensity; AUV data inµg/L chlorophyll. 67 0.30 0.35 0.40 0.45 0.50 RMSE Env: C, Deciles 0.4 0.5 0.6 0.7 0.8 0.9 RMSE Env: C, Quartiles 2.1 2.2 2.3 2.4 2.5 2.6 2.7 RMSE Env: C, Extrema 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 RMSE Env: D, Deciles 0.8 1.0 1.2 1.4 1.6 1.8 RMSE Env: D, Quartiles 3 4 5 6 7 8 9 10 RMSE Env: D, Extrema Figure 6.4: Simulated AUV planning experiments with real data. Error between ground truth quantile values and estimated quantile values. Datasets C and D collected from a reservoir in California using an underwater robot with a chlorophyll fluorescence sensor. Units are µg/L chlorophyll. Each dataset and objective function pairing is run three times. 6.5.1 Informative Path Planning: Objective Functions To evaluate how well our proposed IPP objective functions estimate the ground truth quantile values in real environments, we compare quantile change (Equation (6.3)) and quantile standard error (Equation (6.4)) against a baseline entropy objective function (Equation (6.5)). For the upper extrema quantiles, we also compare against expected improvement (Equation (6.6)), as it is similar to a sequential Bayesian optimiza- tion based IPP task. 68 6.5.1.1 Setup In the planner, we useγ = 0.9, and each trial is run over 3 seeds. The objective c plan parameter is set to the approximate magnitude of the rewards seen for each environment, which we found experimentally to be an adequate value. The GP mean was set to 0 and the datasets were normalized, while the lengthscale was set to 12. The exploration constant c plan was set to 1E− 6 for quantile change and 1E− 2 for quantile standard error. To compare the performance of the approaches, we use the RMSE between the ground truth quantiles, V , and estimated quantile values after performing a survey, ˜ V . We choose this metric rather than, e.g., com- paring the error between the estimated concentration µ(X # ) and the ground truth concentration GT(X # ) because the objective of the task specifically focuses on getting the highest accuracy for the quantile values, and does not optimize for overall concentration accuracy. Drone with Camera The drone is allowed to take 30 simulated pictures out of a grid with about 300 positions. Each picture is downsampled to 8× 5 pixels (40 measurements) with 37.1 ◦ by 27.6 ◦ field of view, similar to the drone used in the field trial reported in Section 6.5.3. While this is downsampled from the true image, it captures the coarse trends of the concentration, which is both still scientifically useful and performant for GP evaluations. For each trial, the GP is seeded with 100 evenly spaced measurements across the workspace, as prior data. The planner uses 300 rollouts per step, and the maximum planning depth is 7. AUV with Chlorophyll Sensor The AUV is simulated for 200 steps in a 12× 14× 2 grid. The planner uses 130 rollouts per step and a maximum depth of 10. The GP is seeded with measurements from 50 locations. 6.5.1.2 Discussion Overall, our results are robust across multiple experimental environments as well as robot sensor types. We find that planning with the quantile standard error objective function has a 10.2% mean reduction in median error across all environments when compared to using entropy. This shows that an objective function tailored 69 5.0 7.5 10.0 12.5 15.0 17.5 20.0 22.5 RMSE AUV: Deciles 0 5 10 15 20 25 RMSE AUV: Quartiles 10.0 12.5 15.0 17.5 20.0 22.5 25.0 27.5 RMSE AUV: Extrema 7.5 10.0 12.5 15.0 17.5 20.0 22.5 25.0 27.5 RMSE Drone: Deciles 0 10 20 30 40 50 RMSE Drone: Quartiles Selection Method Simulated Annealing Cross-Entropy Bayesian Optimization Best Visited 50 60 70 80 RMSE Drone: Extrema Figure 6.5: Quantile location selection results. RMSE between the ground truth values at the selected locations and the corresponding true quantile values. Units areµg/L chlorophyll for AUV , 400nm channel pixel intensity (0− 255) for Drone. to estimating a set of quantiles will outperform a coverage planner, such as entropy, that would typically achieve low overall error in reconstructing the entire environment model. The focus of our approach is to estimate the quantiles in a targeted way rather than to achieve uniform coverage. Figures 6.2 and 6.4 show the results of planning with the proposed objective functions for the drone and AUV experiments, respectively. In the drone tasks (A/B), quantile change and quantile standard error outperform the baseline entropy in estimating the deciles and upper extrema. In quartiles in environment A, both proposed methods perform well, but in environment B entropy outperforms our methods. For the extrema, both methods perform better than expected improvement, and entropy outperforms expected 70 improvement in environment A. We believe this is because expected improvement focuses explicitly on improving the (single) maximal value and does not do a good job of localizing high concentration areas, thus overestimating the quantile values. For the AUV tasks (C/D), quantile standard error outperforms entropy in environment C when estimating deciles and quartiles, and performs equally well as entropy in all other tasks besides estimating the extrema in environment D. Expected improvement performs poorly in estimating the extrema due to similar issues as with the drone. Quantile change performs poorly in most AUV tasks in contrast to the drone tasks, where it performs comparably. This may indicate that quantile change performance decreases with point sensors, or it is more sensitive to variation in the absolute measurement values. Thus, we recommend using quantile standard error unless there are computational constraints, since its performance is more consistent. We also compare our approach using the proposed quantile standard error objective function to the Spiral-STC coverage planner [33]. Figure 6.6 shows that the proposed adaptive solution is able to reduce the error between the estimated quantiles and the ground truth quantiles more rapidly than the non-adaptive coverage planner. 0 20 40 60 80 100 Number of Images Taken 0 10 20 RMSE Env: B, Deciles Method Coverage Quantile Standard Error Figure 6.6: Comparison between quantile standard error with POMDP solver and Spiral-STC coverage planner [33] on Drone Environment B estimating deciles, each over 3 trials. 71 6.5.2 Location Selection To evaluate how close the values at locations suggested by our proposed loss function are to the estimated quantile values, we use the optimization algorithms simulated annealing (SA), cross-entropy (CE), and Bayesian optimization (BO) with the results from the quantile estimation task (Section 6.5.1) using the quantile standard error objective function. 6.5.2.1 Setup We compute the error by RMSE(V, ˜ Q)=∥(V− GT( ˜ Q))∥ 2 . We set c select to 15 for deciles, 200 for quartiles, and 30 for extrema. We compare the results against a Best Visited (BV) baseline which selects the location the robot took a measurement at that is closest to each quantile value, i.e. solve ˜ Q ∗ = arg min ˜ Q bv ⊂ X ∥ ˜ V− µ( ˜ Q bv )∥ 2 . For SA, we use T max = 5, T min = 0.001, and cooling rate cr= 0.005 which leads to approximately 1000 optimization steps, and reset to the best solution every 100 steps. We start the optimization using the solution found by the BV baseline. For CE, we useα = 0.9,η = 0.9, 50 samples per iteration, and 100 iterations. α is a weighting factor on new samples, which is used to prevent premature convergence. For BO, we use the expected improvement acquisition function and initialize the GP with 50 randomly selectedQs as well as the solution found by the Best Visited baseline. We use 100 iterations and report the best found solution. 6.5.2.2 Discussion The error between the values at the estimated quantile locations and the true quantile values for the drone and the AUV experiments can be seen in Figure 6.5. Overall, we find that cross-entropy and Bayesian opti- mization produced the locations with values closest to the true quantile values. Both these methods perform a global search in the space of possible locations, indicating that global search optimizes Equation (6.7) more effectively. Simulated annealing had greater variability in performance. We believe this is because it is 72 Figure 6.7: Physical locations (stars) selected by the cross-entropy optimizer for deciles on the drone ex- periment. [Left] Black lines: true quantile value contours, overlaid on the absolute error between µ(X # ) and GT(X # ). Note that the lowest error tends to follow the quantile contour lines. [Right] Red crosses: locations the robot visited, overlaid on the ground truth image. a local search method and may fail to escape local optima. Best Visited produced a good initialization for the other methods, but was easily outperformed. We find that CE performs the best in three out of six possible scenarios. In particular, we see a 15.7% mean reduction in median error using CE with our proposed loss function compared to the BV baseline across all environments when using our proposed quantile standard error optimization function for quantile estimation during exploration. In general, methods can find the best points when selecting for quartiles or deciles, while the upper extrema are more difficult. Because the robot is limited in the amount of environment it can explore, the upper extrema are less likely to be measured during exploration. This leads to these quantiles being more challenging to select representative points for. Figure 6.7 shows results for one seed of the drone task when monitoring deciles. The suggested loca- tions, shown as stars, align relatively closely with the true quantile values V , shown by the contours on the left image. This demonstrates the ability of the optimizers to produce good location suggestions to guide environmental analysis. The right part of Figure 6.7 shows the same locations on top of the orthomosaic of what the drone could measure during exploration. This part of the figure highlights the difficulty of the problem of IPP 73 m 25 50 75 100 125 150 m 50 100 150 200 m 2 5 8 10 12 15 18 −7.42 19.17 Chl ug / L Figure 6.8: Physical locations (stars) selected by the cross-entropy optimizer for the upper extrema quantiles with an AUV using a chlorophyll point sensor. Blue/green points are the measured locationsX. for quantiles. The robot could only explore 15% of the total environment. With only partial knowledge of the distribution, the robot’s model of the phenomenon will vary based on the particular points it visited, which in turn affects the estimates of the quantiles. Similarly, Figure 6.8 shows the suggested locations for the upper extrema for one seed of the AUV task. For both tasks, all the optimization methods, with the exception of Best Visited, suggest points that may be spatially far from locations the robot has been able to visit. This allows for points with values potentially closer to the true quantile values to be selected for scientific analysis. 6.5.3 Field trial The goal of our final experiment is to demonstrate our method on real hardware for a crop health monitoring task in an open, grassy field, where the objective is to estimate the deciles of the green present in each pixel of the images (we use green as a proxy for plant health). 74 6.5.3.1 Setup We use a commercial, off-the-the shelf drone with a standard camera to take measurements of the field at a constant height of 3m. Similar to the simulated drone, the drone in this experiment moves in a 2D plane with a north-fixed yaw. The drone has a limited budget of 20 pictures (planning steps) in a 16 × 16 m square grid, where|G # |= 10× 10 locations, and each picture is downsampled to 8× 5 pixels. For planning, we use the quantile change objective function because it performs well for camera sensors and is faster than quantile standard error. We operate the drone with a custom app and use onboard GPS and IMU for localization. 6.5.3.2 Discussion Figure 6.9 shows the resulting suggested locations using CE based on the 20 steps the robot took. We find that, although the robot cannot explore the entire workspace due to its limited budget, and in fact does not visit a large green area, the system is able to suggest varied locations for specimen collection. Some selected locations (e.g., those representing the 0.3,0.4,0.5 quantiles) are spatially close to each other, which suggests this area contains a large gradient, or those quantile values are similar. As the goal is to suggest locations that contain the estimated quantile values, such locations may be near one another. 6.6 Conclusion Scientists traditionally collect physical specimens at locations selected using heuristics. They later analyze these specimens in a laboratory to characterize a phenomenon of interest (e.g., the distribution of algae in the water). We propose to, instead, choose these specimen collection locations by first performing an infor- mative path planning survey with a robot and then proposing locations which correspond to the quantiles of interest of the distribution. To accomplish this, we propose two novel objective functions, quantile change and quantile standard error, for improving the estimates of quantile values through informative path planning. We test these in 75 three settings: a drone with a camera sensor over lake imagery, an underwater vehicle taking chlorophyll measurements, and a field trial using a drone for a crop health task. Our objective functions perform well and outperform information-theoretic and Bayesian optimization baselines. In our experiments, our proposed quantile standard error objective function has a 10.2% mean reduction in median error when compared to entropy as the objective function. We also show that using our proposed loss function with black-box optimization methods can select environment locations for analysis that are representative of a set of quantiles of interests. We find that a cross-entropy optimizer using our loss function outperforms a baseline of using the best measured points, with a 15.7% mean reduction in median error in values across all environments. Our approach can be used to guide physical specimen collection in real field scenarios, as demonstrated by our field trial. 76 Figure 6.9: Visualization of a field trial modeling a crop health task. Left: Crosses are locations where the drone took images. The drone is limited to only visit 20% of the possible locations to take images. The quantiles of interest are the deciles and the locations are chosen by cross-entropy. The 9 stars show locations suggested to collect physical specimen. The measurement of interest is the amount of green in each pixel. Right: The drone used in the field trial, in flight. 77 Chapter 7 Multirobot Quantile Estimation in Natural Environments Chapter 6 has given us a framework in which interpretable scientific goals can be expressed in task specifica- tion for a robot in a field exploration deployment. However, in this context, task specifications may include multiple robots, and with that, different multirobot architectures. This leads us to characterize performance on the exploration task in this chapter, with the intention of understanding effects of task specifications abstracted away from single-robot deployments. This chapter is based on work published in [114]. 7.1 Introduction Scientists who study natural environments have used robots to assist in surveying or exploring regions of interest, for example to monitor harmful algal blooms [63, 53]. To describe such phenomena both flexibly and in an interpretable manner, we can specify quantiles of interest that robots can target during exploration [115]. Previously, work in this context has focused on single-robot adaptive surveys; in this chapter, we study multi-robot surveys, motivated by groups that may have more than one robot available, and by collaborative surveys between groups that would like to pool their robot resources to maximize scientific output from a survey. Though the naive assumption may be that more robots will always be better, in this work, we aim to investigate, in a principled manner, under what conditions this is true. Deploying a robot in the environment can be expensive since more resources are typically required with more robots deployed, and it 78 is unclear how having more robots in such a use case will scale. This study assesses the impact of team size, starting location, planning budget, and communication between robots for quantile estimation tasks in field environments. We believe it is an important step toward principled decisions regarding robot deployments for field robotics in aquatic biology. Our contributions are as follows: • We present the first study that investigates multirobot quantile estimation; • we present quantitative results on the effect of team size on performance on real-world aquatic datasets; • we present quantitative results on the effect of parameters including initial location spread, explo- ration budget, and inter-robot communication on performance, giving insight into what matters for a multirobot study; • we discuss the results in the context of field applications and how they may transfer to different experimental setups. 7.2 Background In this chapter, we are interested in the problem of multirobot quantile estimation, and specifically, we ask whether and under what conditions a multirobot approach to this problem is effective. We place our contributions in the context of previous work in these areas, introduced in Chapter 5. Here, we seek to investigate the effect of using multiple homogeneous robots specifically on the task of quantile estimation in natural environments, with the aim of understanding how to most effectively use resources for challenging and resource-constrained field work problems. Our work differs from previous work in multirobot IPP in that it explicitly addresses the question of different team sizes and other factors’ impact on performance. Further, the task we study is to improve the estimates of an arbitrary set of quantiles, rather than coverage or hotspots which has typically been the focus of other work. 79 Figure 7.1: Some variations of multirobot planning approaches. Top row shows initial spread α = 0: All robots start in the center. Bottom row shows initial spread α = 0.5: Robots start spread in an area 50% length and width of entire workspace. Left column shows no communication: Robots have no knowledge of the others; equivalent to each robot planning as if it were the only one. Middle column shows stochastic communication: Robots attempt to share observations but messages fail stochastically based on inter-robot distance. Right column shows full communication: Robots share the same environment model; equivalent to centralized planning. The work presented in this chapter builds upon Chapter 6, but there are several differences. First, the methods in Chapter 6 seek to produce, as a final output, a set of locations at which the desired quantile values can be found, while the goal here is to accurately estimate the desired quantile values. Second, we now generalize the problem to the multirobot domain, where previously only one robot was considered. Finally, where a main contribution previously was to propose new objective functions for planning that are tailored to the quantile estimation problem, our focus is now on investigating the impact of team size and other factors on quantile estimation accuracy, and thus we adopt just one of the previously proposed objective functions in all our experiments. 80 7.3 Methods The problem considered in this paper is that of accurately estimating a set of quantiles of a distribution of interest, such as algae concentration in a lake, by exploring a finite 2D workspace with N robots that take measurements at the locations they visit. The problem setup follows from Chapter 6, with slight modifications and additions to account for the multirobot aspect. There are N homogeneous robots collaborating on the estimation task with a planning step budget of B T . The locations at which a robot has taken a measurement up to time t and the corresponding values measured are represented byX 0:t andY 0:t , respectively. Each robot n maintains an estimate of the quantile values which is given by ˜ V n = quantiles(µ (n) GP t (X # ),Q) (7.1) where µ (n) GP t (X # ) is the estimate of all possible locations using the GP conditioned onX 0:t andY 0:t . Note that, for clarity, we will omit the(n) superscript in the remainder of the section. For a given N and B T , our problem is to select a complete path P ∗ composed of paths P n for each robot n within a planning budget constraint: P ∗ = arg max P f(P) (7.2) where P= [ n∈N P n (7.3) ∀n∈ N : length(P n )≤ B n (7.4) 81 and f is the objective function used during planning. In this work, we are interested in estimating the value of the underlying concentration at quantiles Q. Note that our goal is not finding the highest value in the environment nor optimizing model accuracy at every location. To this end, we use the quantile standard error objective function to evaluate a proposed location [115], which measures the difference in standard error of the estimated quantile values using a robot’s current en- vironment modelµ GP i− 1 , compared to using its current model updated with the expected new measurements at the proposed location µ GP i . It additionally includes a term that encourages exploration of locations with high model variance weighted by the parameter c, called the exploration bias. f(X i )= d |Q| + ∑ x j ∈X i cσ 2 (x j ) (7.5) d=∥se(µ GP i− 1 (X # ),Q)− se(µ GP i (X # ),Q)∥ 1 (7.6) We formulate the planning problem as a POMDP, as defined in Table 5.1, and solve the planning problem using a POMCPOW solver [140]. At each planing step, the robot selects the next location to travel to based on the measurements it has collected and its corresponding environment model. After moving to the next location, the robot takes an image, collecting a set of (noisy) point measurements, which it feeds back into its GP model. In this work, we assume a straight-line low-level motion planner for generating trajectories from one location to the next. We additionally enable a best-effort communication system between robots and compare to two other communication formulations, which are further described in Section 7.3.3. The planning process continues with each of the N robots until the allotted budgets have been reached, at which point planning is complete and the robots can return to the base. 82 At this time, all measurements from all robots are compiled and used to produce ˜ V final as follows: ˜ V final = quantiles(Y aggregate ,Q) (7.7) We now introduce several variations on the multirobot approach which we compare in our experiments, as illustrated in Figure 7.1. In the single robot case, the formulation is the same, simply with N= 1; however, the communication and initial spread variations have no effect on the implementation or execution of the planning. 7.3.1 Initial Location Spread During informative path planning, teams of robots will not cover the entire area possible and are naturally affected by their starting area. To this end, we define an initial location spread parameter α which varies from 0 (all robots start at the same location) to 1 (robots are spread around the entire workspace). At spread 0.5, for example, the robots would be started evenly spaced in a rectangle half the size of the entire workspace. We choose initial locations for robots following a variation of Lloyd’s algorithm [81] which iteratively computes the centroids of an approximate V oronoi tessellation of the space and reassigns the locations to those centroids. We perform the process for 100 iterations using 100N randomly sampled points. The result is N locations approximately uniformly spread in the allotted workspace. 7.3.2 Budget To control for the fact that a team with more robots will in practice simply have more planning steps, and for a fairer comparison to a single robot baseline, we implement a variation on the budget constraint which we call shared budget: ∀n∈ N : B n = B T /N. When(B T mod N)̸= 0, the remainder is split evenly among the most robots possible. The alternative is the default of a complete budget, which gives each robot the full budget, i.e.∀n∈ N : B n = B T . 83 7.3.3 Communication To investigate the impact of communication on the effectiveness of multirobot planning, we distinguish between three versions of information sharing. Full communication assumes perfect, instantaneous communication, which translates to every robot us- ing the same shared GP model of the environment. This is a centralized formulation of the planning problem. After every measurement that any robot takes, the shared GP is updated and is available immediately to other robots for planning. At the other extreme is no communication. In this case, each robot has its own environment model. Essentially, this implementation is equivalent to N robots planning in the environment independently, with no knowledge of the others; when new measurements are taken, they are only used to update that robot’s GP and motion is not coordinated at all. We believe a comparison to no communication provides a valuable baseline to understand how much improvement arises due solely to information sharing. In a practical sense, if separate research groups come together to pool their robots, implementing coordinating mechanisms can demand significant cost of time and effort. In that case, deploying the available robots with no behavioral changes may be simplest. The third variation, stochastic communication, lies in the middle. After taking measurements, a robot attempts to transmit the information to every other robot. We model each attempt as a Bernoulli trial and as- sume data is successfully transmitted with sigmoidal probability p success , following [16] and where distance is the distance between the two robots,η is a constant defining the sigmoid steepness, and r is the distance at which communication quality degrades past a threshold: p success = 1 1+ e η(distance− r) (7.8) 84 0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0 Distance (m) 0 1 Transmission Probability Figure 7.2: Visualization of the message transmission success probability based on inter-robot distance (Equation (7.8)). We assume that if communication is successful between robots n and m at time step i, allX i are received by m (i.e., there are no partial or corrupted measurements transmitted). The receiving robot’s GP is updated with the received data. We additionally compare to splitting the area into regions, or the partitioned case. This uses a V oronoi partition of the workspace based on the initial locations and restricts each robot to stay within its assigned region at all times. No communication is enabled between partitioned robots. 7.4 Experimental Setup In our experiments, we evaluate performance on two different real-world datasets. The datasets are collected using a hyperspectral camera mounted on a drone that flies over a freshwater lake. Each dataset is collected in the same area of the lake but on different days, leading to different algae distributions. The robots are bounded in an area of approximately 80× 60 meters. We use the 400nm channel from the datasets as a proxy for algae and normalize the measured pixel intensity (0− 255) to[0,1]. We test two sets of quantiles of interest in our experiments: quartiles Q=(0.25,0.5,0.75) and extrema Q=(0.9,0.95,0.99). For each combination of parameters in all experiments, we run 2 seeds. The available workspace is discretized into a 25x25 gridG # . The environment is unknown ahead of time to the robots. We assume a communication model as described in Section 7.3.3 with η = 0.5 and r = 10, and a visualization of Equation (7.8) with 85 Table 7.1: Parameters studied in our experiments. Parameter Tested values Initial location spread (α) 0.0, 0.33, 0.66, 1.0 Total budget (B T ) 10, 15, 30 Budget type complete, shared Communication type none, stochastic, full, partitioned these parameters is shown in Figure 7.2. We additionally add zero-mean Gaussian noise withσ noise = 0.05 to each measurement taken by a robot. In the POMCPOW solver, we set number of rollouts per step to 100, rollout depth to 4, and the planner discount factor to 0.8. The lengthscale for the GP is 12. We set the drone altitude to 7m and each image is 25 pixels (measurements). In order to better understand under what conditions multirobot teams are beneficial for quantile estima- tion, we select several parameters to vary in order to investigate their impact on performance. These are listed in Table 7.1. As a measure of performance, we report the root mean squared error (RMSE) between the ground truth quantile values V and the estimated quantile values ˜ V final at the end of the robot surveys. 7.4.1 Initial Location Spread In this experiment, we varyα ={0.0,0.33,0.66,1.0}. Each variation is tested on team sizes N ={2,4,8}, and we keep the budget constant at B T = 15. We additionally compare eachα with no communication and with stochastic communication, but do not consider the single robot case, asα and communication have no effect with only one robot. To quantify the significance of the performance differences observed, we report results from the Wilcoxon signed-rank test [155], which is a nonparametric version of the t-test and tests whether two paired samples originate from different distributions. 7.4.2 Planning Budget Next, we investigate the effect of the planning budget on performance. We vary B T ={10,15,30} in the complete budget case, settingα to the constant value of 0.66. We consider N= 1 as well as the multirobot 86 teams. In addition, we compare shared to complete budgets, setting B T = 15. In all cases, we enable stochastic communication between robots. 7.4.3 Communication Finally, we compare performance with different N across different levels of communication: full, stochas- tic, and none, and compare these to partitioning the space. We hold previous parameters constant: α = 0.66;B T = 15, and use N={1,2,4,8}. Note that in the partitioned case,α = 1.0. 7.5 Results We now present results from our experiments and include a discussion on their implications for real-world multirobot field work. Boxplots show final RMSE between the estimated quantile values ˜ V final and the ground truth quantile values V on the Y axis for an experiment, both aggregated across and refined by team sizes. Bars above indicate the Wilcoxon signed-rank significance levels for paired experiment groups [14]. 7.5.1 Initial Location Spread Figure 7.3 shows the results of the experiments on α using a one-sided Wilcoxon signed-rank test. We observe that, in general, performance tends to improve (error decreases) with increasing α. In particular, in the absence of stochastic communication as seen on the top row of Figure 7.3, we see a statistically signficant performance improvement between α = 0.33 and 0.66 (T=266, p≤ 0.001). On the top right, further separated by N, we similarly observe a statistically significant improvement in performance between α = 0.33 andα = 0.66 for both N = 2 and N = 4 (T=33, p≤ 0.05, T=32, p≤ 0.05, respectively), but we do not see such an improvement betweenα = 0.66 andα = 1.0 in those cases. In general, we also see that α has a more drastic effect on performance the more robots there are, indicated by the decreasing maximum errors as N increases. However, in general, there appear to be diminishing returns with biggerα. 87 0.0 0.33 0.66 1.0 Alpha 0.0 0.1 0.2 0.3 0.4 RMSE n=24 n=24 n=24 n=24 ns *** ns 2 4 8 N Robots 0.0 0.1 0.2 0.3 0.4 RMSE n=32 n=32 n=32 ns ns ns ns ns * ns * ns Alpha 0.0 0.33 0.66 1.0 0.0 0.33 0.66 1.0 Alpha 0.00 0.05 0.10 0.15 0.20 0.25 0.30 RMSE n=24 n=24 n=24 n=24 ns ns ns 2 4 8 N Robots 0.00 0.05 0.10 0.15 0.20 0.25 0.30 RMSE n=32 n=32 n=32 ns ns ns ns ns ns ns ns * Alpha 0.0 0.33 0.66 1.0 Figure 7.3: Error vs. Initial location spread (α). Top: No communication. Bottom: stochastic communica- tion. Right column shows results further separated by team size. Bars above indicate significance under the one-sided Wilcoxon signed-rank test; ***: p≤ 1e− 3, *: p≤ 5e− 2, ns: no significance. The bottom row of Figure 7.3 shows results when robots communicate stochastically with each other at different initial location spreads. Although we see the same general trend on left, in this case, we do not see a statistically significant drop in error. When controlled for team size on the right, we again see increasing α resulting in improved performance as indicated by lower median and maximum errors. Compared to the 88 10 15 30 T otal Budget 0.00 0.05 0.10 0.15 0.20 0.25 RMSE n=32 n=32 n=32 1 2 4 8 N Robots 0.00 0.05 0.10 0.15 0.20 0.25 RMSE n=24 n=24 n=24 n=24 T otal Budget 10 15 30 Figure 7.4: Error vs. Budget (number of planning steps B T ). Right shows results further separated by team size. case where there is no communication, we do not notice the same dependency on N for the effect ofα; in other words, the decrease in error due toα appears relatively similar across different values of N (shown by relative median and maximum errors). We additionally note that for N= 8, there is a statistically significant drop in error fromα = 0.33 to 0.66 (T=33, p≤ 0.05). Based on these results, if robots must be deployed near each other but cannot communicate, then a small robot team may do just as well as a larger team, and vice versa; a small team that cannot communicate does not need to be spread very far to be effective. On the other hand, a larger spread combined with a large team size will be beneficial in the absence of inter-robot communication, but there will be marginal returns with larger spread. If communication is enabled, a larger spread will generally always be beneficial (although again with marginal returns), regardless of the team size. 89 False True Complete Budget 0.00 0.05 0.10 0.15 0.20 0.25 RMSE n=24 n=24 False True Complete Budget 0.00 0.05 0.10 0.15 0.20 0.25 RMSE n=32 n=32 N Robots 1 2 4 8 Figure 7.5: Error vs. Budget Type (Shared and Complete). Here, B T = 15. Left: Shared budget means the combined budget is B T for any N, while complete budget means it is N· B T . Note N = 1 data not included here since budget type has no effect. Right: Results further separated by team size. N= 1 (blue) shown for comparison. 7.5.2 Planning Budget Observing both the median and maximum errors on the left side of Figure 7.4 suggests that in general, a larger budget results in better performance. This is expected, as with a larger budgets, robots have more time to explore the environment and gather information, and we see a similar trend on the right when controlled for team size. The improvements are not entirely consistent, however, as we see that B T = 30 is not always better than B T = 15. This is somewhat surprising, as we would expect improvement with more steps given the results on the left. What we observe on the right, rather, is that the performance improvement comes from an increase in N, indicated by lower median error for higher N. To investigate this further, we control for total path length∑ n B n and show results for shared and com- plete budgets in Figure 7.5. Recall that with a shared budget, ∀n∈ N : B n = B T /N, and for a complete budget,∀n∈ N : B n = B T (see Section 7.4.2). With B T = 15 andα= 0.66 in this experiment, we see similar performance between shared and complete budgets, which is again counterintuitive, as we would generally 90 None Stochastic Full Partitioned Comms Type 0.000 0.025 0.050 0.075 0.100 0.125 0.150 0.175 0.200 RMSE n=24 n=24 n=24 n=24 1 2 4 8 N Robots 0.00 0.05 0.10 0.15 0.20 RMSE n=8 n=32 n=32 n=32 Comms Type None Stochastic Full Partitioned Figure 7.6: Error vs. Communication type. Left: Aggregated across different N. Note that results for N= 1 are not included, since they are not affected by the variations. Right: Same results, further separated by team size. N= 1 included for comparison. expect improvement given more steps. However, we uncover the real effect when we further control for N, shown on the right. With a complete budget, more robots (i.e., more steps) leads to better performance as well, as expected. However, even when the budget is shared, there is a notable improvement in maximum error when using more than 1 robot. What is notable is that, although we see similar median errors for all cases, the improvement in maximum error is also observed when the total path length is held constant. This is particularly remarkable because in the N= 8 case, this equates to each robot taking at most 2 steps. Given these results, it appears that having more robots, even with restricted budgets (e.g., time, path length), is preferable to having one robot with a larger budget. 7.5.3 Team Size and Communication Figure 7.6 shows the quantile estimation error results when the robot team uses different communication styles. Note that on the right, we include the single robot version as a baseline. 91 On the left, we see that overall, all communication types have a median performance around the same as no communication. We see that more communication does, however, lead to improved performance, indicated by smaller upper quartile errors. This is likely due to the fact the robots can avoid multiple measurements of locations that are not of interest. On the right, we see that stochastic communication does not have much of an effect compared to no communication when N = 2 or 4, as seen in the similar median and upper quartile errors. At N = 8, we see that although the median performance between no, stochastic, and full communication is similar, both the upper quartile error and the maximum error for no communication is much higher. This suggests that enabling communication is more beneficial for larger teams for reducing error and improving performance. We also see that in general, partitioning achieves similar median error as when there is stochastic or full communication, as well as maximum error particularly for larger teams. Given that in this experiment, the partitioned case used α = 1.0 rather than 0.66, this suggests that partitioning the space is an effective strategy if the entire area is accessible for deploying robots but communication is not possible. Based on these results, performance improvement can be achieved by enabling inter-robot communication, even when the communication is imperfect, and it is particularly beneficial with large teams. Finally, we show some example plans produced during these experiments as a visualization of the task in Figures 7.7, 7.8 and 7.9. The robots are shown in different colors and the paths they produce are shown as connected crosses. The initial locations are marked in black. If the robots were partitioned, the assigned re- gions are designated with white lines. The background of the figures is the orthomosaic of the hyperspectral images previously collected by drones in real aquatic environments. 7.6 Discussion Here we summarize the main conclusions drawn from the presented results and discuss further areas of interest based on them. 92 0 81 m 0 58 m 0 1 2 3 0 81 m 0 58 m 0 1 2 3 Figure 7.7: Example paths of a four-robot team with B T = 10 (left) and B T = 30 (right). 0 81 m 0 58 m 0 0 81 m 0 58 m 0 1 2 3 4 5 6 7 Figure 7.8: Example paths of a single robot with B T = 15 (left) and an 8-robot team with a shared budget of B T = 15 (right). 0 81 m 0 58 m 0 1 2 3 4 5 6 7 0 81 m 0 58 m 0 1 2 3 4 5 6 7 Figure 7.9: Example paths of an 8-robot team under partitions (left) and no communication (right). 93 With regards to the initial location spread, a large spread appears to be beneficial when N is large, but not when N is small. Additionally, past a certain point between 0.33 and 0.66, a larger spread results in marginal improvement in error. This may be particularly valuable information when planning real-world field deployments, when there are time and monetary costs to getting many robots to spatially diverse areas; avoiding unnecessary costs is crucial. In addition, ifα is restricted to very low values by the survey resources or setup, then using more robots may help regain some performance. Notably, in terms of budget B T , a larger budget in and of itself does not significantly improve perfor- mance for a fixed N (assuming a moderate initial spread). What we observe instead is a stronger dependency on N itself; therefore, a larger number of robots, each with a restricted budget, will likely produce better quantile estimates than a single robot with a B T equal to the total budget of all of them. Our results also indicate that enabling communication is generally beneficial to the quantile estimation problem, and in particular for larger groups. We note that partitioning the space into disjoint areas and restricting each robot to one area can be an effective alternative strategy, particularly if inter-robot commu- nication is not an option. A natural question that arises from these results is to determine what role communication plays in this task, and we address this in Chapter 8. For the purposes of this study, we restricted the communication frameworks to somewhat basic implementations; results may change given a more intelligent communica- tion protocol that takes into account the potential utility of a message, or with more nuanced communication models based on particular hardware capabilities. It would also be interesting to investigate how this scales to team sizes beyond 8 robots. In the context of lake monitoring for algae and other environmental monitoring tasks, swarms are not typically deployed; however, for large-scale studies, it may be useful to understand the impact of teams larger than 8. Based on our results in this study, we would expect to see lower error with more robots up to a certain point. Our results support what previous work has found in terms of diminishing returns with more robots. We 94 generally observe a larger decrease in error between N= 2 and 4 than between 4 and 8. Extrapolating from this, doubling N from 8 to 16 robots or more would likely not be able to achieve 0 error, but would be significantly more resource-intensive. In practice, scientist teams will need to weigh the benefit of lower estimation error with the practicality of acquiring, coordinating, and deploying more robots for their specific application. 7.7 Conclusion In this work, we have presented the first study on multirobot quantile estimation for environmental analy- sis. We investigate the impact of multiple robots on quantile estimation accuracy, as well as the effect of initial location spread, planning budget, and communication. We measure how well different combinations of parameters perform in terms of error in quantile value estimates, and we further provide statistical results quantifying the significance of the differences observed. This work is an important step toward character- izing the elements of an effective multirobot system for environmental analysis and has potential to help scientists interested in larger scale or collaborative survey projects to better understand the benefits and drawbacks of different experiment design choices. In turn, this may improve environmental monitoring and conservation efforts. 95 Chapter 8 Reducing Network Load via Message Utility Estimation for Decentralized Multirobot Teams Our final contribution follows from the analysis of results presented in Chapter 7. With multirobot teams in an environmental exploration task, efficient communication can be crucial to successful task completion. The work described in this chapter considers the problem of abstracting task specification from the angle of autonomously selecting which messages to send to a bandwidth-constrained robot communication network, and follows from [116]. 8.1 Introduction Quantile estimation is useful for gaining an understanding of natural environments [115], and the use of mul- tirobot teams has shown promise for effectively planning exploration and accurately estimating the quantile values [114]. However, realistic multirobot deployments must consider limitations in communication capa- bilities; here, we focus on reducing the network load as measured by the number of attempted transmitted messages with the goal of sacrificing minimal quantile estimation accuracy. We address the question of how to decide whether a message is worth sending. Specifically, we: 96 Figure 8.1: Overview of problem pipeline. The approach consists of 3 phases: Initialization, Exploration, and Aggregation. The bulk of the work happens during the decentralized exploration phase, where robots repeatedly choose the next location to visit, measure the environment, optionally broadcast the measurement in a message, and update their models until their budget B T is exhausted. • Propose methods to reduce network load for bandwidth- and resource-constrained decentralized mul- tirobot teams that leverage utility-based message evaluation, with no guarantees on information from teammates; • Describe a decision-theoretic approach to choosing whether to transmit a message to the network based on the utility-based evaluation and a cost of transmission; • Present results showing significant reductions in network load are achievable with minor or no cor- responding drops in performance on the problem task of estimating quantile values, even with prob- abilistic transmission failure and a distance-based transmission probability, on a dataset from a real aquatic environment; • Discuss results in the context of real-world applications. We assume a decentralized team – each robot plans independently and is not guaranteed to receive information from others. However, we assume a communication protocol such that a robot can, at each step, decide to broadcast its newly acquired measurements to the network. The network has limited bandwidth, so robots are incentivized to only send those that will be most beneficial to the others. We encode the cost of loading the network with a message through a threshold value, where utilities higher than the threshold 97 have a higher benefit than cost. With these assumptions in place, the question of how to evaluate a potential message for its utility emerges as a key consideration to this problem; in this work, we propose several approaches to solve this problem. An overview of the problem pipeline is displayed in Figure 8.1. 8.2 Background There is extensive literature in utility-based communication, and our methods share similarities with previ- ously proposed approaches (see Section 5.6.2). In particular, some other works have made use of a measure of utility based on expected reward or expected effect on robot actions, and the teams are decentralized in terms of planning and reward calculation [90, 149]. While our work in this chapter employs these as well, a major difference is that our end goal is to explore the environment in whatever way results in the best quantile estimates, while those works define a static goal location for the robots to reach. We also exclu- sively choose to plan using a greedy algorithm for computational efficiency and tractability as opposed to using MDPs, motivated by resource-constrained systems, and do not forward-simulate the models of other robots while [90] does. Significantly, we make the (more realistic) assumption of stochastic communication on top of the communication decision; just because a robot decides to send a message does not guarantee reception by all or any of the recipients. We further do not assume perfect measurement sensing. Despite our relatively simple planning and modeling methods and more failure-prone communication assumptions, we are able to achieve a significant decrease in network loading while maintaining reasonable, and in some cases improving, task performance. 8.3 Methods Our formulation extends the setup used in the previous Chapters 6 and 7; we briefly mention relevant def- initions here. The team consists of N aerial robots, each operating with a fixed planning budget B T at a 98 fixed height in a discretized environment G # ⊂ R 2 . Robots can take one of four possible actions at each step:± x or± y. Since our focus is on inter-robot communications, we assume a low-level controller, a col- lision avoidance routine, and perfect localization. The human operator supplies a set of arbitrary quantiles of interest Q, and the end goal is to produce accurate estimates ˜ V of Q where the true values are defined as V = quantiles(Y # ,Q), the set of all possible locations a robot can measure with its sensor areX # , and the corresponding true values at those locations areY # . Each robot has a camera with which it can take noisy measurementsY representing the concentration of algae at those pixel locations in the environment X. Throughout the deployment, each robot collects measurementsX,Y both firsthand via its camera and via any messages received from others through the network. The robots use all their collected measurements to construct and update their internal Gaussian process (GP) model of the environment. Robot i’s estimate of the quantile values is given by ˜ V i = quantiles(µ GP i(X # ),Q) (8.1) whereµ GP i(X # ) is the estimate of all possible locations using the GP conditioned on the measurementsX i andY i that i has collected thus far. In what follows, when viewing the problem from the perspective of a given robot, we will refer to it as i or the ego robot and to any of the other N as j. Variables superscripted with j such as GP j indicate the GP for j from the perspective of i. Motivated by the computational constraints of resource-constrained systems, we opt to use a locally computable, online, greedy planning policy. At each planning step t, i maximizes an objective function f over feasible next measurement locations. We use a modified version of the quantile standard error objective function to evaluate a proposed location for time t proposed in Chapter 6, which measures the difference in the standard error of the estimated quantile values using two versions of the robot model: the current model µ GP i t− 1 , and the model updated with the hypothetical new values at the proposed locations µ GP i t . We modify f such that it is conditioned not only on the proposed measurement locationsX, but also the values 99 associated with the locations (denoted byY generally) and the GP to be used. The reasoning is that f is also used later in the process for computing message utilities, and this allows it to generalize to these cases when Y and GP may take on different values. When f is used in the planning step by i to select i’s next location, Y are the expected values at the proposed locationX: µ GP i t− 1 (X) (just like in Chapter 6). However, when f is used by i to determine the utility of j receiving a real measurement that i has already acquired,Y are thoseY , and the GP may be either GP i or GP j (see Section 8.3.3). The final term encourages exploration of high-variance areas. f(X i ;Y,GP)= d |Q| + ∑ x j ∈X i cσ 2 (x j ) (8.2) d=∥se(µ GP i− 1 (X # ),Q)− se(µ GP i (X # ),Q)∥ 1 (8.3) 8.3.1 Summary of Process The approach consists of 3 phases: Initialization, Exploration, and Aggregation. These are described at a high level here; further details are in the sections following. 8.3.1.1 Initialization Robots begin distributed in the environment according to the initial location spread α (Section 7.3.1 pro- vides details). We assume each robot is given the number of other robots and their corresponding starting locations. 8.3.1.2 Exploration The bulk of the work happens during the decentralized exploration phase, where robots repeatedly choose the next location to visit, measure the environment, optionally broadcast the measurement in a message, and 100 update their models until their budget B T is exhausted. The next measurement locations at t are selected according to X i t = arg max X⊂ X i next f(X;µ GP i(X),GP i ) (8.4) whereX next contains the sets of locations that could be sensed one step from the current location, g i t . On top of this planning cycle, i may receive a message from another j at any point, leading to further model updates (Section 8.3.3). 8.3.1.3 Aggregation Once the robots have reached the allotted budgets, their data is collected. The operator can produce an aggregate GP model of the environment offline using it and, with that, obtain final estimates of quantile values: ˜ V final = quantiles(µ GP aggregate (X # ),Q) (8.5) 8.3.2 Communication Framework The communication network consists of N robots, each with the ability to broadcast a message at each step in its path. A messageM i t =(i,g i t ,X i t ,Y i t ) sent by i at time t is composed of a header containing the sending robot’s ID number and current location, and a body containing the current set of measurements. Messages are transmitted successfully to every other robot j in the network subject to the probability depending on distance and parameterized by the dropoff rateη and communication radius r [114]: p success (distance)= 1 1+ e η(distance− r) (8.6) and a successfully transmitted message will be received whole, uncorrupted, and without delay. In general, we assume i does not receive a handshake or confirmation of whether its message was received successfully 101 or not by j. We assume that robots can self-localize and that, within the radius r, they can accurately sense g j , which they do at each step. 8.3.3 Modeling Others and Updating Models A central consideration is how to model other robots using incomplete information since such a model is necessary to adequately assess the utility any given message will have for others. The ego robot i maintains a separate model (g j , GP j ) for each j. g j is the most up-to-date location i has, and is updated in two cases. First, by i’s sensing capabilities at each step if j is within radius r, and second, anytime i receives a message from j, via the header. GP j is a GP based on the measurements i believes j has. We denote this set of measurements as M j :=(X j ,Y j ). M j , and GP j , which is defined by it, can also be updated in two scenarios. The first is when i receives a messageM from j, since it is clear that j must possess the data contained. The second is more uncertain. If i broadcastsM to the network, it does not know whether it was successfully transmitted to any j; however, recall that i can sense the location of j within r. We use this to determine a proxy for reasonable confidence of transmission success: If j is within r, i believesM was received and thus updates M j . 8.3.4 Computing Message Utility We introduce several methods for computing the utility u j ofM i =(i,g i ,X,Y) (dropping the super- and subscripts for readability). The first, Reward, defines u rw as the reward i believes j would receive, using the objective function f : u rw (X,Y, j)= ∞, if |X j |= 0 f(X;Y,GP j ), otherwise (8.7) We call the second method Action. Intuitively, it considers a message utile if i believes that adding it to GP j would result in j taking a different action than it would without those measurements. Then u ac is 102 defined as ∞ if the actionsX next,w/o andX next,w/ differ as computed using Equation (8.4), and if they do not, u ac takes the value of u rw : u ac (X,Y, j)= ∞, if|X j |= 0 ∞, ifX next,w/o ̸=X next,w/ u rw (X,Y, j), otherwise (8.8) Both u rw and u ac include the caveat that, if i believes j has collected no measurements yet, the message should be sent. The last three methods presented are simpler in that they do not make use of the model of j. The third method, Ego-reward, uses the reward i received from its measurement: u ego (X,Y, j)= f(X,Y ;GP i ) (8.9) Finally, Always and Never are constant baselines, which always assign either∞ or 0: u al (X,Y, j)=∞ (8.10) u nv (X,Y, j)= 0 (8.11) 8.3.5 Deciding to Communicate With the utilities of a message{u j } computed for each j, i now faces the problem of deciding whether the utility overcomes the cost of loading the network with a message. Since the communication is broadcast- based rather than point-to-point, i must first compute a single utility value U for the team and then determine if it crosses a utility threshold T representing the cost. For this, we describe a method for aggregating a 103 Table 8.1: Number of attempted and successful transmitted messages per utility method. Restricting trans- missions with Action results in more than 40% reduction in network load (attempted) while improving mean final quantile estimation error by 1 .84% and only resulting in a less than 25% decrease in successful trans- missions. Results reported from 60 experiments total of length 40 steps. All experiments use N= 4, so the most messages possibly sent total is 40× (N− 1)= 120. Method Attempted Median Successful Median Median (µ,σ) Decrease (µ,σ) Decrease RMSE Increase Action (66.6, 27.95) 42.5% (38.2, 23.85) 24.32% -1.84% Always (120.0, 0.0) – (44.93, 16.78) – – Ego-reward (76.2, 28.33) 27.5% (34.4, 20.27) 32.2% 5.96% Never (0.0, 0.0) 100.0% (0.0, 0.0) 100.0 73.98% Reward (68.2, 32.1) 52.5% (36.0, 22.77) 35.96% 16.03% set of utilities which computes expected utility, weighting utilities by the estimated transmission success probability: U EU ({u j })= 1 N− 1 ∑ j (p est ( j)· u j ) (8.12) p est ( j)= p success (d j ), if d j ≤ r p success (r), otherwise (8.13) where d j =||g i − g j || 2 and r is as defined in Section 8.3.2. The estimated success probability, computed in Equation (8.13), directly uses the probability based on the distance to j if it is known (i.e., if j is within the sensing radius r), but defaults to an optimistic probability estimate for any j that is beyond the sensing radius by using r as the assumed distance. This optimistic heuristic prevents overly restrictive utility values in the face of unknown teammate positions. Note that if any u j is∞, then U is considered past the threshold T . If U≥ T , i decides to transmitM . 8.4 Experiments and Discussion We now present experimental results for the methods described in Section 8.3. We test performance on 3 sets of quantiles Q: Quartiles (0.25, 0.5, 0.75), Median-Extrema (0.5, 0.9, 0.99), and Extrema (0.9, 0.95, 0.99). 104 −50% −25% 0% 25% 50% 75% 100% 125% Percent change from sending all messages Decrease in messages sent Increase in final error Measure Action Ego-reward Never Reward Figure 8.2: Tradeoff in network load vs. task performance for different utility methods, compared to always sending messages. Top: Percent decrease in messages sent (network load). Higher is better. Bottom: Percent increase in final quantile estimation error. Lower is better. For each combination of parameters, we run 5 seeds, resulting in 60 total instances. We fix the following parameters: N= 4; budget B T = 10 (40 steps total); initial spreadα = 0.2;η = 0.4; communication radius r = 15.0; thresholds T rw = 2.8× 10 − 4 and T ego = 8.3× 10 − 5 . T rw is used for Reward and Action, while T ego is used for Ego-reward. We selected these T experimentally by observing the received u values during testing and setting T to the approximate 25th percentile of observed rewards. This can be interpreted as only messages that are in the top 75% of utility being worth the cost of sending to the network, while those in the lower quarter are not. The robots operate in simulation using a real-world dataset collected with a hyperspectral camera mounted on a drone from a lake in California. The environment is roughly 80× 60 meters, discretized toG # = 25× 25 locations. Each set of measurements is an image of 5× 5 pixel intensities for the 400nm 105 5 10 15 20 25 30 35 40 Step 0 20 40 60 80 100 120 Cumulative Attempted Comms Cumulative network load over time Action Always Ego-reward Never Reward Figure 8.3: Cumulative network load, as measured by messages sent, over time. Each step on the x-axis represents a robot taking a planning step. 106 5 10 15 20 25 30 35 40 Step 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Count Utility: Action Attempted Successful 5 10 15 20 25 30 35 40 Step 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Count Utility: Always Attempted Successful 5 10 15 20 25 30 35 40 Step 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Count Utility: Ego-reward Attempted Successful 5 10 15 20 25 30 35 40 Step 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Count Utility: Never Attempted Successful 5 10 15 20 25 30 35 40 Step 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Count Utility: Reward Attempted Successful Figure 8.4: Amount of attempted and successful transmissions over time for different utility methods. Each step on the x-axis represents a robot taking a planning step. 107 channel of the hyperspectral image, normalized to[0,1]. To each measurement, we add zero-mean Gaussian noise with a standard deviation 0.05. 8.4.1 Utility Methods Comparison We investigate the effectiveness of the different methods on performance in terms of both the network load as well as the final error (RMSE) between the estimated quantile values ˜ V final and V . Table 8.1 displays numerical results, including a comparison of each method to Always in terms of the median percent change of each reported metric. These results are shown graphically in Figure 8.2, illustrating the tradeoff between network load and task performance. Figure 8.3 shows the cumulative network load as measured by the attempted transmissions over time, while Figure 8.4 shows the number of attempted and successful message transmissions plotted over time. Figure 8.5 shows boxplots of the final RMSE for different utility methods. Figure 8.6 illustrates example paths taken superimposed on the hyperspectral image of the lake. We observe in Figure 8.4 that across all methods, successful transmissions generally decrease over time due to robots spreading out. Comparing Never to the other methods in Figure 8.5, we can confirm our previous findings that inter-robot communication improves performance. However, we are mainly interested in the effect of decreasing network load. When we analyze results in Table 8.1 and Figure 8.2, we see that severe reductions in communication do not necessarily result in severe decreases in performance. We see that Action results in the best performance: Compared to Always, there is more than 40% decrease in network load as measured by attempted message transmissions, but this translates to less than 25% decrease in actual (successful) message transmissions. Most notably, in terms of final error, there is a 1 .84% decrease in mean RMSE, indicating that performance, on average, improves slightly even with restricted communication. Other reward-based methods, Reward and Ego-reward, also perform favorably. Figures 8.3 and 8.4 sug- gest that all three proposed methods have message transmission rates that generally decrease over time. We 108 speculate that this may be due to two reasons. First, as robots explore more of the environment and receive more information from the others, their beliefs of their teammates will change such that|X j | increases, leading to more accurate environment models and thus less value in receiving an additional set of measure- ments. Second, with more exploration time, robots are more likely to be situated at further distances from each other. With distance, the expected utility will decrease according to the transmission probability, and even messages believed to be useful may not be deemed worthy because of their unlikelihood of successful delivery. We note, however, that the optimistic assumption in Equation (8.13) will prevent a highly useful message from not being sent due to distance. As shown qualitatively in Figures 8.3 and 8.4 and quantitatively in Table 8.1, Action, Reward, and Ego- reward all have lower network load than Always, but their relative loads vary as do their final estimation error. In particular, Reward results in the highest decrease in network load (excluding Never) at 52.5%, but also the largest increase in final error of more than 16%. Thus, such a metric may be valuable when network bandwidth is severely limited. Ego-reward, on the other hand, has the smallest decrease in network load at 27.5%, but a relatively small increase in error at less than 6%. Ego-reward has the benefit of not requiring any onboard modeling of teammate beliefs; thus, this method could be powerful when onboard compute is limited. Action is the overall best performer, with a decrease in network load of 42.5% and a decrease in final error of 1 .84%. We suspect Action performs better because it directly considers the impact of a message on future behavior, leading to more informed and targeted measurement acquisition at the next step. We note that the comparison of relative network loads is more valuable here than the absolute value, as the threshold values used in Equation (8.12) are tunable parameters that should reflect actual network constraints in a real deployment. 109 Action Always Never Reward Comms Utility Method 0.000 0.025 0.050 0.075 0.100 0.125 0.150 0.175 RMSE n=15 n=15 n=15 n=15 Figure 8.5: Final estimation error for different utility methods. 8.4.2 Oracle Handshaking We implement a modification to the communication protocol where we assume an oracle handshaking rou- tine, which gives i an accurate confirmation handshake from each j indicating whether the message was successfully received; i then only updates M j when successful, leading to likely more correct models. Fig- ure 8.7 shows the final RMSE comparing the two; we see that there is a significant improvement with oracle handshaking (Wilcoxon signed-rank test T=1143, p≤ 0.05). Though practically impossible, this suggests that implementing such a (probabilistic) routine in the network could improve performance further; however, the resulting burden of these messages must also be taken into account. 8.5 Conclusion In this chapter, we presented a study on restricted communication in the context of quantile estimation for environmental analysis. We show that restricting message transmission based on whether the actions that the sender believes other robots will take at the next state would change with the message results in a 110 0 81 m 0 58 m 0 1 2 3 0 81 m 0 58 m 0 1 2 3 Figure 8.6: Example paths using two different utility methods with Q = (0.9,0.95,0.99). Top: Action. Bottom: Reward. 111 F alse T rue Handshak e 0.000 0.025 0.050 0.075 0.100 0.125 0.150 0.175 0.200 RMSE n=60 n=60 * Figure 8.7: Right: the sender receives an oracle handshake when a message is successfully transmitted. Bar indicates significance under one-sided Wilcoxon signed-rank test; *: p≤ 5e− 2. 42% decrease in network load while also decreasing quantile estimation error by 1.84%, when compared to transmitting every message possible. Our analysis indicates that targeted communication during quantile estimation can have real benefits to real-world multirobot deployments when robots and their network are resource-constrained. Importantly, these results allow users to relegate the task of deciding which messages to send to a network in an intelligent manner to the robots. In addition to receipt handshaking, there are several interesting directions of future work. First, we have chosen to implement robot planning using greedy action selection, but it would be valuable to extend this method to use a non-myopic planning strategy. We also believe allowing the sender to choose the most valuable measurements from its collected data rather than restricting it to the most recent measurements may improve performance. However, this would lead to challenges in scalability as the environment is explored, and could be addressed by applying a rolling window over the previous measurements to mitigate this or by keeping and updating a limited set of high-utility measurements from which messages are selected. Further, we are interested in methods for better approximating the models that a robot maintains for each of its 112 teammates, which could be achieved by augmenting message headers with metadata such as that robot’s current dataset size or current quantile estimates. Finally, choosing the threshold value T is an important hyperparameter choice. It would be interesting and valuable to investigate ways in which this value could be automatically extracted or learned either offline or during exploration. 113 Chapter 9 Conclusion In this thesis, we have proposed solutions which have advanced robot autonomy for long-horizon tasks in multiple different ways, by considering how task specification can be abstracted to interpretable and flexible descriptions. To this end, we presented work in the domain of constrained manipulation and mobility, where we were able to avoid hand-specification of transition points between subtasks of a sequenced task by auto- matically discovering optimal constraint manifold intersection points. We further proposed a way to learn the constraint manifolds themselves, bypassing the need to manually specify each constraint analytically. We also presented work in the domain of robotic environmental monitoring. We proposed a flexible task specification framework allowing direct solutions to scientifically-grounded goals, as opposed to a generic solution. This led us to systematically study how task specification with multirobot teams can be effec- tively used in these types of problems, finding that among various factors, inter-robot communication is key. Finally, we developed a method which exploited this fact while respecting realistic network bandwidth re- strictions, by autonomously selecting useful but limited information to share with the robot network. These works, together, present advances along several axes for task specification abstraction, which we believe has the potential to lead to more interpretable, useful, and autonomous robot deployments for real-world, long-horizon tasks. 114 Bibliography [1] Eddie Aamari, Jisu Kim, Frédéric Chazal, Bertrand Michel, Alessandro Rinaldo, Larry Wasserman, et al. “Estimating the Reach of a Manifold”. In: Electronic journal of statistics 13.1 (2019), pp. 1359–1399. URL: https://projecteuclid.org/euclid.ejs/1555056153. [2] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems. 2015. URL: http://download.tensorflow.org/paper/whitepaper2015.pdf. [3] Nancy Marie Amato and Yan Wu. “A randomized roadmap method for path and manipulation planning”. In: Proceedings of the International Conference on Robotics and Automation. 1996. URL: https://ieeexplore.ieee.org/abstract/document/503582. [4] Brenna D. Argall, Sonia Chernova, Manuela Veloso, and Brett Browning. “A survey of robot learning from demonstration”. In: Robotics and Autonomous Systems 57.5 (2009), pp. 469–483. [5] Jennifer Barry, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. “A hierarchical approach to manipulation with diverse actions”. In: 2013 IEEE International Conference on Robotics and Automation. IEEE. 2013, pp. 1799–1806. URL: https://ieeexplore.ieee.org/abstract/document/6630814?casa_token=RNM-qYKIPVMAAAAA: bQHrsYnAR3iIWtz79JBUNAEoPf5mx2t0tF-XAxz1J1- TZHWWDQKsyj95wWxtMGtNiYEV6vTy. [6] Dmitry Berenson, Siddhartha Srinivasa, and James Kuffner. “Task space regions: A framework for pose-constrained manipulation planning”. In: The International Journal of Robotics Research 30.12 (2011), pp. 1435–1460. URL: https://journals.sagepub.com/doi/abs/10.1177/0278364910396389. [7] Dmitry Berenson, Siddhartha S Srinivasa, Dave Ferguson, and James J Kuffner. “Manipulation planning on constraint manifolds”. In: 2009 IEEE International Conference on Robotics and Automation. IEEE. 2009, pp. 625–632. URL: https://ieeexplore.ieee.org/abstract/document/5152399. 115 [8] Antoine Blanchard and Themistoklis Sapsis. “Informative Path Planning for Extreme Anomaly Detection in Environment Exploration and Monitoring”. In: arXiv:2005.10040 (2021). [9] Riccardo Bonalli, Abhishek Cauligi, Andrew Bylard, Thomas Lew, and Marco Pavone. “Trajectory Optimization on Manifolds: A Theoretically-Guaranteed Embedded Sequential Convex Programming Approach”. In: Proceedings of Robotics: Science and Systems. 2019. URL: https://arxiv.org/abs/1905.07654. [10] William Munger Boothby. An introduction to differentiable manifolds and Riemannian geometry; 2nd ed. Pure Appl. Math. Orlando, FL: Academic Press, 1986. [11] Ricard Bordalba, Lluís Ros, and Josep M Porta. “Randomized kinodynamic planning for constrained systems”. In: Proceedings of the International Conference on Robotics and Automation. 2018. URL: https://ieeexplore.ieee.org/abstract/document/8460753. [12] Stéphane Cambon, Rachid Alami, and Fabien Gravot. “A hybrid approach to intricate motion, manipulation and task planning”. In: The International Journal of Robotics Research 28.1 (2009), pp. 104–126. URL: https://journals.sagepub.com/doi/abs/10.1177/0278364908097884?casa_token= qaICubcZMOMAAAAA:EuGMWAglhdZ6BgzMWvRxF9b7ZDFL_X3Dk4D2QR2_Q7_ IOEviR0Z8EMhtwVEtO8BB3p6bN5jjr08S. [13] Nannan Cao, Kian Hsiang Low, and John M Dolan. “Multi-robot informative path planning for active sensing of environmental phenomena: A tale of two algorithms”. In: arXiv preprint arXiv:1302.0723 (2013). [14] Florian Charlier, Marc Weber, Dariusz Izak, Emerson Harkin, Marcin Magnus, Joseph Lalli, Louison Fresnais, Matt Chan, Nikolay Markov, Oren Amsalem, Sebastian Proost, Agamemnon Krasoulis, getzze, and Stefan Repplinger. Statannotations. Version v0.5. Oct. 2022. DOI: 10.5281/zenodo.7213391. [15] N. Chen, M. Karl, and P. van der Smagt. “Dynamic movement primitives in latent space of time-dependent variational autoencoders”. In: IEEE-RAS 16th Humanoids. 2016, pp. 629–636. URL: https://ieeexplore.ieee.org/abstract/document/7803340. [16] Lillian Clark, Joseph Galante, Bhaskar Krishnamachari, and Konstantinos Psounis. “A queue-stabilizing framework for networked multi-robot exploration”. In: IEEE Robotics and Automation Letters 6.2 (2021), pp. 2091–2098. [17] M. Corah and N. Michael. “Distributed matroid-constrained submodular maximization for multi-robot exploration: theory and practice”. In: Autonomous Robots 43.2 (Feb. 2019), pp. 485–501. [18] Juan Cortés and Thierry Siméon. “Sampling-based motion planning under kinematic loop-closure constraints”. In: Algorithmic Foundations of Robotics VI. Springer, 2004, pp. 75–90. URL: https://link.springer.com/chapter/10.1007/10991541_7. [19] Neil T Dantam, Swarat Chaudhuri, and Lydia E Kavraki. “The Task-Motion Kit: An open source, general-purpose task and motion-planning framework”. In: IEEE Robotics & Automation Magazine 25.3 (2018), pp. 61–70. URL: https://ieeexplore.ieee.org/abstract/document/8360542. 116 [20] Neil T. Dantam, Zachary K. Kingston, Swarat Chaudhuri, and Lydia E. Kavraki. “Incremental Task and Motion Planning: A Constraint-Based Approach”. In: Proceedings of Robotics: Science and Systems. 2016. URL: http://www.roboticsproceedings.org/rss12/p02.pdf. [21] Jnaneshwar Das, Frédéric Py, Julio B.J. Harvey, John P. Ryan, Alyssa Gellene, Rishi Graham, David A. Caron, Kanna Rajan, and Gaurav S. Sukhatme. “Data-driven robotic sampling for marine ecosystem monitoring”. In: IJRR 34 (Oct. 2015). [22] Pieter-Tjerk De Boer, Dirk P Kroese, Shie Mannor, and Reuven Y Rubinstein. “A tutorial on the cross-entropy method”. In: Annals of operations research 134 (2005). [23] Chris Denniston, Aravind Kumaraguru, and Gaurav S. Sukhatme. “Comparison of Path Planning Approaches for Harmful Algal Bloom Monitoring”. In: OCEANS MTS/IEEE. 2019. DOI: 10.23919/OCEANS40490.2019.8962687. [24] Christopher E. Denniston, Gautam Salhotra, David A. Caron, and Gaurav S. Sukhatme. “Adaptive Sampling using POMDPs with Domain-Specific Considerations”. In: ICRA. 2020. [25] Shay Deutsch and Gérard Medioni. “Unsupervised Learning Using the Tensor V oting Graph”. In: Scale Space and Variational Methods in Computer Vision. Ed. by Jean-François Aujol, Mila Nikolova, and Nicolas Papadakis. Cham: Springer International Publishing, 2015, pp. 282–293. ISBN: 978-3-319-18461-6. [26] Piotr Dollár, Vincent Rabaud, and Serge Belongie. “Non-isometric manifold learning: Analysis and an algorithm”. In: 24th ICML. 2007, pp. 241–248. [27] Richard O Duda, Peter E Hart, and David G Stork. “Pattern Classification”. In: John Wiley & Sons, Inc. 2 (2001). [28] Ayan Dutta, Amitabh Bhattacharya, O Patrick Kreidl, Anirban Ghosh, and Prithviraj Dasgupta. “Multi-robot informative path planning in unknown environments through continuous region partitioning”. In: International Journal of Advanced Robotic Systems 17.6 (2020), p. 1729881420970461. [29] Ayan Dutta, Anirban Ghosh, and O Patrick Kreidl. “Multi-robot informative path planning with continuous connectivity constraints”. In: 2019 International Conference on Robotics and Automation (ICRA). IEEE. 2019, pp. 3245–3251. [30] Peter Englert, Isabel M. Rayas Fernández, Ragesh K. Ramachandran, and Gaurav S. Sukhatme. Sampling-Based Motion Planning on Manifold Sequences. arXiv:2006.02027. 2020. URL: https://arxiv.org/abs/2006.02027. [31] Peter Englert, Ngo Anh Vien, and Marc Toussaint. “Inverse KKT — Learning Cost Functions of Manipulation Tasks from Demonstrations”. In: IJRR 36.13-14 (2017), pp. 1474–1488. [32] Chelsea Finn, Sergey Levine, and Pieter Abbeel. “Guided cost learning: Deep inverse optimal control via policy optimization”. In: Proceedings of the International Conference on Machine Learning. 2016. 117 [33] Y . Gabriely and E. Rimon. “Spiral-STC: an on-line coverage algorithm of grid environments by a mobile robot”. In: ICRA. 2002. [34] Alkis Gotovos. “Active Learning for Level Set Estimation”. en. MA thesis. Zürich: ETH Zurich, 2013. [35] Carlos Guestrin, Andreas Krause, and Ajit Paul Singh. “Near-optimal sensor placements in Gaussian processes”. In: ICML. 2005. [36] A. Hatcher, Cambridge University Press, and Cornell University. Dept. of Mathematics. Algebraic Topology. Algebraic Topology. Cambridge University Press, 2002. [37] Kris Hauser and Victor Ng-Thow-Hing. “Randomized multi-modal motion planning for a humanoid robot manipulation task”. In: The International Journal of Robotics Research 30.6 (2011), pp. 678–698. URL: https: //journals.sagepub.com/doi/abs/10.1177/0278364910386985?casa_token=auiT_SW7epMAAAAA: hwPka6jrCCp_2T0nxDGj1fEPjP19Tt-PjC_lZTh8xU0q1uCqzq1n1osi8fRYZoCtElG6CowpRq3X. [38] Ioannis Havoutis and Subramanian Ramamoorthy. “Motion synthesis through randomized exploration on submanifolds of configuration space”. In: Robot Soccer World Cup. Springer. 2009, pp. 92–103. [39] Xi He. Quantum subspace alignment for domain adaptation. arXiv:2001.02472. 2020. URL: https://arxiv.org/abs/2001.02472. [40] High-Resolution Mapping of a Harmful Algal Bloom: UNDERSTANDING FINE-SCALE ENVIRONMENTAL FACTORS IN CLEAR LAKE, CALIFORNIA. Tech. rep. 2020. URL: https://video.ysi.com/webinar-expand-your-hab-horizons. [41] Jeff C. Ho, Anna M. Michalak, and Nima Pahlevan. “Widespread global increase in intense lake phytoplankton blooms since the 1980s”. In: Nature (2019). [42] Daniel Holden, Jun Saito, Taku Komura, and Thomas Joyce. “Learning Motion Manifolds with Convolutional Autoencoders”. In: ACM SIGGRAPH Asia 2015 Technical Briefs. Kobe, Japan, 2015. [43] Geoff A Hollinger and Gaurav S Sukhatme. “Sampling-based robotic information gathering algorithms”. In: IJRR (2014). [44] Geoffrey Hollinger and Sanjiv Singh. “Multi-robot coordination with periodic connectivity”. In: 2010 IEEE International Conference on Robotics and Automation. IEEE. 2010, pp. 4457–4462. [45] Geoffrey A. Hollinger, Brendan Englot, Franz S. Hover, Urbashi Mitra, and Gaurav S. Sukhatme. “Active planning for underwater inspection and the benefit of adaptivity”. In: IJRR (2013). [46] Andrew Howard. “Multi-robot simultaneous localization and mapping using particle filters”. In: The International Journal of Robotics Research 25.12 (2006), pp. 1243–1256. 118 [47] Andrew Howard, Lynne E Parker, and Gaurav S Sukhatme. “Experiments with a large heterogeneous mobile robot team: Exploration, mapping, deployment and detection”. In: The International Journal of Robotics Research 25.5-6 (2006), pp. 431–447. [48] Aya Hozumi, Ilia Ostrovsky, Assaf Sukenik, and Hezi Gildor. “Turbulence regulation of Microcystis surface scum formation and dispersion during a cyanobacteria bloom event”. In: Inland Waters (2020). [49] Rob J. Hyndman and Yanan Fan. “Sample Quantiles in Statistical Packages”. In: The American Statistician 50 (1996). [50] Brian Ichter, James Harrison, and Marco Pavone. “Learning sampling distributions for robot motion planning”. In: 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE. 2018, pp. 7087–7094. [51] Brian Ichter and Marco Pavone. “Robot motion planning in learned latent spaces”. In: IEEE Robotics and Automation Letters 4.3 (2019), pp. 2407–2414. [52] Auke Jan Ijspeert, Jun Nakanishi, Heiko Hoffmann, Peter Pastor, and Stefan Schaal. “Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors”. In: Neural Computation 25.2 (2013), pp. 328–373. [53] Woods Hole Oceanographic Institution. Harmful Algal Blooms: Understanding the Threat and Actions Being Taken to Address It. 2022. [54] Léonard Jaillet and Josep M Porta. “Asymptotically-optimal path planning on manifolds”. In: Proceedings of Robotics: Science and Systems. 2013. URL: https://books.google.com/books?hl= en&lr=&id=NOrxCwAAQBAJ&oi=fnd&pg=PA145#v=onepage&q&f=false. [55] Léonard Jaillet and Josep M Porta. “Path planning under kinematic constraints by rapidly exploring manifolds”. In: IEEE Transactions on Robotics 29.1 (2013), pp. 105–117. URL: https://ieeexplore.ieee.org/abstract/document/6352929. [56] Nikolay Jetchev and Marc Toussaint. “TRIC: Task space retrieval using inverse optimal control”. In: Autonomous Robots 37.2 (2014), pp. 169–189. [57] Donald R. Jones, Matthias Schonlau, and William J. Welch. “Efficient Global Optimization of Expensive Black-Box Functions”. In: Journal of Global Optimization (1998). [58] Leslie Pack Kaelbling and Tomás Lozano-Pérez. “Integrated task and motion planning in belief space”. In: The International Journal of Robotics Research 32.9-10 (2013), pp. 1194–1227. URL: https://journals.sagepub.com/doi/abs/10.1177/0278364913484072. [59] Peter Kaiser, Dmitry Berenson, Nikolaus Vahrenkamp, Tamim Asfour, Rüdiger Dillmann, and Siddhartha Srinivasa. “Constellation - An algorithm for finding robot configurations that satisfy multiple constraints”. In: Proceedings of the International Conference on Robotics and Automation. 2012. URL: https://ieeexplore.ieee.org/abstract/document/6224753. 119 [60] Nandakishore Kambhatla and Todd K. Leen. “Dimension Reduction by Local Principal Component Analysis”. In: Neural Comput. 9.7 (Oct. 1997), pp. 1493–1516. ISSN: 0899-7667. DOI: 10.1162/neco.1997.9.7.1493. [61] S. Karaman and E. Frazzoli. “Sampling-based algorithms for optimal motion planning”. In: International Journal of Robotics Research 30 (2011), pp. 846–894. URL: https://journals.sagepub.com/doi/abs/10.1177/0278364911406761. [62] Lydia Kavraki and J-C. Latombe. “Randomized preprocessing of configuration for fast path planning”. In: Proceedings of the International Conference on Robotics and Automation. 1994. URL: https://ieeexplore.ieee.org/abstract/document/350966. [63] Stephanie Kemna, Oliver Kroemer, and Gaurav S. Sukhatme. “Pilot Surveys for Adaptive Informative Sampling”. In: ICRA. 2018. [64] Stephanie Kemna, John G Rogers, Carlos Nieto-Granda, Stuart Young, and Gaurav S Sukhatme. “Multi-robot coordination through dynamic V oronoi partitioning for informative adaptive sampling in communication-constrained environments”. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE. 2017, pp. 2124–2130. [65] Beobkyoon Kim, Terry Taewoong Um, Chansu Suh, and Frank C Park. “Tangent bundle RRT: A randomized algorithm for constrained motion planning”. In: Robotica 34.1 (2016), pp. 202–225. URL: https://www.cambridge.org/core/journals/robotica/article/tangent-bundle-rrt-a-randomized- algorithm-for-constrained-motion-planning/73C83E4CB58FA3C424FED7A3C5936403. [66] Jinkyu Kim, Inyoung Ko, and Frank C Park. “Randomized path planning on foliated configuration spaces”. In: 2014 11th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI). IEEE. 2014, pp. 209–214. URL: https://ieeexplore.ieee.org/abstract/document/7057533?casa_token=456GlCUkbYsAAAAA: geZdL8ELH95XHTwV- z0E2sYHlYmHXHv8syAgzuzMTze8t9BYsPgB73eMRiBh1JAi5C5H4khD. [67] Diederik P. Kingma and Max Welling. “Auto-Encoding Variational Bayes”. In: CoRR abs/1312.6114 (2014). URL: https://arxiv.org/abs/1312.6114. [68] Zachary Kingston, Mark Moll, and Lydia E Kavraki. “Exploring implicit spaces for constrained sampling-based planning”. In: The International Journal of Robotics Research 38.10-11 (2019), pp. 1151–1178. URL: https://journals.sagepub.com/doi/full/10.1177/0278364919868530. [69] Zachary Kingston, Mark Moll, and Lydia E Kavraki. “Sampling-based methods for motion planning with constraints”. In: Annual review of control, robotics, and autonomous systems 1 (2018), pp. 159–185. URL: https://www.annualreviews.org/doi/abs/10.1146/annurev-control-060117-105226. [70] Zachary Kingston, Andrew M Wells, Mark Moll, and Lydia E Kavraki. “Informing Multi-Modal Planning with Synergistic Discrete Leads”. In: Proceedings of the International Conference on Robotics and Automation. 2020, pp. 3199–3205. 120 [71] Scott Kirkpatrick, C Daniel Gelatt Jr, and Mario P Vecchi. “Optimization by Simulated Annealing”. In: Readings in Computer Vision. Elsevier, 1987. [72] M. Kleinbort, K. Solovey, Z. Littlefield, K. E. Bekris, and D. Halperin. “Probabilistic Completeness of RRT for Geometric and Kinodynamic Planning With Forward Propagation”. In: IEEE Robotics and Automation Letters 4.2 (2019). URL: https://ieeexplore.ieee.org/abstract/document/8584061. [73] George Konidaris, Leslie Pack Kaelbling, and Tomas Lozano-Perez. “From skills to symbols: Learning symbolic representations for abstract high-level planning”. In: Journal of Artificial Intelligence Research 61 (2018), pp. 215–289. URL: https://www.jair.org/index.php/jair/article/view/11175. [74] Joseph B Kruskal. Multidimensional scaling. Sage, 1978. [75] James J Kuffner Jr and Steven M LaValle. “RRT-connect: An efficient approach to single-query path planning”. In: Proceedings of the International Conference on Intelligent Robots and Systems. 2000. URL: https://ieeexplore.ieee.org/abstract/document/844730. [76] Steven M. LaValle. Planning Algorithms. USA: Cambridge University Press, 2006. ISBN: 0521862051. URL: https://books.google.com/books?hl=en&lr=&id=- PwLBAAAQBAJ&oi=fnd&pg=PT9#v=onepage&q&f=false. [77] Steven M. LaValle. Rapidly-exploring random trees: A new tool for path planning. Tech. rep. TR 98-11. Computer Science Department, Iowa State University, 1998. URL: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.35.1853. [78] John M Lee. Riemannian manifolds: an introduction to curvature. V ol. 176. Springer Science & Business Media, 2006. URL: https://books.google.com/books?hl=en&lr=&id=92PgBwAAQBAJ& oi=fnd&pg=PA1#v=onepage&q&f=false. [79] Sergey Levine and Vladlen Koltun. “Continuous inverse Optimal Control with Locally Optimal Examples”. In: ICML. 2012. [80] Daniel Liberzon. Calculus of Variations and Optimal Control Theory: A Concise Introduction. USA: Princeton University Press, 2011. ISBN: 0691151873. [81] Stuart Lloyd. “Least squares quantization in PCM”. In: IEEE Transactions on Information Theory 28.2 (1982), pp. 129–137. DOI: 10.1109/TIT.1982.1056489. [82] Kai-Chieh Ma, Zhibei Ma, Lantao Liu, and Gaurav S Sukhatme. “Multi-robot informative and adaptive planning for persistent environmental monitoring”. In: Distributed Autonomous Robotic Systems. Springer, 2018, pp. 285–298. [83] Yunqian Ma and Yun Fu. Manifold learning theory and applications. CRC press, 2011. [84] Ziyuan Ma, Yudong Luo, and Jia Pan. “Learning selective communication for multi-agent path finding”. In: IEEE Robotics and Automation Letters 7.2 (2021), pp. 1455–1462. 121 [85] Ratnesh Madaan, Sam Zeng, Brian Okorn, and Sebastian Scherer. “Learning Adaptive Sampling Distributions for Motion Planning by Self-Imitation”. In: Workshop on Machine Learning in Robot Motion Planning, IROS 2018 (2018). [86] Jeffrey Mahler, Sachin Patil, Ben Kehoe, Jur Van Den Berg, Matei Ciocarlie, Pieter Abbeel, and Ken Goldberg. “Gp-gpis-opt: Grasp planning with shape uncertainty using gaussian process implicit surfaces and sequential convex programming”. In: IEEE ICRA. 2015, pp. 4919–4926. [87] Sandeep Manjanna, Alberto Quattrini Li, Ryan N. Smith, Ioannis Rekleitis, and Gregory Dudek. “Heterogeneous Multi-Robot System for Exploration and Strategic Water Sampling”. In: ICRA. 2018. [88] Evanthia Mantzouki et al. “Temperature Effects Explain Continental Scale Distribution of Cyanobacterial Toxins”. In: Toxins (2018). [89] Roman Marchant, Fabio Ramos, and Scott Sanner. “Sequential Bayesian optimisation for spatial-temporal monitoring”. In: UAI. 2014. [90] Ryan J Marcotte, Xipeng Wang, Dhanvin Mehta, and Edwin Olson. “Optimizing multi-robot communication under bandwidth constraints”. In: Autonomous Robots 44.1 (2020), pp. 43–55. [91] J. S. Maritz and R. G. Jarrett. “A Note on Estimating the Variance of the Sample Median”. In: Journal of the American Statistical Association 73.361 (Mar. 1978). [92] Seth McCammon and Geoffrey A. Hollinger. “Topological Hotspot Identification for Informative Path Planning with a Marine Robot”. In: ICRA. 2018. [93] Alvine C. Mehinto, Jayme Smith, Ellie Wenger, Beckye Stanton, Regina Linville, Bryan W. Brooks, Martha A. Sutula, and Meredith D. A. Howard. “Synthesis of ecotoxicological studies on cyanotoxins in freshwater habitats – Evaluating the basis for developing thresholds protective of aquatic life in the United States”. In: Science of The Total Environment (2021). [94] Joseph Mirabel, Steve Tonneau, Pierre Fernbach, Anna-Kaarina Seppälä, Mylene Campana, Nicolas Mansard, and Florent Lamiraux. “HPP: A new software for constrained motion planning”. In: Proceedings of the International Conference on Intelligent Robots and Systems. 2016. URL: https://ieeexplore.ieee.org/abstract/document/7759083. [95] Xuan Son Nguyen, Luc Brun, Olivier Lézoray, and Sébastien Bougleux. “A neural network based on SPD manifold learning for skeleton-based hand gesture recognition”. In: IEEE CVPR. 2019. [96] Takayuki Osa. “Learning the Solution Manifold in Optimization and Its Application in Motion Planning”. In: arXiv (2020). eprint: 2007.12397 (cs.RO). URL: https://arxiv.org/pdf/2007.12397.pdf. [97] M.H. Overmars. A random approach to motion planning. Tech. rep. RUU-CS-92-32. Department of Computer Science, Utrecht University, 1992. 122 [98] Paritosh Padhy, Rajdeep K Dash, Kirk Martinez, and Nicholas R Jennings. “A utility-based adaptive sensing and multihop communication protocol for wireless sensor networks”. In: ACM Transactions on Sensor Networks (TOSN) 6.3 (2010), pp. 1–39. [99] Lishuo Pan, Sandeep Manjanna, and M Ani Hsieh. “MARLAS: Multi Agent Reinforcement Learning for cooperated Adaptive Sampling”. In: arXiv preprint arXiv:2207.07751 (2022). [100] Alexandros Paraschos, Christian Daniel, Jan Peters, and Gerhard Neumann. “Probabilistic Movement Primitives”. In: NIPS. 2013. [101] Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. “Deepsdf: Learning continuous signed distance functions for shape representation”. In: IEEE CVPR. 2019. [102] Peter Pastor, Mrinal Kalakrishnan, Sachin Chitta, Evangelos Theodorou, and Stefan Schaal. “Skill learning and task outcome prediction for manipulation”. In: ICRA. 2011. [103] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. “Automatic differentiation in PyTorch”. In: NIPS-W. 2017. [104] Max Pflueger and Gaurav S Sukhatme. “Multi-step planning for robotic manipulation”. In: Proceedings of the International Conference on Robotics and Automation. 2015, pp. 2496–2501. [105] Mike Phillips, Benjamin Cohen, Sachin Chitta, and Maxim Likhachev. “E-Graphs: Bootstrapping Planning with Experience Graphs”. In: Robotics: Science and Systems (2012). [106] Marija Popovi´ c, Teresa Vidal-Calleja, Gregory Hitz, Jen Jen Chung, Inkyu Sa, Roland Siegwart, and Juan Nieto. “An informative path planning framework for UA V-based terrain monitoring”. In: Autonomous Robots 44 (2020). [107] Anne-Sophie Puydupin-Jamin, Miles Johnson, and Timothy Bretl. “A convex approach to inverse optimal control and its application to modeling human locomotion”. In: ICRA. 2012. [108] Chao Qin, Diego Klabjan, and Daniel Russo. “Improving the Expected Improvement Algorithm”. In: NeurIPS. Long Beach, California, USA, 2017. [109] Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning. MIT press, 2006. [110] Nathan Ratliff. “Multivariate calculus II: The geometry of smooth maps”. In: Lecture notes: Mathematics for Intelligent Systems series (2014). URL: https://ipvs.informatik.uni-stuttgart.de/mlr/wp- content/uploads/2014/12/mathematics_for_intelligent_systems_lecture6_notes.pdf. [111] Nathan Ratliff, Marc Toussaint, and Stefan Schaal. “Understanding the geometry of workspace obstacles in motion optimization”. In: Proceedings of the International Conference on Robotics and Automation. 2015. URL: https://ieeexplore.ieee.org/abstract/document/7139778. 123 [112] Nathan D. Ratliff, J. Andrew Bagnell, and Martin Zinkevich. “Maximum margin planning”. In: 23rd ICML. 2006, pp. 729–736. [113] Nathan D. Ratliff, David Silver, and J. Andrew Bagnell. “Learning to search: Functional gradient techniques for imitation learning”. In: Autonomous Robots 27 (2009), pp. 25–53. [114] Isabel M Rayas Fernández, Christopher E Denniston, and Gaurav S Sukhatme. “A Study on Multirobot Quantile Estimation in Natural Environments”. In: arXiv e-prints (2023), arXiv–2303. [115] Isabel M. Rayas Fernández, Christopher E. Denniston, David A. Caron, and Gaurav S. Sukhatme. “Informative Path Planning to Estimate Quantiles for Environmental Analysis”. In: IEEE Robotics and Automation Letters 7.4 (2022), pp. 10280–10287. DOI: 10.1109/LRA.2022.3191936. [116] Isabel M. Rayas Fernández, Christopher E. Denniston, and Gaurav S. Sukhatme. “Reducing Network Load via Message Utility Estimation for Decentralized Multirobot Teams”. In: (2023). [117] Avi Rosenfeld, Gal A Kaminka, and Sarit Kraus. “A study of marginal performance properties in robotic teams”. In: (2004). [118] Avi Rosenfeld, Gal A Kaminka, Sarit Kraus, and Onn Shehory. “A study of mechanisms for improving robotic group performance”. In: Artificial Intelligence 172.6-7 (2008), pp. 633–655. [119] Sam T Roweis and Lawrence K Saul. “Nonlinear dimensionality reduction by locally linear embedding”. In: science 290.5500 (2000), pp. 2323–2326. [120] Guillaume Sartoretti, Yue Wu, William Paivine, TK Kumar, Sven Koenig, and Howie Choset. “Distributed reinforcement learning for multi-robot decentralized collective construction”. In: Distributed autonomous robotic systems. Springer, 2019, pp. 35–49. [121] S Schaal, A Ijspeert, and A Billard. “Computational Approaches to Motor Learning by Imitation”. In: Philosophical Transactions of the Royal Society of London 358 (2003), pp. 537–547. [122] Philipp S Schmitt, Florian Witnshofer, Kai M Wurm, Georg v Wichert, and Wolfram Burgard. “Modeling and planning manipulation in dynamic environments”. In: International Conference on Robotics and Automation. 2019, pp. 176–182. [123] John Schulman, Jonathan Ho, Alex X Lee, Ibrahim Awwal, Henry Bradlow, and Pieter Abbeel. “Finding Locally Optimal, Collision-Free Trajectories with Sequential Convex Optimization”. In: Robotics: Science and Systems. 2013. URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.387.4642&rep=rep1&type=pdf. [124] Bridget Seegers, James Birch, Roman Marin, Chris Scholin, David Caron, Erica Seubert, Meredith Howard, George Robertson, and Burton Jones. “Subsurface seeding of surface harmful algal blooms observed through the integration of autonomous gliders, moored environmental sample processors, and satellite remote sensing in southern California: Harmful algal blooms subsurface seeding”. In: Limnology and Oceanography (2015). 124 [125] Samantha L. Sharp, Alexander L. Forrest, Keith Bouma-Gregson, Yufang Jin, Alicia Cortés, and S. Geoffrey Schladow. “Quantifying Scales of Spatial Variability of Cyanobacteria in a Large, Eutrophic Lake Using Multiplatform Remote Sensing Tools”. In: Frontiers in Environmental Science (2021). DOI: 10.3389/fenvs.2021.612934. [126] David Silver and Joel Veness. “Monte-Carlo Planning in Large POMDPs”. In: NeurIPS. V ol. 23. 2010. [127] Thierry Siméon, J-P Laumond, and Carole Nissoux. “Visibility-based probabilistic roadmaps for motion planning”. In: Advanced Robotics 14.6 (2000), pp. 477–493. URL: https://www.tandfonline.com/doi/abs/10.1163/156855300741960. [128] Thierry Siméon, Jean-Paul Laumond, Juan Cortés, and Anis Sahbani. “Manipulation planning with probabilistic roadmaps”. In: The International Journal of Robotics Research 23.7-8 (2004), pp. 729–746. URL: https://journals.sagepub.com/doi/abs/10.1177/0278364904045471. [129] Andrea Simonetto, Leon Kester, and Geert Leus. “Distributed time-varying stochastic optimization and utility-based communication”. In: arXiv preprint arXiv:1408.5294 (2014). [130] Jasper Snoek, Hugo Larochelle, and Ryan P Adams. “Practical Bayesian Optimization of Machine Learning Algorithms”. In: Neurips. V ol. 25. 2012. [131] J. R. Souza, R. Marchant, L. Ott, D. F. Wolf, and F. Ramos. “Bayesian optimisation for active perception and smooth navigation”. In: ICRA. 2014. [132] Mandavilli Srinivas and Lalit M Patnaik. “Genetic algorithms: A survey”. In: computer 27.6 (1994), pp. 17–26. [133] Cyrill Stachniss, Óscar Martínez Mozos, and Wolfram Burgard. “Efficient exploration of unknown indoor environments using a team of mobile robots”. In: Annals of Mathematics and Artificial Intelligence 52.2 (2008), pp. 205–227. [134] Elias M Stein and Rami Shakarchi. Real analysis: measure theory, integration, and Hilbert spaces. Princeton lectures in analysis. Princeton, NJ: Princeton Univ. Press, 2005. [135] Mike Stilman. “Global manipulation planning in robot joint space with task constraints”. In: IEEE Transactions on Robotics 26.3 (2010), pp. 576–584. URL: https://ieeexplore.ieee.org/abstract/document/5467152. [136] Mike Stilman. “Task constrained motion planning in robot joint space”. In: Proceedings of the International Conference on Intelligent Robots and Systems. 2007. URL: https://ieeexplore.ieee.org/abstract/document/4399305. [137] Oskar von Stryk and Roland Bulirsch. “Direct and Indirect Methods for Trajectory Optimization”. In: Annals of Operations Research 37.1 (1992), pp. 357–373. URL: https://link.springer.com/article/10.1007/BF02071065. 125 [138] Ioan A ¸ Sucan and Sachin Chitta. “Motion planning with constraints using configuration space approximations”. In: Proceedings of the International Conference on Intelligent Robots and Systems. 2012. URL: https://ieeexplore.ieee.org/abstract/document/6386092. [139] Chansu Suh, Terry Taewoong Um, Beobkyoon Kim, Hakjong Noh, Munsang Kim, and Frank C Park. “Tangent space RRT: A randomized planning algorithm on constraint manifolds”. In: Proceedings of the International Conference on Robotics and Automation. 2011. URL: https://ieeexplore.ieee.org/abstract/document/5980566. [140] Zachary N Sunberg and Mykel J Kochenderfer. “Online algorithms for POMDPs with continuous state, action, and observation spaces”. In: Twenty-Eighth International Conference on Automated Planning and Scheduling. 2018. [141] Giovanni Sutanto, Nathan Ratliff, Balakumar Sundaralingam, Yevgen Chebotar, Zhe Su, Ankur Handa, and Dieter Fox. “Learning latent space dynamics for tactile servoing”. In: IEEE ICRA. 2019, pp. 3622–3628. [142] Giovanni Sutanto, Isabel M Rayas Fernández, Peter Englert, Ragesh K Ramachandran, and Gaurav S Sukhatme. “Learning Equality Constraints for Motion Planning on Manifolds”. In: Conference on Robot Learning. 2020. [143] Joshua B Tenenbaum, Vin De Silva, and John C Langford. “A global geometric framework for nonlinear dimensionality reduction”. In: science 290.5500 (2000), pp. 2319–2323. [144] K. Thopalli, R. Anirudh, J. J. Thiagarajan, and P. Turaga. “Multiple Subspace Alignment Improves Domain Adaptation”. In: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2019, pp. 3552–3556. [145] E. Todorov, T. Erez, and Y . Tassa. “MuJoCo: A physics engine for model-based control”. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2012, pp. 5026–5033. [146] Marc Toussaint. “A tutorial on Newton methods for constrained trajectory optimization and relations to SLAM, Gaussian Process smoothing, optimal control, and probabilistic inference”. In: Geometric and Numerical Foundations of Movements. Springer, 2017. URL: https://link.springer.com/chapter/10.1007/978-3-319-51547-2_15. [147] Marc Toussaint. “The Bayesian Search Game”. In: Theory and Principled Methods for the Design of Metaheuristics. Ed. by Yossi Borenstein and Alberto Moraglio. 2014. [148] Marc Toussaint, Kelsey Allen, Kevin A Smith, and Joshua B Tenenbaum. “Differentiable Physics and Stable Modes for Tool-Use and Manipulation Planning”. In: Proceedings of Robotics: Science and Systems. 2018. URL: https://ipvs.informatik.uni-stuttgart.de/mlr/papers/18-toussaint-RSS.pdf. [149] Vaibhav Unhelkar and Julie Shah. “Contact: Deciding to communicate during time-critical collaborative tasks in unknown, deterministic domains”. In: Proceedings of the AAAI Conference on Artificial Intelligence . V ol. 30. 1. 2016. 126 [150] William Vega-Brown and Nicholas Roy. “Asymptotically Optimal Planning under Piecewise-Analytic Constraints”. In: Workshop on the Algorithmic Foundations of Robotics. May 2020, pp. 528–543. ISBN: 978-3-030-43088-7. DOI: 10.1007/978-3-030-43089-4_34. [151] Prasanna Velagapudi, Paul Scerri, Katia Sycara, Huadong Wang, Michael Lewis, and Jijun Wang. “Scaling effects in multi-robot control”. In: 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE. 2008, pp. 2121–2126. [152] Huadong Wang, Michael Lewis, Prasanna Velagapudi, Paul Scerri, and Katia Sycara. “How search and its subtasks scale in N robots”. In: Proceedings of the 4th ACM/IEEE international conference on Human robot interaction. 2009, pp. 141–148. [153] Wei Wang, Yan Huang, Yizhou Wang, and Liang Wang. “Generalized autoencoder: A neural network framework for dimensionality reduction”. In: IEEE CVPR Workshops. 2014, pp. 490–497. [154] Rand Wilcox. Introduction to Robust Estimation and Hypothesis Testing (Third Edition). 3rd Edition. Statistical Modeling and Decision Science. Academic Press, 2012, pp. 103–136. [155] Frank Wilcoxon. Individual comparisons by ranking methods. Springer, 1992. [156] Fusheng Zha, Yizhou Liu, Wei Guo, Pengfei Wang, Mantian Li, Xin Wang, and Jingxuan Li. “Learning the metric of task constraint manifolds for constrained motion planning”. In: Electronics 7.12 (2018), p. 395. [157] Sai Qian Zhang, Qi Zhang, and Jieyu Lin. “Efficient communication in multi-agent reinforcement learning via variance based control”. In: Advances in Neural Information Processing Systems 32 (2019). [158] Yanwu Zhang, Brian Kieft, Brett W. Hobson, John P. Ryan, Benedetto Barone, Christina M. Preston, Brent Roman, Ben-Yair Raanan, Roman Marin III, Thomas C. O’Reilly, Carlos A. Rueda, Douglas Pargett, Kevan M. Yamahara, Steve Poulos, Anna Romano, Gabe Foreman, Hans Ramm, Samuel T. Wilson, Edward F. DeLong, David M. Karl, James M. Birch, James G. Bellingham, and Christopher A. Scholin. “Autonomous Tracking and Sampling of the Deep Chlorophyll Maximum Layer in an Open-Ocean Eddy by a Long-Range Autonomous Underwater Vehicle”. In: IEEE Journal of Oceanic Engineering (2020). DOI: 10.1109/JOE.2019.2920217. [159] Zhenyue Zhang and Hongyuan Zha. “Principal manifolds and nonlinear dimensionality reduction via tangent space alignment”. In: SIAM journal on scientific computing 26.1 (2004), pp. 313–338. [160] Brian D. Ziebart, Andrew Maas, J. Andrew Bagnell, and Anind K. Dey. “Maximum Entropy Inverse Reinforcement Learning”. In: In Proceedings of the AAAI Conference on Artificial Intelligence (2008). 127 Appendices Appendix A: PSM ∗ As a reference, we provide in Chapter 9 a variant of PSM ∗ that grows a single tree over the manifold sequence without splitting it into individual subtrees, which we use as a baseline in the experiments. Af- terwards, we summarize in Chapter 9 the extension step of RRT ∗ [61] that is called in the inner loop of the PSM ∗ algorithm. In Chapter 9 and Chapter 9, we provide a theoretical analysis regarding the probabilistic completeness and asymptotic optimality of the proposed algorithm. PSM ∗ (Single Tree) Algorithm 3 describes a variant of PSM ∗ that grows a single tree over the manifold sequence without split- ting it into individual subtrees. For each node, we store the corresponding current and target manifold as well as the free configuration space, which is extracted by the functions GetCurrentManifold, GetTargetMani- fold, and GetFreeSpace in line 5 – line 7. Given this information, we call the same steering function that PSM ∗ uses and the RRT ∗ extend routine described in Algorithm 4. The main difference between PSM ∗ (Sin- gle Tree) algorithm and PSM ∗ is that PSM ∗ (Single Tree) grows the tree on all manifolds while PSM ∗ grows a subtree for every manifold inM in a sequential manner. 128 Algorithm 3 PSM ∗ (Single Tree)(M,q start ,α,β,ε,ρ,r,m) 1: V ={q start }; E = / 0; n= len(M)− 1 2: for k= 1 to nm do 3: q rand ← Sample(C) 4: q near ← Nearest(V,q rand ) 5: M i ← GetCurrentManifold(q near ) 6: M i+1 ← GetTargetManifold(q near ) 7: C free,i ← GetFreeSpace(q near ) 8: q new ← PSM ∗ _STEER(α,β,r,q near ,M i ,M i+1 ) 9: if||h M i (q new )||>ε then 10: continue 11: RRT ∗ _EXTEND(V i ,E i ,q near ,q new , C free,i ) 12: return OptimalPath(V,E,q start ,M n+1 ) RRT ∗ Extension Step The RRT ∗ _EXTEND routine (Algorithm 4) is a straightforward adaptation of the extension step in RRT ∗ [61] to the PSM ∗ problem. This routine checks the newly projected q new configuration for collision with the current free configuration space C free,i , performs rewiring steps, and eventually adds it as a node to the tree. The rewiring step is equivalent to the one in RRT ∗ [61], which checks if any nearby points can be reached with a shorter distance and updates its parent nodes accordingly. It uses a distance function c(q 0 ,q 1 )∈R ≥ 0 between two nearby configurations and a function Cost(q) that stores the path costs from the root of the tree to a node q in order to reconnect a new node q to its neighbor that results in the shortest path from root q start to q. Note that c(q 0 ,q 1 )≜ J(q 0 q 1 ), where q 0 q 1 denotes the geodesic [10] joining points q 0 and q 1 on the current manifold. Probabilistic Completeness In this section, we prove the probabilistic completeness of Algorithm 3. Note that, for ease of analysis, the analysis presented in this section and in the subsequent section assumes that ρ = 0. Moreover, in this analysis we refer to the collision-free region of a manifold when we allude to M i . 129 Algorithm 4 RRT ∗ _EXTEND(V,E,q near ,q new , C free,i ) 1: if CollisionFree(q near ,q new , C free,i ) then 2: Q near = Near V,q new ,min n γ RRT* log(|V|) |V| 1/k ,α o 3: V← V∪{q new } 4: q min = q near ; c min = Cost(q near )+ c(q near ,q new ) 5: for q near ∈ Q near do 6: if CollisionFree(q near ,q new , C free,i ) and 7: Cost(q near )+ c(q near ,q new )< c min then 8: q min = q near ; c min = Cost(q near )+ c(q near ,q new ) 9: E← E∪{(q min ,q new )} 10: for q near ∈ Q near do 11: if CollisionFree(q new ,q near , C free,i ) and 12: Cost(q new )+ c(q new ,q near )< Cost(q near ) then 13: q parent = Parent(q near ) 14: E← E\{(q parent ,q near )} 15: E← E∪{(q new ,q near )} 16: return True 17: else 18: return False Although in this section, we prove probabilistic completeness and asymptotic analysis for Algorithm 3, they extend to Algorithm 1 as well. We make this claim based on the following arguments. Algorithm 1 is essentially a sequence of n infinite loops in which, at the termination of the i th loop, the algorithm will have computed the optimal path from the start location to the intersection of manifold M i and M i+1 . On the contrary, Algorithm 3 consists of a single infinite loop which upon termination outputs the optimal path from start point through the sequence of manifolds to the goal manifold. It is important to note that each loop in the sequence of infinite loops in Algorithm 1 can be initiated prior to terminating the previous loop. This argument stems from the fact that a search tree in a manifold can be initiated immediately when the nodes in the search tree associated with the previous manifold reach the intersection of the manifold and the previous manifold. Therefore, each infinite loop in Algorithm 1 can be viewed as been executed in parallel and can be assumed to terminate simultaneously. Additionally, it is straightforward to understand that at the end of each infinite loop, Algorithm 1 solves a subproblem of the problem that Algorithm 3 solves. Therefore, by 130 the principle of optimality [80, Chapter 5], both Algorithm 3 and Algorithm 1 solve the same problem and are identical for our analysis. Definition 1. A collision-free path is said to have strong δ-clearance if the path lies entirely inside the δ-interior of∪M , where∪M≜∪ n+1 i=1 M i [61]. We start by assuming that there exists a path ˆ τ with strong δ-clearance connecting the goal manifold M n+1 with the start configuration q start embedded on the sequence of manifolds under consideration. Let L be the total length of the path, computed based on the pullback metric [78] of the manifolds due to their embedding inR k . Let ξ > 0 be the minimum over the reach [1] of all manifolds in the sequence and the manifolds resulting from the pairwise intersection of adjacent manifolds in the sequence. Informally, the reach of a manifold is the size of an envelope around the manifold such that any point within the envelope and the manifold has an unique projection onto the manifold. For the analysis presented here, we pick the steering parameterα such thatξ≥ α. We use the notation Tube(M i ,ξ) to denote the set{x∈R k | d(x,M i )< ξ}, where d(x,M i )= inf{∥x− y∥ R k| y∈ M i } (9.1) is the minimum distance of the point x to the manifold. Now, if we define ζ i = sup q∈Tube(M i ,ξ) ∥h M i (q)∥, (9.2) 131 then for the sake of analysis we assume that r= max{ζ 1 ,··· ,ζ n+1 }. If ν = min(δ,α), then we define a sequence of points {[q 1 0 = q start ,q 1 1 ,··· ,q 1 m 1 ],[q 2 0 ,q 2 1 ,··· ,q 2 m 2 ],··· , [q n 0 ,q n 1 ,··· ,q n m 1 ],[q n+1 0 ]} (9.3) on ˆ τ, such that[q i 0 ,q i 1 ,··· ,q i m i ]∈ M i and∑ n i=1 m i = m, where m= 5L ν(n+1) is the total number of points in the path. Without loss of generality, we assume that for every M i with 1< i< n+ 1 there exists a non-negative integer j< m i such that q i 0 ,q i 1 ,··· ,q i j ∈ M i− 1 ∩ M i and q i j+1 ,q i j+2 ,··· ,q i m i ∈ M i \ M i+1 . In other words, there exist some points at the beginning of[q i 0 ,q i 1 ,··· ,q i m i ] that belong to M i− 1 ∩ M i and the rest of the points on the manifold belong exclusively to M i . For ease of analysis, the sequence of points on ˆ τ is chosen such that ∥q i j − q i j+1 ∥ R k≤∥ q i j − q i j+1 ∥ M i ≤ ν/5 (9.4) where∥·∥ R k and∥·∥ M i are the distances between the points according to the metrics on the ambient space and manifold respectively. We use B(q i j ,ν)⊂ R k to denote a ball of radiusν around q i j under the standard Euclidean norm onR k . We denote the tree that is grown with RRT ∗ as T . We prove the probabilistic completeness of our strategy in two parts. The first part proves the proba- bilistic completeness of RRT ∗ on a single manifold. In the second part, we prove that with probability one, the tree T grown on a manifold can be expanded onto the next manifold as the number of samples tends to infinity. For the first part, as suggested in [68, Section 5.3], the Lemma 1 in [72] can be shown to hold for the single manifold case and probabilistic completeness of RRT/RRT ∗ on a manifold can be easily proven using [72, Theorem 1]. We now focus on proving the second part that shows the probabilistic completeness of our strategy. We start by proving Lemma 1 which enables us to prove that a tree grown with RRT ∗ on a manifold can be extended to the next manifold. 132 Lemma 1. Suppose that T has reached M i and contains a vertex ˜ q i m i such that ˜ q i m i ∈ B(q i m i ,ν/5). If a random sample q i+1 rand is drawn such that q i+1 rand ∈ B(q i+1 0 ,ν/5), then the straight path between Project(q i+1 rand ,M i ∩M i+1 ) and the nearest neighbor q near of q i+1 rand in T lies entirely in C free,i+1 . Proof. By definition ∥q near − q i+1 rand ∥≤∥ ˜ q i m i − q i+1 rand ∥, then using the triangle inequality and some algebraic manipulation similar to that used in the proof of [72, Lemma 1], we can show that ∥q near − q i m i ∥ ≤∥ ˜ q i m i − q i m i ∥+ 2∥x i+1 0 − q i+1 rand ∥ + 2∥q i m i − q i+1 0 ∥ (9.5) which leads to∥q near − q i m i ∥≤ 5 ν 5 ≤ ν and q near ∈ B(q i m i ,ν). Again, by the triangle inequality, we can show that∥q near − q i+1 rand ∥≤ ν. As the sample q i+1 rand is taken from within the reach of M i there exists a unique nearest point of q i+1 rand on M i [1]. In other words, the operation Project(q i+1 rand ,M i ∩ M i+1 ) is well-defined. Therefore, as ∥q near − Project(q i+1 rand ,M i ∩ M i+1 )∥≤∥ q near − q i+1 rand ∥≤ ν, (9.6) the straight path between Project(q i+1 rand ,M i ∩ M i+1 ) and q near lies entirely in C free,i+1 . Note that the above lemma is an extension of [72, Lemma 1]. The next theorem will prove that with probability one PSM ∗ will yield a path as the number of samples goes to infinity. Since we are only con- cerned about the transition of T from one manifold to the next, we focus on the iterations in PSM ∗ after T reaches a neighborhood of q i m i ∈ M i . We refer to such an iteration as a boundary iteration. Theorem 1. The probability that PSM ∗ fails to reach the final manifold M n+1 from an initial configuration after t boundary iterations is bounded from above by aexp(− bt), for some positive real numbers a and b. 133 Proof. Let B(q i m i ,ν/5) contain a vertex of T . Let p be the probability that in a boundary iteration a vertex contained in B(q i+1 0 ,ν/5) is added to T . From Lemma 1, if we obtain a sample q i+1 rand ∈ B(q i+1 0 ,ν/5), then T can reach Project(q i+1 rand ,M i ∩ M i+1 ). The value p can be computed as a product of the probabilities of two events: 1) a sample is drawn from B(q i+1 0 ,ν/5), and 2) T is extended to include Project(q i+1 rand ,M i ∩ M i+1 ). The probability that a sample is drawn from B(q i+1 0 ,ν/5) is given by |B(q i+1 0 ,ν/5)|/|C|, where |B(q i+1 0 ,ν/5)| and|C| are the volumes of B(q i+1 0 ,ν/5) and C respectively. From the proof of Lemma 1, we infer that the line joining q near and q i+1 rand is collision-free. Thus T will be augmented with a new vertex contained in M i ∩ M i+1 if line 9 and 14 in Algorithm 1 are executed. The probability of execution of line 9 and 14 is(1− β) r−∥ h M i+1 (q new )∥ r , which results in the joint probability p= |B(x i+1 0 ,ν/5)| |C| (1− β) r−∥ h M i+1 (q new )∥ r . (9.7) Further, ∥h M i+1 (q new )∥≈ 0 as q new is very close to M i+1 and|B(x i+1 0 ,ν/5)|≪| C|, thus β can be picked such that p< 0.5. For PSM ∗ to reach M i+1 from the initial point, the boundary iteration should successfully extend T for at least n times (there are n intersections between the n+ 1 sequential manifolds). The process can be viewed as t> n Bernoulli trials with success probability p. LetΠ t denote the number of successes in t trials, then P[Π t < n]= n− 1 ∑ i=0 t i p i (1− p) t− i , (9.8) whereP[·] denotes the probability of occurrence of an event. By using the fact that n≪ t, this can be upper bounded as P[Π t < n]≤ n− 1 ∑ i=0 t n− 1 p i (1− p) t− i , (9.9) 134 as p< 0.5, P[Π t < n]≤ t n− 1 n− 1 ∑ i=0 (1− p) t . (9.10) Applying(1− p)≤ exp(− p) yields P[Π t < n]≤ n t n− 1 (exp(− pt)). (9.11) Through further algebraic simplifications, we can show that P[Π t < n]≤ n (n− 1)! t n exp(− pt). (9.12) As the failure probability of PSM ∗ exponentially goes to zero as t→∞, PSM ∗ is probabilistically com- plete. Asymptotic Optimality For ease of reference, we begin by giving some definitions and stating some lemmas initially introduced in [61], which are required for proving the asymptotic optimality of PSM ∗ . Definition 2. A pathτ 1 is said to be homotopic toτ 2 if there exists a continuous function H :[0,1]× [0,1]→ ∪M , called the homotopy [36], such that H(t,0)=τ 1 (t), H(t,1)=τ 2 (t), and H(·,α) is a collision-free path for allα∈[0,1]. Definition 3. A collision-free pathτ :[0,1]→∪M is said to have weakδ-clearance [61] if there exists a pathτ ′ that has strongδ-clearance and there exists a homotopy H :[0,1]× [0,1]→∪M with H(t,0)=τ(t), H(t,1)=τ ′ (t), and for allα∈(0,1] there existsδ α > 0 such that H(t,α) has strongδ α -clearance. 135 Lemma 2. [61, Lemma 50] Let τ ∗ be a path with weak δ-clearance. Let{δ n } n∈N be a sequence of real numbers such that lim n→∞ δ n = 0 and 0≤ δ n ≤ δ for all n∈N. Then, there exists a sequence{τ n } n∈N of paths such that lim n→∞ τ n =τ ∗ andτ n has strongδ n -clearance for all n∈N. The above lemma establishes the relationship between the weak and strongδ-clearance paths. If the configuration space admits a vector space structure, then it can be shown that the space of paths on the configuration space above also admits a vector space structure. Moreover, the space of path becomes a normed space if it is endowed with the bounded variation norm [134] ∥τ∥ BV ≜ Z 1 0 |τ(t)|dt + TV(τ). (9.13) TV(τ) denotes the total variation norm [134] defined as TV(τ)= sup {n∈N,0=t 1 <t 2 <...<t n =1} n ∑ i=1 |τ(t i )− τ(t i− 1 )|. (9.14) Using the norm in the space of paths, the distance between the pathsτ 1 andτ 2 can defined as ∥τ 1 − τ 2 ∥ BV = Z 1 0 |τ 1 (t)− τ 2 (t)|dt + TV(τ 1 − τ 2 ). (9.15) The normed vector space of paths enables us to mathematically formulate the notion of the convergence of a sequence of paths to a path. Formally, given a sequence of paths{τ n },n∈N, the sequence converges to a path ¯ τ, denoted as lim n→∞ τ n = ¯ τ, if lim n→∞ ∥τ n − ¯ τ∥ BV = 0. LetP denote the set of weak δ-clearance paths which satisfies the constraints in Equation 1. Let τ ∗ ∈P be a path with the minimal cost. Due to the continuity of the cost function, any sequence of paths {τ n ∈P},n∈N such that lim n→∞ τ n =τ ∗ also satisfies lim n→∞ J(τ n )= J(τ ∗ ). For brevity, we identify J(τ ∗ ) 136 with J ∗ and J PSM ∗ n denotes the random variable modeling the cost of the minimum-cost solution returned by PSM ∗ after n iterations. The PSM ∗ algorithm is asymptotically optimal if P h lim n→∞ J PSM ∗ n = J ∗ i = 1. (9.16) A weaker condition than Equation (9.16) is P limsup n→∞ J PSM ∗ n = J ∗ = 1. (9.17) Note that from [61, Lemma 25], we infer that the probability that limsup n→∞ J PSM ∗ n = J ∗ is either zero or one. Under the assumption that the set of points traversed by an optimal path has measure zero, [61, Lemma 28] proves that the probability that PSM ∗ returns a tree containing an optimal path in finite number of iterations is zero. Since PSM ∗ is based on RRT ∗ , we focus on how our technique affects the proofs of asymptotic optimal- ity for RRT ∗ . Also, the work in [69] has shown that RRT ∗ is optimal when applied on a single manifold. Furthermore, it is shown in [69] that the steering parameter γ in the single manifold case can be bounded from below by 2 1+ 1 k µ(∪ q∈C free D q) ζ M (1) 1 k , where D q is the set of points inR k which are projected on q and ζ M (1) is the set points inR k which are projected onto a unit open ball contained in the manifold M. Also, µ(·) denotes a measure [134] on the measurable space [134]R k . Intuitively, a measure maps the subset of a measurable space to its volume. In this section, we show that under the assumptionρ = 0 in Algorithm 1, with probability one PSM ∗ eventually returns the optimal path. Let {Q 1 ,Q 2 ,...,Q n } be a set of independent uniformly distributed points drawn from C free and let {I 1 ,I 2 ,...,I n } be their associated labels that outlines the order of the points with support[0,1]. In other words, 137 a point Q i is assumed to be drawn after another point Q ¯ i if I i < I ¯ i . Let{ ˆ Q 1 , ˆ Q 2 ,..., ˆ Q n } be the set of points re- sulting from projecting the point set onto the manifolds as delineated in Algorithm 1. Similar to [61], we con- sider the graph formed by adding an edge( ˆ Q i , ˆ Q ¯ i ), whenever (i) I i < I ¯ i and (ii)∥ ˆ Q i − ˆ Q ¯ i ∥≤ r n =γ( log(|V n |) |V n | ) 1 k , where V n is the vertex set of the graph andγ is the steering parameter used in Algorithm 1. Let this graph be denoted byG n =(V n ,E n ), where V n and E n are the set of vertices and edges ofG n respectively. With slight abuse of notation, J PSM ∗ n ( ˆ Q i ) denotes the cost of the best path starting from q start to the vertex ˆ Q i in the graph G . Consider the tree ¯ G n which is a subgraph ofG n where the cost of reaching the vertex ˆ Q i equals J PSM ∗ n ( ˆ Q i ). Since PSM ∗ uses RRT ∗ for graph construction, it is easy to see that the tree returned by PSM ∗ at the n-th iteration is equivalent to ¯ G n . Therefore, if limsup n→∞ J PSM ∗ n (M n+1 ) converges to J ∗ with probability one with respect toG n , then it implies that with probability one PSM ∗ will eventually return a tree that contains the optimal path connecting q start and the goal manifold M n+1 . Hence, our next step is focused on showing that the optimal path inG n converges toτ ∗ . According to Lemma 2, there exists a sequence of strongδ− clearance paths{τ m } m∈N that converges to an optimal pathτ ∗ . Let B m ≜{B m,1 ,B m,2 ,...,B m,p } be a set of open balls of radius r n whose centers lie on the pathτ m such that adjacent balls are placed 2r n distance apart. The number of balls p is assumed to be large enough to cover τ m , i.e. τ m \ ∪ p i=1 B m,i ∩τ m is a set of measure zero. Moreover, we denote ˜ B m,i as the region obtained as the intersection of the open ball B m,i with the manifold containing its center. LetΘ m,i denote the event that there exists vertices ˆ Q i and ˆ Q ¯ i such that ˆ Q i ∈ ˜ B m,i , ˆ Q ¯ i ∈ ˜ B m,i+1 and I i ≥ I ¯ i . Recall that, I i and I ¯ i are the labels associated with projected points ˆ Q i and ˆ Q ¯ i respectively. Also note that the edge( ˆ Q i , ˆ Q ¯ i ) is included inG n . D q denotes the set of points that can be projected on the point q∈∪M . Additionally, ζ(1) is defined as ζ(1)≜ max M∈M min q∈M µ ∪ q∈ ˜ B M (q ′ ,1) D q , (9.18) 138 where ˜ B M i (q ′ ,1) is formed by the intersection of a open unit ball centered at point q ′ ∈ M with M. IfΘ m = ∩ p i=1 Θ m,i , then the following lemma proves that with probability one, the eventΘ m,i for all i∈{1,2,..., p} occurs for large m. Lemma 3. If γ > 2 1+ 1 k µ(∪ q∈∪M D q ) ζ(1) 1 k , (9.19) thenΘ m occurs for all large m, with probability one, i.e.,P(liminf n→∞ Θ m )= 1. The proof of the above lemma follows from the proof of [61, Lemma 71] if we replace µ(C free ) with µ(∪ q∈∪M D q ) and infer that the probability of finding a vertex of the graph in ˜ B m,i is ζ(1) µ(∪ q∈∪M D q ) . IfL n denotes the set of all paths that satisfy the constraints in Equation (3.1) and are contained in the tree returned by PSM ∗ after n iterations such that ¯ τ PSM ∗ n ≜ min τ PSM ∗ ∈L n ∥τ PSM ∗ − τ m ∥ BV , then the following lemma can be proven. Lemma 4. [61, Lemma 72] The random variable∥ ¯ τ PSM ∗ n − τ m ∥ BV converges to zero with probability one: P h lim n→∞ ∥ ¯ τ PSM ∗ n − τ m ∥ BV = 0 i = 1. (9.20) Recall that by construction lim m→∞ τ m =τ ∗ . Expressing Equation (9.20) as P h lim n→∞ ∥ ¯ τ PSM ∗ n − τ ∗ − (τ m − τ ∗ )∥ BV = 0 i = 1 (9.21) and applying the triangle inequality yields ∥ ¯ τ PSM ∗ n − τ ∗ − (τ m − τ ∗ )∥ BV ≥∥ ¯ τ PSM ∗ n − τ ∗ ∥ BV −∥ τ m − τ ∗ ∥ BV . 139 From Equation (9.20) and sinceP h lim m→∞ ∥τ m − τ ∗ ∥ BV = 0 i = 1 yields the following result: P h lim n→∞ ∥ ¯ τ PSM ∗ n − τ ∗ ∥ BV = 0 i = 1. (9.22) From the continuity of the cost function and due to the fact that J PSM ∗ i+1 ≤ J PSM ∗ i , i∈N and J PSM ∗ i ≥ J ∗ we obtain the required result (Equation (9.16)). 140 Appendix B: ECoMaNN Orthogonal Subspace Alignment In previous work [144, 39], subspace alignment techniques – without orthogonality constraints – have been introduced to improve domain adaptation. For our purposes here, we require a subspace alignment algorithm that preserves orthogonality of the subspaces being aligned, which we present in this section. Given a set of orthonormal vectorsF ={b 1 ,b 2 ,...,b n } which spans a space, the matrix B= b 1 b 2 ... b n ∈ R n× n belongs to the Orthogonal Group O(n). The Orthogonal Group has two connected components, where one connected component called the Special Orthogonal Group SO(n) is characterized by determinant 1, and the other is characterized by determinant− 1. However, if B= b 1 b 2 ... b n has determinant 1 (i.e. if B∈ SO(n)), then substituting b 1 with its additive inverse (− b 1 ) will result in ¯ B= − b 1 b 2 ... b n with determinant− 1. Aligning two coordinate framesF a andF c to have a common origin and associated basis matrices B a and B c , respectively, is equivalent to finding an R∈ SO(n) such that B a R= B c . The solution to this problem exists if and only if B a and B c come from the same connected component of O(n), i.e. if either both B a ,B c ∈ SO(n) or both determinants of B a and B c are− 1. For a subspace such as the normal space N q M associated with an on-manifold data point q on M spanned by the eigenvectorsF N ={v n− l+1 ,...,v n }, the concept of a determinant does not apply to V N = v n− l+1 ... v n ∈R n× l , as it is not a square matrix. However, the normal space N q M can be described with infinitely-many orthonormal bases V N0 , V N1 , V N2 , ... V N∞ where the set of column vectors of each is an orthonormal basis of N q M. Each of these is a member ofR n× l . Moreover, we can pick the transpose of one of them, for example V N ⊤ 0 , as a projection matrix, and V N0 as the inverse projection matrix. Applying the projection operation to each of the orthonormal bases, we get W N0 = V N ⊤ 0 V N0 = I l× l , W N1 = V N ⊤ 0 V N1 , W N2 = V N ⊤ 0 V N2 , ... W N∞ = V N ⊤ 0 V N∞ , and we will show that W N0 , W N1 , W N2 , ..., W N∞ are mem- bers of O(l), which also has two connected components like O(n). To show this, first note that although V N ⊤ 0 V N0 = I l× l , the matrix V N0 V N ⊤ 0 ̸= I n× n . Hence, for any matrix A∈R n× n in general, V N0 V N ⊤ 0 A̸= A. However, we will show that V N0 V N ⊤ 0 v= v for any vector v in the vector space N q M. Suppose V N0 = 141 b 1 b 2 ... b l ∈R n× l , then we can write V N0 V N ⊤ 0 =∑ l i=1 b i b ⊤ i . Since the collection{b 1 ,b 2 ,...,b l } spans the vector space N q M, any vector v in this vector space can be expressed as v=∑ l i=1 α i b i . Moreover, b ⊤ i v= b ⊤ i ∑ l j=1 α j b j =α i for any i= 1,2,...,l, because by definition of orthonormality b ⊤ i b j = 1 for i= j and b ⊤ i b j = 0 for i̸= j. Hence, V N0 V N ⊤ 0 v=(∑ l i=1 b i b ⊤ i )v=∑ l i=1 (b ⊤ i v)b i =∑ l i=1 α i b i = v. Similarly, be- cause the column vectors of V N0 , V N1 , V N2 , ..., V N∞ are all inside the vector space N q M, it follows that V N0 V N ⊤ 0 V N0 = V N0 , V N0 V N ⊤ 0 V N1 = V N1 , V N0 V N ⊤ 0 V N2 = V N2 , ..., V N0 V N ⊤ 0 V N∞ = V N∞ . Similarly, it can be shown that V Ni V N ⊤ i v= v for any vector v in the vector space N q M for any i= 0,1,2,...,∞. Furthermore, W N ⊤ 0 W N0 = V N ⊤ 0 (V N0 V N ⊤ 0 V N0 )= V N ⊤ 0 V N0 = I l× l , W N ⊤ 1 W N1 = V N ⊤ 1 (V N0 V N ⊤ 0 V N1 )= V N ⊤ 1 V N1 = I l× l , ... W N ⊤ ∞ W N∞ = V N ⊤ ∞ (V N0 V N ⊤ 0 V N∞ )= V N ⊤ ∞ V N∞ = I l× l , and W N0 W N ⊤ 0 = V N ⊤ 0 (V N0 V N ⊤ 0 V N0 )= V N ⊤ 0 V N0 = I l× l , W N1 W N ⊤ 1 = V N ⊤ 0 (V N1 V N ⊤ 1 V N0 )= V N ⊤ 0 V N0 = I l× l , ... W N∞ W N ⊤ ∞ = V N ⊤ 0 (V N∞ V N ⊤ ∞ V N0 )= V N ⊤ 0 V N0 = I l× l . All these show that W N0 , W N1 , W N2 , ..., W N∞ ∈ O(l). Moreover, using V N0 as the inverse projec- tion matrix, we get V N0 = V N0 W N0 , V N1 = V N0 W N1 , V N2 = V N0 W N2 , ... V N∞ = V N0 W N∞ . Therefore, there is a one-to-one mapping between V N0 , V N1 , V N2 , ..., V N∞ and W N0 , W N1 , W N2 , ..., W N∞ . Further- more, between any two of V N0 , V N1 , V N2 , ..., V N∞ , e.g. V Ni and V N j , there exists R N ∈ SO(l) such that V Ni R N = V N j if their SO(l) projections W Ni and W N j both are members of the same connected component of O(l). Now, suppose for nearby on-manifold data points q a and q c , their approximate normal spaces N q a M and N q c M are spanned by eigenvector basesF a N ={v a n− l+1 ,...,v a n } andF c N ={v c n− l+1 ,...,v c n }, respectively. Due to the curvature on the manifold M, the normal spaces N q a M and N q c M may intersect, but in general are different subspaces ofR n× n . For the purpose of aligning the basis of N q a M to the basis of N q c M, one may think to do projection of the basis vectors of N q a M into N q c M. Problematically, this projection may result in a non-orthogonal basis of N q c M. Hence, we resort to an iterative method using a differentiable Special Orthogonal Group SO(l). In particular, we form an l× l skew-symmetric matrix L∈ so(l) with l(l− 1)/2 differentiable parameters, where so(l) is the Lie algebra of SO(l), i.e, the set of all skew-symmetric 142 l× l matrices, and transform it through a differentiable exponential mapping (or matrix exponential) to get R N = exp(L) with exp : so(l)→ SO(l). With V a N = v a n− l+1 ... v a n and V c N = v c n− l+1 ... v c n , we can do an iterative training process to minimize the alignment error between V a N R N and V c N , that is L osa = I l× l − (V a N R N ) ⊤ V c N 2 2 . Depending on whether both W a N and W c N (which are the projections of V a N and V c N , respectively, to O(l)) are members of the same connected component of O(l) or not, this alignment process may succeed or fail. However, if we define ¯ V a N = − v a n− l+1 v a n− l+2 ... v a n and ¯ V c N = − v c n− l+1 v c n− l+2 ... v c n , two out of the four pairs(V a N ,V c N ),( ¯ V a N ,V c N ),(V a N , ¯ V c N ), and( ¯ V a N , ¯ V c N ) will be pairs in the same connected component. Thus, two of these pairs will achieve minimum alignment errors after training the differentiable Special Orthogonal Groups SO(l) on these pairs, indicating successful alignment. These are the main insights for our local alignment of neighboring normal spaces of on-manifold data points. For the global alignment of the normal spaces, we represent the on-manifold data points as a graph. Our Orthogonal Subspace Alignment (OSA) is outlined in Algorithm 5. We begin by constructing a sparse graph of nearest neighbor connections of each on-manifold data point, followed by the construction of this graph into an (un-directed) minimum spanning tree (MST), and eventually the conversion of the MST to a directed acyclic graph (DAG). This graph construction is detailed in line 2 – line 8 of Algorithm 5. Each directed edge in the DAG represents a pair of on-manifold data points whose normal spaces are to be aligned locally. Our insights for the local alignment of neighboring normal spaces are implemented in line 9 – line 21 of Algorithm 5. In the actual implementation, these local alignment computations are done as a vectorized computation which is faster than doing it in a for-loop as presented in Algorithm 5; this for- loop presentation is made only for the sake of clarity. We initialize the l(l− 1)/2 differentiable parameters of the l× l skew-symmetric matrix L with near zero random numbers, which essentially will map to a near identity matrix I l× l of R N via the exp() mapping, as stated in line 14 of Algorithm 5 1 . This is reasonable 1 Although most of our ECoMaNN implementation is done in PyTorch [103], the OSA algorithm is implemented in TensorFlow [2], because at the time of implementation of the OSA algorithm, PyTorch did not support the differentiable matrix exponential (i.e. the exponential mapping) computation yet while TensorFlow did. 143 because we assume that most of the neighboring normal spaces are already/close to being aligned initially. We optimize the alignment of the four pairs(V a N ,V c N ),( ¯ V a N ,V c N ),(V a N , ¯ V c N ), and( ¯ V a N , ¯ V c N ) in line 16 – line 19 of Algorithm 5. Once the local alignments are done, the algorithm then traverses the DAG in breadth-first order, starting from the root r, where the orientation of the root is already chosen and committed to. During the breadth- first traversal of the DAG, three things are done: First, the orientation of each point is chosen based on the minimum alignment loss; second, the local alignment transforms are compounded/integrated along the path from root to the point; and finally, the (globally) aligned orthogonal basis of each point is computed and returned as the result of the algorithm. These steps are represented by line 22 – line 48 of Algorithm 5. 144 Algorithm 5 Orthogonal Subspace Alignment (OSA) 1: function OSA({(q∈C M ,orthogonal basis stacked as matrix V N associated with N q M)}) 2: # construct a sparse graph between each data point q∈C M with its H nearest neighbors, 3: # followed by minimum spanning tree and directed acyclic graph computations 4: # to obtain directed edgesE ; H needs to be chosen to be a value as small as possible that 5: # still results in all non-root points{q∈C M \{q r }} being reachable from the root point q r : 6: G ← computeNearestNeighborsSparseGraph({q∈C M },H) 7: T ← computeMinimumSpanningTree(G) 8: E ← computeDirectedAcyclicGraphEdgesByBreadthFirstTree(T) 9: for each directed edge e=(q c ,q a )∈E do 10: Obtain V a N = v a n− l+1 ... v a n ∈R n× l associated with the source subspace N q a M 11: Obtain V c N = v c n− l+1 ... v c n ∈R n× l associated with the target subspace N q c M 12: Define ¯ V a N = − v a n− l+1 v a n− l+2 ... v a n ∈R n× l 13: Define ¯ V c N = − v c n− l+1 v c n− l+2 ... v c n ∈R n× l 14: Define differentiable SO(l) R − → a − → c N , R − → a ← − c N , R ← − a − → c N , and R ← − a ← − c N , initialized near identity 15: # try optimizing the alignment of the 4 possible pairs: 16: (R − → a − → c N ,L− → a − → c )← iterativelyMinimizeAlignmentError(V a N R − → a − → c N ,V c N ) 17: (R − → a ← − c N ,L− → a ← − c )← iterativelyMinimizeAlignmentError(V a N R − → a ← − c N , ¯ V c N ) 18: (R ← − a − → c N ,L← − a − → c )← iterativelyMinimizeAlignmentError( ¯ V a N R ← − a − → c N ,V c N ) 19: (R ← − a ← − c N ,L← − a ← − c )← iterativelyMinimizeAlignmentError( ¯ V a N R ← − a ← − c N , ¯ V c N ) 20: # record optimized local alignment rotation matrices and its associated loss w/ the edge: 21: Associate(R − → a − → c N ,L− → a − → c ),(R − → a ← − c N ,L− → a ← − c ),(R ← − a − → c N ,L← − a − → c ),(R ← − a ← − c N ,L← − a ← − c ) with e 22: # commit on the orientation of the root point as un-flipped ( − → r ) instead of flipped ( ← − r ): 23: ori(r)= − → r 24: # define the compound/global alignment rotation matrix of the root as an identity matrix: 25: R r G = I l× l 26: # aligned orthogonal basis of the root is: 27: V N aligned,r = V r N 28: # do breadth-first traversal from root to: 29: # (1) select the orientation ori() of each point based on the minimum alignment loss, 30: # (2) compound/integrate the local alignment transforms R G along the path to the point, 31: # (3) and finally compute the aligned orthogonal basis V aligned N : 32: Q= Queue() 33: Q.enqueue(childrenOfNodeInGraph(r,E)) 34: while size(Q)> 0 do 35: d= Q.dequeue() 36: Q.enqueue(childrenOfNodeInGraph(d,E)) 37: p= parentOfNodeInGraph(d,E) 38: # select the local alignment rotation matrix based on the minimum alignment loss 39: # among the two possibilities: 40: ifL− → d ori(p) <L← − d ori(p) then 41: ori(d)= − → d 42: R d G = R − → d ori(p) N R p G 43: V N aligned,d = V d N R d G 44: else 45: ori(d)= ← − d 46: R d G = R ← − d ori(p) N R p G 47: V N aligned,d = ¯ V d N R d G 48: return{V aligned N associated with N q M for each q∈C M } 145
Abstract (if available)
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Efficiently learning human preferences for proactive robot assistance in assembly tasks
PDF
Planning and learning for long-horizon collaborative manipulation tasks
PDF
Active sensing in robotic deployments
PDF
Leveraging prior experience for scalable transfer in robot learning
PDF
Robust loop closures for multi-robot SLAM in unstructured environments
PDF
Algorithms and systems for continual robot learning
PDF
Scaling robot learning with skills
PDF
High-throughput methods for simulation and deep reinforcement learning
PDF
Characterizing and improving robot learning: a control-theoretic perspective
PDF
Program-guided framework for your interpreting and acquiring complex skills with learning robots
PDF
Remote exploration with robotic networks: queue-aware autonomy and collaborative localization
PDF
Decentralized real-time trajectory planning for multi-robot navigation in cluttered environments
PDF
Leveraging cross-task transfer in sequential decision problems
PDF
Sample-efficient and robust neurosymbolic learning from demonstrations
PDF
Intelligent robotic manipulation of cluttered environments
PDF
Motion coordination for large multi-robot teams in obstacle-rich environments
PDF
Decision support systems for adaptive experimental design of autonomous, off-road ground vehicles
PDF
Coordinating social communication in human-robot task collaborations
PDF
Learning objective functions for autonomous motion generation
PDF
Data-driven robotic sampling for marine ecosystem monitoring
Asset Metadata
Creator
Rayas Fernández, Isabel M.
(author)
Core Title
Advancing robot autonomy for long-horizon tasks
School
Viterbi School of Engineering
Degree
Doctor of Philosophy
Degree Program
Computer Science
Degree Conferral Date
2023-08
Publication Date
07/18/2023
Defense Date
05/08/2023
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
environmental robotics,informative path planning,manifold learning,motion planning,multirobot communication,multirobot systems,OAI-PMH Harvest,quantile estimation
Format
theses
(aat)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Sukhatme, Gaurav (
committee chair
), Caron, David (
committee member
), Nikolaidis, Stefanos (
committee member
)
Creator Email
isabel.rayas@gmail.com,rayas@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-oUC113281397
Unique identifier
UC113281397
Identifier
etd-RayasFernn-12090.pdf (filename)
Legacy Identifier
etd-RayasFernn-12090
Document Type
Dissertation
Format
theses (aat)
Rights
Rayas Fernández, Isabel M.
Internet Media Type
application/pdf
Type
texts
Source
20230719-usctheses-batch-1070
(batch),
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright.
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Repository Email
cisadmin@lib.usc.edu
Tags
environmental robotics
informative path planning
manifold learning
motion planning
multirobot communication
multirobot systems
quantile estimation