A lane-based approach to large-scale unmanned aircraft systems traffic management

Unmanned Aircraft Systems (UAS) promise to be a disruptive technology, providing businesses and the general public with advanced robotic tools at low costs. However, the anticipated scale and density of operations make complete human control impractical, and it is necessary to develop the cyber-physical science, technology and engineering methods to design, develop and safely control these tightly coupled cyber-physical systems. The research presented here is motivated by a specific problem in UAS Traffic Management (UTM) known as Strategic Deconfliction, a conflict management strategy that is achieved through airspace organization and management, demand and capacity balancing, and traffic synchronization. This research introduces a lane-based approach to strategic conflict management. Methods for constructing aerial corridors and scheduling flights within them are given, and issues that arise in UTM are explored under this framework. It is shown that the computational complexity of strategic deconfliction under the lane-based approach is reduced for individual operators, and additionally enables operators to assess the state of the airspace more effectively for contingency handling. To assess the performance characteristics of lane-based systems, and to compare this strategy to what is currently proposed by the FAA and NASA, a set of measures including flow, density, and other network properties that have been traditionally used for ground-based traffic are applied to UTM. The trajectory of this research was informed by an effort to commercialize it, and it is currently being funded by two grants from the Utah Department of Transportation, as well as a Small Business Technology Transfer (STTR) program in collaboration with the United States Air Force. Additionally, the author of this dissertation participated in NASA's Advanced Air Mobility (AAM) National Campaign Developmental Test (NC-DT) X3 simulations. The X3 simulations were the first time that multiple industry partners were gathered by NASA to test and collaborate on software implementations for AAM, with particular emphasis on strategic conflict management.

A LANE-BASED APPROACH TO LARGE-SCALE UNMANNED AIRCRAFT SYSTEMS TRAFFIC MANAGEMENT by David Sacharny A dissertation submitted to the faculty of The University of Utah in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computing School of Computing The University of Utah May 2022 Copyright c David Sacharny 2022 All Rights Reserved The University of Utah Graduate School STATEMENT OF DISSERTATION APPROVAL The dissertation of David Sacharny has been approved by the following supervisory committee members: Thomas C. Henderson , Chair(s) 4/5/2022 Date Approved Xiaoyue Cathy Liu , Member 4/5/2022 Date Approved Cem Yuksel , Member 4/8/2022 Date Approved Vivek Srikumar , Member 4/4/2022 Date Approved Kam K. Leang , Member 4/8/2022 Date Approved by Mary W. Hall , Chair/Dean of the Department/College/School of Computing and by David B. Kieda , Dean of The Graduate School. ABSTRACT Unmanned Aircraft Systems (UAS) promise to be a disruptive technology, providing businesses and the general public with advanced robotic tools at low costs. However, the anticipated scale and density of operations make complete human control impractical, and it is necessary to develop the cyber-physical science, technology and engineering methods to design, develop and safely control these tightly coupled cyber-physical systems. The research presented here is motivated by a specific problem in UAS Traffic Management (UTM) known as Strategic Deconfliction, a conflict management strategy that is achieved through airspace organization and management, demand and capacity balancing, and traffic synchronization. This research introduces a lane-based approach to strategic conflict management. Methods for constructing aerial corridors and scheduling flights within them are given, and issues that arise in UTM are explored under this framework. It is shown that the computational complexity of strategic deconfliction under the lane-based approach is reduced for individual operators, and additionally enables operators to assess the state of the airspace more effectively for contingency handling. To assess the performance characteristics of lane-based systems, and to compare this strategy to what is currently proposed by the FAA and NASA, a set of measures including flow, density, and other network properties that have been traditionally used for ground-based traffic are applied to UTM. The trajectory of this research was informed by an effort to commercialize it, and it is currently being funded by two grants from the Utah Department of Transportation, as well as a Small Business Technology Transfer (STTR) program in collaboration with the United States Air Force. Additionally, the author of this dissertation participated in NASA’s Advanced Air Mobility (AAM) National Campaign Developmental Test (NC-DT) X3 simulations. The X3 simulations were the first time that multiple industry partners were gathered by NASA to test and collaborate on software implementations for AAM, with particular emphasis on strategic conflict management. Words could not express my appreciation and love for Ashley, whose unwavering support gave me the confidence to keep learning and building. This dissertation, and the adventure that preceded it, is in no small part a result of the drive and mentorship of my advisor, Tom Henderson. Your productivity and relentless push to think and write will perpetually inspire me. CONTENTS ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii ACKNOWLEDGEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiv CHAPTERS 1. 2. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 1.2 1.3 1.4 2 3 6 7 Research Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Visuals and Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Technology Opportunity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . INTRODUCTION TO UAS TRAFFIC MANAGEMENT . . . . . . . . . . . . . . . . . . . 10 2.1 NASA/FAA UTM Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 UTM Scheduling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 The Air Traffic Flow Management Problem . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 The Job-Shop Scheduling Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.3 The Multi-Robot Motion Planning Problem . . . . . . . . . . . . . . . . . . . . . . . 2.2.4 The Traffic Assignment Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.5 The Optimization Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3. LANE NETWORKS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.1 Lane-Base Urban Airways . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 Multi-Altitude Airways . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.2 Single-Altitude Airways . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Spatial Network Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Static Spatial Network Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Dynamic Spatial Network Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4. 11 13 13 14 15 16 16 19 20 22 24 25 28 STRATEGIC DECONFLICTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.1 FAA-NASA Strategic Deconfliction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.1 A Detailed Analysis of Strategic Deconfliction . . . . . . . . . . . . . . . . . . . . . 4.2 Lane-Based Strategic Deconfliction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1 The Label Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 The Backprojection Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 Backprojection Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.4 FAA-NASA versus Lane-Based SD Comparison . . . . . . . . . . . . . . . . . . . . 4.2.5 Lane Stream Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 42 45 46 49 51 52 54 5. AIR TRAFFIC OPERATIONS CENTER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 5.1 Example ATOC Requirements by Utah Department of Transportation . . . . . . 5.2 Lane-Based Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 NAB Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1.1 The Lane-Based Trajectory Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1.2 Sensor Data to Track Rogue Flights . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1.2.1 Radar placement. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.2 NAB Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Next Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6. 67 69 69 69 72 73 76 79 UAS BELIEF-DESIRE-INTENTION AGENT ARCHITECTURE . . . . . . . . . . . . . 96 6.1 Knowledge Representation and Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 6.1.1 Probabilistic Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 6.2 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 6.2.1 Complexity and Cognitive Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 6.2.2 Complexity and Airspace Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.2.3 Learning and BDI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.2.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.2.4.1 Learning at a Low Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 6.2.4.1.1 State representation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.2.4.1.2 Possible actions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.2.4.1.3 State transition function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.2.4.1.4 Reward function. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 6.2.4.1.5 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 6.2.4.2 Learning at a Cognitive Level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 6.2.4.3 Environment Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.2.4.4 Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 6.2.4.5 Transition Probabilities and Rewards . . . . . . . . . . . . . . . . . . . . . . . . 108 6.2.5 Policy Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.2.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 6.2.7 Conclusion and Future Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 7. CONTINGENCY HANDLING . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 7.1 7.2 7.3 7.4 7.5 Lane-Based UTM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126 Contingencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127 Pre-Planned Contingency Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128 Dynamic Contingency Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 Real-Time Tactical Deconfliction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 7.5.1 Approximate Global Deconfliction Using CPAD . . . . . . . . . . . . . . . . . . . 132 7.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 7.7 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 8. AGENT BASED MODELING AND SIMULATION . . . . . . . . . . . . . . . . . . . . . . . 137 8.1 Lane Systems and Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137 8.2 Lane Systems and Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138 8.3 ABMS Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 8.3.1 Contingency Handling as a Measure of Effectiveness . . . . . . . . . . . . . . . 140 vi 8.3.2 UAS Behaviors for Contingency Handling . . . . . . . . . . . . . . . . . . . . . . . . 141 8.3.2.1 The Role of Cognition in Contingency Handling . . . . . . . . . . . . . . . 141 8.3.3 UTM Policies for Contingency Handling . . . . . . . . . . . . . . . . . . . . . . . . . . 142 8.4 ABMS Test: FAA versus Lane-Based Approach . . . . . . . . . . . . . . . . . . . . . . . . . 143 8.4.1 Experiments to Determine Parameter Impact on Scheduling Algorithms144 9. STRATEGIC DEPLOYMENT OF DRONE CENTERS AND FLEET SIZE PLANNING FOR DRONE DELIVER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154 9.1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155 9.1.2 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 9.1.3 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 9.2 Research Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 9.2.1 Data Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 9.2.2 Optimization Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158 9.3 Web-Based Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159 9.3.1 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 9.3.1.1 Simulation Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 9.3.2 Simulation Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 9.3.2.1 Load Input Location Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 9.3.2.2 Initialize UAS Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 9.3.2.3 Filter TAZ Zones Based on Requested City Areas . . . . . . . . . . . . . . 161 9.3.2.4 Create Demand (Parcel Requests for a Day) . . . . . . . . . . . . . . . . . . . 161 9.3.2.5 Source Parcel Requests to Nearest Vertiport . . . . . . . . . . . . . . . . . . . 161 9.3.2.6 Calculate Truck/UAS Job Mix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 9.3.2.7 Generate UAS Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 9.3.2.8 Generate Animation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 9.3.2.9 Generate Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 9.4 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 vii LIST OF FIGURES 1.1 Map of selected features in Utah for urban airspace modeling. . . . . . . . . . . . . . 9 2.1 NASA/FAA proposed UTM architecture (from [1]). . . . . . . . . . . . . . . . . . . . . . . 18 2.2 S2 cells covering a portion of New York. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.1 A simple 2x2 grid network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.2 Three types of road layouts over the same locale: actual San Francisco roads (left); grid layout (middle); Delaunay triangulation (right). . . . . . . . . . . . . . . . . 30 3.3 An example two-level grid lane layout of San Francisco roads. . . . . . . . . . . . . . 31 3.4 UTM airways over the San Francisco downtown area. . . . . . . . . . . . . . . . . . . . . 31 3.5 Geometry for determination of minimum required radius. A is the point of intersection of the circle and the line between the vertex and one neighbor; B is the point of intersection of the circle and the line between the vertex and the other neighbor; r is the circle radius; θ is the angle between the neighbors. . 32 3.6 Single altitude lane connections for a vertex with one neighbor. . . . . . . . . . . . . . 32 3.7 Single altitude lane connections for a vertex with more than one neighbor. . . . 33 3.8 Rings must be connected so as to satisfy the lane constraints. . . . . . . . . . . . . . . . 33 3.9 A single altitude airway developed from a set of grid points. . . . . . . . . . . . . . . . 34 3.10 A single altitude airway developed from a set of random points. . . . . . . . . . . . . 34 3.11 Example spatial measures; (a) The Betweenness Centrality of the grid network; (b) Delaunay network (right). Note that the nodes in the center of the grid are likely to be pinch points. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.12 The Betweenness Centrality of an airway created from the ground road network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.13 The Betweenness Centrality of an airway created from a grid. . . . . . . . . . . . . . . 36 3.14 The Betweenness Centrality of an airway created from a Delaunay triangulation of a set of random points above the same area. . . . . . . . . . . . . . . . . . . . . . . . 36 3.15 The time occupancy measure for the supply delivery scenario. . . . . . . . . . . . . . 37 4.1 Sketch of current FAA-NASA proposed strategic deconfliction. . . . . . . . . . . . . . 57 4.2 The basic experimental layout with a 4x4 grid. . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.3 Space-Time Lane Diagram (STLD) for two UASs in a lane. The abscissa is time and the ordinate is distance along the lane. ht is the time headway (distance between UASs in time in lane), and h x is the space headway (distance between UASs in lane). Note that ht and h x are linearly related due to the constant speed. The two trajectories in this scenario intersect at t = 4 and x = 2, however, they violate space-headway before then. . . . . . . . . . . . . . . . . . 58 4.4 Space-Time Lane Diagrams: (a) trajectory boundaries for requested launch time interval [q1 , q2 ]. (b) The headway boundary trajectories for a scheduled flight which enters the lane at time p and exits at time p0 . . . . . . . . . . . . . . . . . . . 59 4.5 Space-Time Lane Diagram Labels. 1,2,3,4,5 indicate intervals and times at the entry to the lane, and A,B,C,D,E indicate times at the lane exit. . . . . . . . . . . . . . 59 4.6 Space-Time Lane Diagrams for the first 13 possible label combinations. . . . . . . 60 4.7 Space-Time Lane Diagrams for possible label combinations 57 through 71. . . . 60 4.8 Required distance along lane to avoid violating headway constraint. . . . . . . . . 61 4.9 The lane-based (blue) and FAA-NASA (red) routes from ground vertex 1 to 36. 61 4.10 The stacked set of space time lane diagrams for flight 10 in Scenario 1. . . . . . . . 62 4.11 The stacked set of space time lane diagrams for flight 10 in Scenario 2. . . . . . . . 62 4.12 The stacked set of space time lane diagrams for flight 10 in Scenario 3. . . . . . . . 63 5.1 Lane-based airway lanes above downtown San Francisco, CA. . . . . . . . . . . . . . 80 5.2 ATOC business plan (from [2]). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 5.3 Radar and other sensing capabilities proposed by UDOT. . . . . . . . . . . . . . . . . . . 81 5.4 GPS coverage and example hardware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.5 ATMS and fiber network for Salt Lake Valley airway corridor. . . . . . . . . . . . . . . 82 5.6 Proposed airspace layout for the Salt Lake Valley airway corridor. . . . . . . . . . . 82 5.7 Set of airway lanes created over Salt Lake City, UT. . . . . . . . . . . . . . . . . . . . . . . . 83 5.8 Trajectory point set model of airway lanes over East Bench of Salt Lake City, UT. Red circles are lane endpoints; blue points are samples along lane. . . . . . . . 83 5.9 Trajectory direction vector model of airway lanes over East Bench of Salt Lake City, UT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 5.10 The Nominal versus Anomalous Behavior (NAB) method. . . . . . . . . . . . . . . . . . 84 5.11 NAB measures for a nominal flight. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 5.12 Examples of the five anomalous flight trajectories. . . . . . . . . . . . . . . . . . . . . . . . 86 5.13 Demonstration for how a vertical radar field could be placed in a grid layout lane system simulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 ix 5.14 Displays a couple of different packing arrangements for conical shapes. The image on the left is the optimal conical packing but not the most realistic solution. The middle image is the easiest solution to optimize coverage but it is too costly of a solution to be implemented. The rightmost image is our solution, which provides a high percentage of coverage with minimal overlaps. 87 5.15 The basic components of a conical shape; these are used to establish the required distance between radar placements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.16 Difference comparison between varying beam angles in the Monte Carlo experiment of 100,000 randomly generated sample points. This displays the inverse relationship between the angle size and the number of radar systems that would be needed for coverage. The image on the left uses a significantly smaller angle, resulting in more radars to cover the given area; whereas, the image on the right uses a larger angle of detection. The green dots represent the sample points that were within the radar fields. The empty spaces are areas in the lane-system that are not covered by the radar systems that have been placed. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 5.17 Properties that are used to determine if an object is within the radar’s field of view. These properties include the radar’s maximum range, and the angle between the object and the radar. The left drawing shows a vertical radar field, and the right drawing shows a more horizontal radar field. . . . . . . . . . . . 88 5.18 Visualization of the radar detection process during a simulation. The colorful circle represents the radar field, and the red dot represent the location of the detected object within the field. The first number in the title represents the time step and is followed by the radar identification number. . . . . . . . . . . . . . . 89 5.19 Results of feed forward neural net classification of flight trajectories into two classes: nominal (first fifty) and anomalous (remaining 250). On the left is the network learning performance data, and on the right is the classification result. 89 5.20 Amateur trajectory flying in circles with respect to the ground. . . . . . . . . . . . . . 90 5.21 Amateur flight with respect to airway network. . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.22 Amateur flight distance signature compared to those of noisy flights. . . . . . . . . 91 5.23 Noisy flights vector cosine wrt lanes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.24 Amateur flight vector cosine wrt lanes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 5.25 Amateur flight distance signature in a more dense lane area. . . . . . . . . . . . . . . . 92 5.26 Amateur flight location in a more dense lane area. . . . . . . . . . . . . . . . . . . . . . . . 93 5.27 Example rogue flight path. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 5.28 Rogue flight distance error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.29 Rogue flight direction error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 6.1 Example scenario: ground communication disrupted for multiple UAS. . . . . . . 111 x 6.2 (a) BDI architecture (taken from [3]). (b) Reinforcement learning Focus 1: cognitive level plan selection to achieve goal, and Focus 2: actions in the individual plan (this figure is adapted from [4]). . . . . . . . . . . . . . . . . . . . . . . . . . 111 6.3 MDP representation of BDI architecture (this figure was adapted from [5].) . . . . 112 6.4 Actions for UAS low-level reinforcement learning. . . . . . . . . . . . . . . . . . . . . . . . 112 6.5 States for UAS low-level reinforcement learning. . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.6 The state utilities and path through the highest utility states. . . . . . . . . . . . . . . . 113 6.7 The convergence of the state utilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 6.8 Optimal policies for the states (pictorially). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 6.9 The number of times each cell is traversed in 1000 trials. . . . . . . . . . . . . . . . . . . 115 6.10 The context-based probabilistic state transition probabilities for the case of a strong wind in the Y axis direction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 6.11 The convergence of value iteration with the context-based method. . . . . . . . . . 116 6.12 The policy produced by the context-based state transition method. . . . . . . . . . . 116 6.13 The number of times each space volume is traversed over 1000 trials. . . . . . . . . 117 6.14 Combined sequence and state diagram showing agent architecture. . . . . . . . . . 117 6.15 Flight path generation and assignment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 6.16 Wind and rain placement for training. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 6.17 Value iteration trace. N-Nominal, W-Wind, R-Rain. . . . . . . . . . . . . . . . . . . . . . . . 119 6.18 Transition probabilities for the Follow Lane plan. . . . . . . . . . . . . . . . . . . . . . . . . . 119 6.19 Nominal behavior without contingencies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 6.20 Learned policy with rain contingency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 6.21 Programmed policy with rain contingency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 7.1 Set of UAS on airways during discrete event simulation. Red dots represent UAS in flight; blue lanes are launch lanes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 7.2 CPA Algorithm: two flights at closest points Ptmin and Qtmin . . . . . . . . . . . . . . . . . 135 8.1 Sample straight-line paths between launch and land vertices. . . . . . . . . . . . . . . 146 8.2 Unstructured airspace density and path intersections. . . . . . . . . . . . . . . . . . . . . 146 8.3 Agent-based modeling framework for learning UTM policies and UAS behaviors. The UTM policies are set, and then UAS agents learn both at the BDI cognitive level (beliefs) as well as at the platform control level (plans). UAS behaviors are fixed and UTM policies are then optimized with respect to global lane stream properties as well as safety measures. . . . . . . . . . . . . . . . . 147 8.4 Single lane contingency effects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 8.5 Contingency effects on system state and individual behavior in a lane. . . . . . . . 148 xi 8.6 Lane diagram for a single lane, showing six flight reservations, planned trajectories, and simulated telemetry. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 8.7 Simulation results: averages for launch frequency of 100 flights per hour. The upper row describes the parameter combination enumeration in the lower three rows which give the mean number of failed fights, mean delay, and mean deconfliction for those combinations of parameters. . . . . . . . . . . . . . . . . . 149 8.8 Simulation results: averages for launch frequency of 1000 flights per hour. . . . 150 8.9 Simulation results: maxes for launch frequency of 1000 flights per hour. . . . . . . 151 8.10 Simulation results: maxes for launch frequency of 100 flights per hour. . . . . . . 152 9.1 The Utah Advanced Air Mobility Simulator (UAAMS). . . . . . . . . . . . . . . . . . . . 164 9.2 Methodology framework of proposed drone network optimization. . . . . . . . . . 164 9.3 TAZs with post office locations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 9.4 The location of drone centers affect which addresses trucks and drones service.166 9.5 Optimization iterations showing divergent behavior. . . . . . . . . . . . . . . . . . . . . . 166 9.6 Utah buildings dataset overlaid on population projections dataset. . . . . . . . . . . 167 9.7 The simulation form. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168 9.8 One frame of the simulation animation results. . . . . . . . . . . . . . . . . . . . . . . . . . . 169 9.9 Simulation-produced plots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 xii LIST OF TABLES 3.1 Some measures for the three network types used in the earthquake scenario simulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 4.1 Results of comparison experiments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 5.1 Classification results. The 6 trajectory types are (1) nominal, (2) hobbyist type I, (3) hobbyist type II, (4) hobbyist type III, (5) rogue type I, and (6) rogue type II. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.1 State transitions for UAS low-level reinforcement learning. . . . . . . . . . . . . . . . . 122 6.2 State transition probabilities for UAS low-level reinforcement learning. . . . . . . 122 6.3 Optimal policies for the states. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 6.4 UAS KB at initialization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 6.5 UAS desires and precedence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 6.6 UAS high-level plans. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 7.1 Delays and failures in experimental simulations. . . . . . . . . . . . . . . . . . . . . . . . . . 136 8.1 Contingency table; V2V (Vehicle to Vehicle V2V), V2G (Vehicle to Ground), and V2X (Vehicle to Everywhere). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153 9.1 The simulation parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 ACKNOWLEDGEMENTS This research was supported in part by the Utah Department of Transportation under grants UT-21.33 and UT-21.501A. I would like to thank Vista Marston for the contribution of the radar discussion. CHAPTER 1 INTRODUCTION The anticipated proliferation of autonomous, semi-autonomous, and human-operated vehicles in low-altitude airspace over densely populated areas requires a new system to automate air traffic control (ATC). The current system relies heavily on the intuition of human pilots and controllers who benefit from over one-hundred years of ATC history recorded from trial and error. Even after millions of test cases per year, contingencies occur that confound experts and result in disastrous outcomes (e.g., the failure of coordination that resulted in a mid-air collision over Uberlingen, Germany in 2002). The computational intractability of enumerating all possible sequences of actions and outcomes that lead to contingencies is the root-cause for these disasters; if engineers could have predicted the Uberlingen scenario, they would have prevented it (after the accident, TCAS software was patched to handle it, albeit at great expense due to the complexity of the software and subsequent testing). However, the density and dynamism of the anticipated low-altitude air traffic mandates an automated approach; it is difficult to imagine human controllers managing the separation of thousands of flights per hour. This is the conclusion of most professionals in aerospace across the United States, particularly the Federal Aviation Administration (FAA) and the National Aeronautics and Space Administration (NASA), as well as in Europe and Asia, where UAS Traffic Management (UTM) systems are being developed rapidly. However, if current ATC systems still experience contingencies after a hundred years, and millions of flights per year, what hope do engineers have in constructing a safe automated traffic management system? This question underlies the research presented in this dissertation. Contingencies typically require a diagnosis and prognosis to conceive of mitigating plans. For example, a communication problem could be due to a temporary power-outage at a ground station, or due to a failure on-board a UAS integrated circuit. Both cases 2 demonstrate the same high-level symptom, but have very different diagnoses and prognoses. However, in any contingency scenario concerning UTM, maintaining safe separation with other aircraft is of singular importance to avoid a catastrophe and therefore must be taken into account by any mitigating plan. It follows that the problem of safe separation should be made as simple as possible. In other words, the computational complexity for maintaining safe separation must be kept low for individual operators and this can be accomplished by structuring the airspace. A number of structured and unstructured airspaces have been proposed for UTM, and they are explored in this research. The proposed structured airspaces, however, do not consider the computational complexity of planning within these frameworks (i.e., they do not consider the motion planning problem) and simultaneously their aggregate properties (e.g., the average expected maximum density). For example, to this author’s knowledge, this research is the first to bring Renyi’s theoretical observations of sequential interval packing into light regarding on-demand airspace reservations. This provides a analytical basis for designing UTM policies for lane-based airspace structures, for example answering the question of what should be the maximum density allowed on any given corridor to ensure priority access within five seconds of a request with 99% probability. This dissertation presents a lane-based approach to large-scale UAS traffic management and demonstrates its effectiveness for ensuring safe-separation with low computational complexity and predictability with respect to aggregate system behavior. These traits are demonstrated using metrics inspired from ground-transportation engineering and with agent-based simulation. 1.1 Research Goals The specific goals of this research, and the areas in which we intend to advance knowledge, are to develop an air-traffic control system with human-like intelligence (state abstraction and learning abilities) for contingency handling, efficient deconfliction methods using the lane-based approach, and methods for developing synergistic system-level (UTM) policies and UAS agent behaviors using the ABMS paradigm. UTM system designers, stakeholders, and regulators must be able to understand how their decisions can affect overall system behavior. Vehicle dynamics, infrastructure place- 3 ment, demand and capacity can have significant effects on system behavior in the same way they do for ground transportation systems. Currently, the FAA and NASA have proposed an unstructured airspace for UTM, and operators may reserve airspace (through their associated UAS Service Supplier or Provider of Services for Urban Air Mobility) consisting of any 2 dimensional polygon extruded between two altitudes and between two times. Originally conceived for low-altitude small UAS performing local missions, for example inspecting a bridge, this strategy has significant drawbacks for operations that involve transport. For example, this discretization allows operators to monopolize the airspace and deconflicting flights could involve an arbitrary number of comparisons between trajectories. Additionally, the aggregate behavior of the system requires a host of assumptions about the individual behaviors of operators and offers few explicit controls over traffic in general. The focus of this research is to characterize a structure for airspace that enables stakeholders to control the aggregate behavior of UTM and to reduce the complexity for individual autonomous agents to ensure safe separation. Unlike what NASA and the FAA currently propose, the lane-based approach explored in this dissertation takes inspiration from ground-based transportation management. This research characterizes the lanebased approach and compares it to the FAA/NASA model using aggregate measures, as well as metrics that are important to individual operators, such as delay. Additionally, this dissertation explores several applications that can utilize the proposed airspace structure, including anomaly detection, contingency handling, and package delivery. In all cases, the structure imposed on the airspace confers benefits in terms of predictability of behaviors (both aggregate and individual). 1.2 Background Our research team entered the Advanced Air Mobility (AAM) arena in August of 2018, when Andrew W. Buffmire, Research Corporate Ambassador for the College of Engineering at University of Utah, invited our research team to a meeting titled “UAS Modeling and Management,” held in the Halverson Conference Room in the Warnock Engineering Building. Among the twelve attendees was Jared Esselman, the Director of Aeronautics at the Utah Department of Transportation (UDOT), as well as a representative from the 4 Governor’s Office of Economic Opportunity, the University of Utah Hospital System, the university’s Entertainment Arts Engineering program, and Fortem Technologies. Before the meeting, an invitee from Utah’s Automated Geographic Reference Center (AGRC), sent an email to the group apologizing in advance and providing some data that foreshadowed the discussion: Hello all, I am unable to attend the meeting this Friday, but wanted to send out a few maps that may be helpful while having this discussion and let you know that we have access to several GIS data sets that may be helpful in putting together the air space model. Please let me know how I can help. Sean *Sean Fernandez, PLS* State Cadastral Surveyor/Division Manager State of Utah AGRC/DTS In one attachment (Figure 1.1), a map of features and the Salt Lake valley included locations of GPS sensors for Real Time Kinematic (RTK) positioning, liquor stores, libraries, post offices, correctional facilities, and schools. At the meeting, we discussed how the GPS network could be used by autonomous vehicles to accurately report their telemetry and maintain safe-separation. The other features on the map were endpoints in an imagined airspace network that transported cargo and passengers quickly, quietly and efficiently between locations within city limits. We treated the availability of vehicles that could meet the challenges of Urban Air Mobility (UAM) as an inevitability, and we focused on how our city could become a platform for developing this new transportation system. In Utah, a well-established and growing aeronautics and technology industry, research institutions with a running start on robotics, and a forward-looking governmental body at UDOT provided a means for making serious contributions to this vision of the future. However, it was also clear that this vision entailed considerable technical, logistical and business challenges to overcome. Advanced Air Mobility describes an emerging aviation market for local, regional, intraregional, and urban use-cases, supported by a set of disruptive technologies. The most salient technologies are the vehicles, and the promise that they will fly themselves effortlessly throughout the city has generated billions of dollars in private and public invest- 5 ment. Just a few of the publicly traded companies in this space, Joby Aviation, Lilium, Archer, and Volocopter represent over $4 Billion in market cap and have yet to transport a single paying customer. With so much investment and engineering effort going into vehicle development, it’s no wonder that early adopters and innovators are bullish, and it is easy to imagine, given the state of traffic in many cities, that a scenic ride inside a Tesla-like aircraft would be popular (assuming the price was right). Within each vehicle also exists numerous other disruptive technologies, electric and hybrid propulsion systems, energy storage systems, guidance and control software, advanced materials, etc. Each of these systems must interoperate or contend with an ecosystem of other vehicles and disruptive technologies in infrastructure, simulation, monitoring, and air traffic management. The minimum set of disruptive technologies necessary to enable this vision of urban air mobility is a subject of debate, and in the United States it will be determined by the businesses that are commercially successful. Disruptive technologies are innovations that alter the way people and industries operate, and the technologies that transform urban mobility are as certain to be disruptive as when selective availability was discontinued for GPS in the year 2000. Unlike GPS, however, the trajectory to enable mass adoption and commercial viability is less clear. In 2018, NASA hired two companies, Crown Consulting, Inc. and Booz Allen Hamilton, to study the market viability of urban air mobility. A couple months after our first meeting with UDOT and local stakeholders, NASA published the reports which identified key technologies and barriers. In one figure [6], Crown Consulting identified 34 technologies on the critical path of development, and divided them into 15 categories: autonomy, sensing, cybersecurity, propulsion, energy storage, emissions, structures, safety, pilot training, certification, communications, controls, operations, traffic management, and infrastructure. This categorization is not to say that these technologies do not depend on each other, there are complex relationships that must be managed between them during both development and production. Additionally, the airspace is heavily regulated, particularly in the United States where regulations have been developed over the past 100 years; this increases the barrier of entry for innovators due to the capital requirements and consequences of liability. The National Aerospace and Aeronautics Administration (NASA) and the Federal Aviation Administration (FAA) have stepped in to help facilitate 6 the coordination between industry and government; a mission statement from NASA’s AAM website (https://www.nasa.gov/aam/overview) provides a concise description of how they see their role: NASA’s vision for Advanced Air Mobility (AAM) Mission is to help emerging aviation markets to safely develop an air transportation system that moves people and cargo between places previously not served or underserved by aviation - local, regional, intraregional, urban - using revolutionary new aircraft that are only just now becoming possible. AAM includes NASA’s work on Urban Air Mobility and will provide substantial benefit to U.S. industry and the public. The Aeronautics Research Mission Directorate (ARMD) initiated the AAM Mission Integration Office during the 2020 fiscal year with the objective to promote flexibility and agility while fostering AAM mission success and to promote teamwork across ARMD projects contributing to the AAM Mission. The AAM Mission will address a broad set of barriers necessary to enable AAM which will be accomplished with the contributions made by projects across the mission directorate. Aside from governmental players, large corporations with institutional reputations have also stepped in to offer commercial solutions that revolve around an ecosystem product concept, a location for providers of services for AAM to market their products. 1.3 Visuals and Concepts After the first meeting with UDOT, our research group produced a conceptual simulation of airspace corridors over Salt Lake City. One of the issues that we had discussed was the problem of where low-altitude aircraft would fly and how the public concerns about privacy and noise might be addressed. At the state-level, the roads are public property and so the idea was floated that aircraft should simply fly over the roads. Presented as a short video with a camera that rotated over the city, we demonstrated how 3D semi-transparent rectangular corridors could be constructed from geographic information system (GIS) data that was available from state agencies. Although the simulation lacked precise placement of corridors and their merge points, the visualization had the effect of catalyzing more conversation in the state and generating interest from multiple industry stakeholders, including GE (AirXOS), AirMap, and Bell Aircraft. Airspace visualizations and simulations are a powerful tool to guide conversation and facilitate coordination. Visit any website of the major players, Joby Aviation, Uber, Google, 7 AirBus, etc., and you’re bound to find an animated visualization depicting the future of air travel. However, simulations are normally constructed to answer more specific engineering problems, rather than as a marketing tool to generate interest. Our initial visualization was constructed using an open source library called NASA Worldwind. This is a 3D geospatial visualization library with bindings for Java and web technologies such as Javascript. The video that we shared with UDOT and others was a Java program that read GeoJSON data describing the road network around the University of Utah, then constructed 3-dimensional corridors at a fixed height above the ground. A small applet with 3D controls then allowed the user to pan and rotate around the area-of-interest. A movie was then created by programming incremental rotations about a fixed center and storing frames from the applet at a rate of 15 frames-per-second (fps). The width and height of these corridors were chosen arbitrarily to make the visualization appealing. Additionally, several spherical objects, representing aircraft, were programmed to “fly” through the corridors. That same year, NASA announced that it would end support for Worldwind, so our team looked elsewhere for a visualization tool. Most of our rapid prototyping efforts utilize MATLAB, with visualization presented on a generic 3D canvas (using the plot3 function). However, there is also a sense, garnered through many conversations with stakeholders, that to make AAM research palatable to a large audience it would require more specialized geospatial visualization tools. To this end, a business opportunity arose: a platform for AAM related products could support and accelerate advanced air mobility by making it easier to pitch, develop, test, and deploy research and software technology. 1.4 Technology Opportunity In the push to adopt Advanced Air Mobility, stakeholders include governmental bodies charged with overseeing air space utilization, as well as Providers of Services for Urban Air Mobility (PSU), and UAS Service Suppliers (USS). UAS operators such as Amazon, UPS, hospitals, etc. are anxiously awaiting operational Unmanned Aircraft System (UAS) Traffic Management (UTM) which will enable package and drug delivery, as well as unmanned air taxi services. Companies like Airmap, Bell Helicopter, GE and others have expressed great interest in exploiting such a system. NASA has done market surveys which indicate 8 that by 2030 there may be 750M air taxi flights and 500M package deliveries per year in 15 major cities. In addition, this work may allow efficient integration and synergy between ground and air vehicles. Finally, the existence of such a system will also enable the acquisition of a whole new source of big data (flight data, sensor data, communications data, weather data, etc.) which may form the basis for a wide variety of new services. Current research and product development aim to catalyze the adoption cycle that underlies the nascent industry of Urban Air Mobility (UAM). In its 2020 forecast publication [7], the FAA acknowledges that “it is extremely difficult to put a floor on the growth of the commercial UAS sector due to its composition and the varying business opportunities and growth paths.” However in the same study they say, “if, for example, professional grade small UAS (sUAS) meet feasibility criteria of operations, safety, regulations, and satisfy economics and business principles and enter into the logistics chain via small package delivery, the growth in this sector will likely be phenomenal;” phenomenal, relative to the forecast of about one million non-model aircraft operating for commercial reasons in 2024, each registering multiple flights per day [7]. This fleet does not include the vehicles expected to deliver about one million express packages in that same year, according to a study conducted by NASA [6, 8]. The FAA also estimates between 12,000 and 23,000 passenger-carrying autonomous aircraft operating within urban environments by the year 2030. As the FAA suggested in their assessment however, these estimates rely on the assumption that UAM technology will be adopted and that efficient concepts of operations (CONOPS) can be developed. Consulting reports and conversations with industry stakeholders indicate that most believe regulation to be the highest inhibiting factor to growth of the UAM industry. However, NASA’s own funded study regarding the barriers to adoption indicate a much more complex landscape, including technical factors as well as market conditions. Therefore, the more realistic view sees regulation as an outcome of progress in the technological development of this industry. The more realistic characterization is where conflicts exist between every pair of stakeholders, and it is the complexity of these relationships that inhibits growth. 9 # k j k & $ # SOB Law Enforcement ! Liquor Stores State Fuel Sites Libraries Post Offices Correctional Facilities ! ! Schools ! TURN GPS Stations ' 4 ! ! ! ! ! ! ! #$! $ !& k ! ! ! ! !! ! ! ! ! k ! ! ! ! ! ! ! ! k! ! ! ! ! ! ! ! ! ! ! ! & ! ! ! ! & $! ! ! ! ! ! ! ! ! ! ! ! $ !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! & ! ! k !!! ! ! ! ! ! !! ! ! ! ! ! ! $ ' 4 !! ! ! ! ! ! ! ! ! ! # !& ! !! ! ! ! ! ! ! ! ! ! ! ! ! ! ! #!$ ! & ! ! ! !! k & #!! !! # !! ! ! ! ! ! ! ! $ ! ! ! #! !$ ! ! ! ! ! !# k! $ ! ! ! ! ! ! ! & # ! $ $! !! ! ! ! ! ' 4 !!$ ! ! k ! ! ! ! ! ! ! ! ! # k ! k ! ! !! # $ & ! ! ! !! ! !! & #! !! $! ! ! ! ! ! ! ! ! ! ! $ ' k jk ! 4 ! ! $ !!! $ $ ! ! ! $ !#! !!! !! ! $ ! ! ! k k $k k!$$! ! $$!& ! ! !$ !! ! ! ! ! ! # !k ! ! ! ! ! ! ! ! & ! !$ ! ! ! ! ! ! k ! 4$!!! ! ! k' k k k k ! $ & & ! !! # # ! !! ! ! ! ! ! $ !! ! ! ! k !! ! ! ! ! ! $ !! & ! ! & ! ! ! ! ! & ! ! ! ! ! ! ! ! ! & !! ! ! ! ! ! k & # ! ! $ ! k & ! ! #!k ! # ! !! ! ! $ ! ! ! $ ! ! ! ! ! ! ! !!! ! ! ! ! ' 4 ! ! ! ! ! $ ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! & ! ! ! & ! ! ! ! !$ ' 4! !$ ! ! $ # ! ! ! k ! $ ! ! #!!! ! ! ! ! ! ! k ! ! ! !!! ! !!! ! ! ! !$ !! ! ! &$ k ! ! ! # ! & ! ! ! # ! !! '$& # 4 ! ! TURN GPS BaseLines ! ! ! ! & k ! ! ! 7300m Reference Figure 1.1: Map of selected features in Utah for urban airspace modeling. CHAPTER 2 INTRODUCTION TO UAS TRAFFIC MANAGEMENT In early 2018, the director of the division of aeronautics at the Utah Department of Transportation (UDOT), invited our research team to a working group discussion about enabling Urban Air Mobility (UAM) in the Salt Lake Valley in Utah. At the time we had been working on a program sponsored by the Air Force Office of Scientific Research (AFOSR) to develop a Dynamic Data-Driven Application System (DDDAS) for Geospatial Intelligence [9, 10]1. Our research into data fusion techniques were relevant to UAM since they involved Unmanned Aircraft Systems (UAS), weather, and decision making. Representatives from industry, such as Fortem Technologies Inc., raised practical concerns about the positioning of radar systems for tracking low-altitude aircraft, while urban-planners discussed constraints for zoning, and local government stakeholders addressed public funding and perception. Over the course of that year, we met with vehicle designers like Bell Textron Inc., designers of the concept Bell Nexus, and system designers like General Electric AirXOS and AirMap. At the same time, NASA and other industry partners were gearing up for Technical Capability Level-4 (TCL4) flight tests for enabling small UAS (sUAS) (55 lbs. or less) operations in low-altitude airspace (typically uncontrolled or Class G airspace under 400 feet above ground level (AGL)), conducted at a Nevada, USA test site [12, 13]. While the operational requirements for sUAS (usually small business or hobby use-cases) differ from medium to large UAS at higher altitudes, the methods proposed by NASA and the FAA for distributed coördination rest on the same basic architecture, shown in Figure 2.1. This architecture describes the roles and responsibilities of agents in the UTM system, as well as the delegation of authority. However, questions about the inherent safety, predictability and scalability of the system require a deeper look into the 1 This chapter is based in part on enhanced versions of [11] 11 algorithms and behaviors that define the individual agents. This dissertation presents a structured airspace for UTM systems to help answer practical questions about expected system behavior. The lane-based approach is an effective organizational strategy because it allows efficient strategic deconfliction as well as the determination of the impact on contingency analysis and handling due to the interaction between agent behaviors and UTM policies. Complex systems deployed in the real-world are bound to experience contingencies: i.e., possible future events, usually causing problems or making further plans and arrangements necessary. Planning for contingencies, including conflict management, is a core computational issue underlying large-scale autonomous systems because computing optimal plans is an intractable problem (for both software systems and human operators). Therefore, the interplay between UTM policies (including airspace structure, communications, etc.), and UAS behaviors, which encode individual preferences and autonomy, can have a dramatic effect on contingency handling. The lane-based approach to conflict management structures the airspace with one-way volumes and special constraints on intersections, and represents both a practical organizational strategy as well as a tool for analyzing individual and system behavior. It is neither completely decentralized nor centralized, allowing system designers to make explicit compromises between preferences, and design for contingencies. Through agent based modeling and simulation (ABMS), new applications of spatial measures and analytical tools, and a comprehensive review of related research, the effectiveness of this approach will be shown. 2.1 NASA/FAA UTM Background NASA’s proposed architecture for UAS traffic management (UTM), shown in Figure 2.1, draws inspiration from the current national airspace system (NAS). It is a distributed computing system, where operators are individually responsible for planning their flights, ensuring they don’t conflict with any other planned flights, and obtaining the requisite permissions to fly from authorities. The overarching governing authority, represented by the Flight Information Management System (FIMS), delegates authorization to UAS 12 service suppliers (USS)2 , which are certified automated systems that ensure flights are strategically deconflicted3 before authorized. USS may then authorize flights for multiple operators or vehicles, and could represent an organization such as Amazon or Google, which may serve thousands of aircraft, or a smaller entity with one or a few aircraft. The main constraints that this proposed architecture applies is in terms of interfaces between the high-level operating agents (namely the PSUs, operators, and regulators). The sequences of interactions between them are still mostly undefined and flexible. For example, there is no rule that inhibits PSU from designing a structured airspace for its operators and then scheduling them using a proprietary protocol. There is also nothing stopping state regulators from imposing a structured airspace within which all PSU must operate. Consequently, these engineering questions remain open-ended and experimentation by NASA and industry is underway. In particular, there still remains the question of exactly how safe separation is achieved across all operations at all times. The FAA and NASA have delegated the responsibility of ensuring strategic deconfliction to the USS, but the specific methods for trajectory planning within this system are left unspecified. During several developmental test efforts held by NASA and the FAA, the concept of a Discovery and Synchronization Service (DSS) was invented to address the problem of locating other operations within an area. This role is critical to ascertain the state of the airspace and acts as a centralized database. The internal representation of the DSS is not a specific requirement, however the interfaces are being developed as an industry standard [15]. A prototype DSS has been constructed by Google’s Project Wing, called InterUSS [16], that has been used by NASA’s developmental efforts. The internal representation of InterUSS utilizes S2 cells at a fixed zoom level (see Figure 2.2). The unit sphere is decomposed into a hierarchy of cells by a framework called the 2 In this dissertation, USS and Providers of Services for UAM (PSU) will be used interchangeably. At the time of this writing, PSU was a relatively new acronym for a functionally similar role as the USS, with specific interface requirements for mid-to-large autonomous aircraft. 3 Strategic deconfliction, or strategic conflict management, refers to the first of three layers of conflict management defined by the International Civil Aviation Organization (ICAO), “achieved through the airspace organization and management, demand and capacity balancing, and traffic synchronization” [14]. This is generally understood to mean that before an operation takes flight, its planned trajectory does not violate minimum separation constraints with any other planned flight. The next layers are applied in order of the shrinking conflict horizon, they are tactical in nature and termed “separation provision” and “collision avoidance.” 13 S2 library, where four geodesics bound each cell of a quadrilateral. The cell hierarchy is created by projecting the faces of a cube onto the unit sphere, and then recursively dividing cells into 4 sub-cells. When an operation is planned, the USS queries these cells to find other USS operating in the area, as well as a unique token to mark the current state of operations in the environment. The other USS are then contacted to find the operational volumes they have reserved in order to deconflict the planned operation. Once deconflicted, the USS creates a unique reference to the new operation and registers it with the DSS, along with the previously obtained state token. If the airspace state has mutated during trajectory deconliction (a process which is not standardized), then the DSS rejects the new operation reference and the USS is required to restart the process. To date there is no published work on specific guarantees offered by this approach - one can conceive of a situation in which the environment state mutates so often and so quickly that a particular agent may never be able to fly. 2.2 UTM Scheduling Problem The online-over-time aspect of the UTM problem means that any algorithm, currently known to computer science, can only be an approximation of an algorithm that produces globally optimal solutions (for the most desired cost formulations, such as maximum delay). In other words, trade-offs between different types of UTM systems are essentially trade-offs between algorithmic heuristics. A number of problem formulations, developed for different application areas, provide a foundation for understanding the heuristic tradeoffs, as well as different perspectives represented as quantifiable aspects of the system. The following subsections describe some of the comparable problem formulations. 2.2.1 The Air Traffic Flow Management Problem A natural problem model comes from research into the Air Traffic Flow Management Problem (TFMP) [17]. In this model the airspace is partitioned into sectors that are controlled by regional regulators who provide separation services. The sectors are characterized by capacities that represent the maximum number of aircraft that may be in a sector at any time, and depend on factors such as weather. TFMP implements two control strategies to ensure that sector capacity constraints are not violated: ground-holding and 14 speed adjustment. Ground holding shifts the entire flight in time by delaying the departure of an aircraft. Speed adjustment is applied to each sector in flight and represents an “air delay.” Optimal ground-hold time and speeds for every planned flight are calculated, but each operation does not deviate spatially (this is called the Air Traffic Flow Management Rerouting Problem (TFMRP) [17]). Rios and Lohn [18] compare techniques for finding a solution to the Bertsimas and Stock-Patterson (BSP) model: binary integer programming, genetic algorithms, and simulated annealing. They also compare a greedy scheduler that schedules flights on a “firstcome, first-serve basis by finding the first available departure time for each flight in turn that will not violate sector capacities when combined with previously scheduled flights.” The greedy scheduler is so named because it is locally optimal for the flight in question, but it does not guarantee globally optimal solutions. Solutions to the BSP model provide time intervals during which a flight must enter each segment, and the solution is guaranteed to minimize total delay. Although the size of the problem formulation is bounded by a linear relationship between the number of intervals, the number of flights, and the number of sectors, the integer linear programming formulation suggests that there is no known time-polynomial algorithm to solve it [19]. Despite its non-deterministic features, this representation is appealing because it supports the goals of strategic conflict management, namely “airspace organization” via sectors, and “demand and capacity balancing, and traffic synchronization” via ground-holding and speed adjustment. What it lacks is an explicit representation for the in¡tersection of routes in 4-D space. The primary issue is that the sectors are large and there is no way to tell if routes intersect. One way to adapt this representation is to shrink the size of each sector such that capacity is fixed to one aircraft per sector. Bertsimas and Patterson explored this assumption and determined that the computational complexity of the TFMP is NP-hard [17]. Also, reducing the size of the sectors dramatically increases the space complexity. 2.2.2 The Job-Shop Scheduling Problem By shrinking sector capacities to one, the TFMP can be reformulated as a scheduling problem (see [19, 20] for a definition of the general scheduling problem, and [21] and 15 [22] for an overview of the job shop scheduling problem). Bertsimas and Patterson [17] reformulate the problem as follows: for each job create an aircraft and for each processor associate a sector (sectors include airports). Each job is composed of tasks that represent a flight segment (time spent in a sector). A solution to this formation is a total ordering of sectors for every job, and a list of flight times for each task such that the total delay is minimized, and all flights are performed by a deadline. This formulation guarantees that no aircraft will occupy the same sector at the same time and therefore satisfies the non-intersection requirement. There are however, several practical issues with this formulation. To begin with, the job-shop scheduling problem is NP-hard – this makes it a poor choice for USS that may need to contest with tens of thousands of “jobs.” Furthermore, it is not clear what the sector size should be, given the variation of UAS sizes expected to utilize the airspace. Too large a sector could result in an unreasonable amount of tactical separation maneuvers, while too small a sector could become computationally intractable. To account for uncertain speeds, the scheduling model can incorporate probabilistic durations. This formulation still suffers from the time complexity as before (and likely worse if the durations are not assumed to be independent random variables) [23]. 2.2.3 The Multi-Robot Motion Planning Problem Strategic deconfliction may be cast as a multi-robot motion planning problem. The key concept for any motion planning problem is the configuration space, which combines the kinematic constraints of the robot and the environment. Multiple robots may be combined into a conceptual “composite robot” [24] and the motions are planned in a joint configuration space. Centralized, or coupled, algorithms provide a path for every robot, while decentralized, or decoupled, algorithms usually provide solutions for a subset of the robots. In [24], the authors decompose a multiple robot planning problem into partitions of robots that are planned together. While this approach does reduce the complexity of the joint configuration space, it does not guarantee a reduced complexity of the problem because each partition can still be very complex. Other approaches, such as incremental coördination [25], combine the centralized and decentralized algorithms into a single iteration. 16 Multi-robot motion planning is also a more natural representation for autonomous vehicle coördination because solution methods, such as rapidly-exploring random trees, can incorporate dynamic constraints and uncertainty directly. The desire for optimality, however, results in a worse-case time complexity comparable to the job-shop scheduling problem. The two-phase decoupled approach [25] involves first computing a path for each robot individually while ignoring other robots, then operations are applied to the resulting path set to avoid collisions. The advantage of this approach is that the “search space explored by the decoupled planner has lower dimensionality than the joint configuration space explored by the centralized planner” [25]. The drawback is that it is an incomplete algorithm, meaning it is not guaranteed to find a solution even if one existed by considering the system as a whole. This approach resembles the greedy, first-come, first-serve algorithm described by Rios in the sense that previously planned paths are considered as static obstacles and each new flight is delayed until the capacity constraints are met [26]. 2.2.4 The Traffic Assignment Problem The traffic assignment problem (TAP) is a sub-problem in the transportation planning process that models the route-choice behavior of travelers given a set of possible routes [27]. This problem is mentioned here because prior research such as [28] measure the performance of the airway system by simulating origin-destination data from population centers. When determining the capacity of a particular network configuration, the traffic assignment problem should be considered separately because its benefit is mainly to predict the demand on the system. Solutions to TAP result in aggregate measures, “a macroscopic description or prediction of the traffic volume” [27]. The relationship between volume of travelers and their average travel-time are modeled by link performance functions [27]. Queuing models also play a role in the development of link performance functions. 2.2.5 The Optimization Problem The FAA expects tens of thousands of UAS to utilize the airspace in close proximity, therefore the problem model composition is important to ensure that minimum-separation requirements are met. There are two ways in general to represent the safety requirements, as a constraint and as an objective function. The objective is to maximize the separation (or headway) between UAS. Assuming the solution is optimal, the question of whether it 17 meets the safety requirement is determined by a threshold, e.g., “the minimum separation is at least 10 meters,” or “the minimum separation is at least 10 meters with 99.9% probability.” Given the complexity of the UAS strategic deconfliction problem, we propose to constrain consideration to a linear ordering model, that is, well-separated flow along lanes. Thus, the general 4-dimensional space-time trajectory problem, becomes one of deconflicting flights through a connected set of one-way lanes. This approach is developed in the rest of the dissertation. Previous work on lane related approaches has mostly been confined to manned aircraft. Devasia et al. [29] choose flight segments along established routes, aiming to decouple the problem of route optimization from safety considerations. For example, in commercial flights, airlines want to choose the best routes whereas the air traffic control system works to maintain safety locally. The authors propose a token-based system for entry into a designated area, and these tokens may be exchanged between airlines. Several aspects of their proposal are problematic, especially with respect to its application to UAS flight management: the “central idea is to hold all aircraft and let the aircraft with the lowest expected time of arrival [i.e., exit from the area] pass through,” and this may involve an indefinite hold for some aircraft. Another problem is that as flights are merged, “uncertainty [in time to pass through] in travel time grows linearly with the number of mergers.” Devasia et al. [30] again address manned aircraft and present a decentralized Air Traffic Control Method called “Conflict Resolution Procedure (CRP) based on highway-like routes” and give a way to choose flight segments during flight. However, this method does not deconflict segment endpoints (where routes merge), allows only two routes to be in conflict at a time, changes the route structure to resolve the conflict, and does not ensure fairness or liveness. Finally, Yoo and Devasia [31] extend CRP to consider turn rate limitations when routes are modified. 18 Figure 2.1: NASA/FAA proposed UTM architecture (from [1]). Figure 2.2: S2 cells covering a portion of New York. CHAPTER 3 LANE NETWORKS There are many reasons to fly UAS in an urban environment. Several expected high usage applications are package delivery (e.g., food, medical supplies, general goods), inspection (e.g., buildings, bridges, power infrastructure, etc.), and air taxi service. Major companies like Amazon, UPS, the Postal Service, etc., may deploy hundreds or thousands of UAS regionally per day. Every one of these UAS will follow some trajectory according to a specified time schedule; this is a 1-dimensional curve in a 4-dimensional space. If every UAS creates an individual and arbitrary 4-D curve, then every pair of trajectories must be checked to ensure safety (i.e., minimal separation at all times) and to meet the strategic deconfliction requirements proposed by NASA and the FAA. Moreover, given thousands of densely located UAS in the air at one time, ensuring safe operation may be too complex to allow for human pilots, and system-wide monitoring may be too complex for human air traffic controllers. This means that to enable large-scale UAS flight coordination flights must be autonomous and scheduled so as to avoid conflicts. 3.1 Lane-Base Urban Airways An alternative to a set of arbitrary trajectories is to create a pre-defined set of lanes through the air and to require that all UAS flights follow these lanes. Each flight must consist of a set of lanes; this starts with a launch lane that takes the UAS from the ground to the air, followed by a sequence of lanes through the air, and terminating with a landing lane that takes the flight from the air to the ground. To ensure safety, a time slot through each lane (i.e., a lane entry time and a lane exit time) must be reserved for each flight so that at no time are any two flights too close. This is called strategic deconfliction, and an efficient method for lane-based networks is provided in the next chapter. To make such a method possible, some constraints are placed on the network: 20 • each lane is one-way (i.e., the network is a directed graph), • the direction of an edge is related to compass heading (except those in a roundabout which are always counterclockwise), • each lane is at least as long as the headway distance, • no two disjoint lanes are within headway distance, and • either the in-degree or out-degree of every vertex is less than two. 3.1.1 Multi-Altitude Airways An easy way to obtain the basic layout of the airways over a given urban area is to begin by defining an undirected graph at ground level. For example, a simple grid may be affixed to ground locations, or the existing ground road network may be used wherein every road intersection or termination point is a vertex, and road segments between vertexes are edges. To achieve travel in both directions between air vertexes corresponding to a ground vertex, the air lanes must be placed either side by side at the same altitude above ground level, or one above the other. The convention used here is that lanes with travel in opposite directions will be vertically separated; moreover, travel in directions [0, π ) will be at one altitude and in directions [π, 2π ) in the other. To implement this, there are two levels of airways. In addition, a roundabout is created at each level above a ground vertex. Lanes to enter the airways from the ground are called launch lanes and connect a ground location to the lower altitude airway level. A landing lane connects the lower altitude airway level to a ground location. Both of these ground locations are near the ground network vertex. To achieve these connections, a vertex is placed in the roundabout to connect to a corresponding launch or lane location, respectively. Figure 3.1 shows a 2x2 ground grid network (an undirected graph with four nodes and four edges) and the corresponding air network (a directed graph with 40 nodes and 56 edges). Note that there are lanes connecting the the two roundabouts (separated by altitude) at a vertex location – one up and one down. This allows a flight to enter an air vertex at either altitude and exit along any lane leaving the vertex. Each ground vertex has an associated distinct ground location for its launch and landing lanes, if they exist. 21 As can be seen in the figure, these launch and lane ground vertexes connect directly to corresponding vertexes in the low altitude roundabout above the ground vertex. The lane-based approach defines a set of one-way lanes where each lane is defined by an entry point, an exit point, and a one-dimensional curve between the two (here we use straight line segments). UAS travel in three dimensions, and thus lanes are understood to be virtual 3D corridors (e.g., cylindrical-like tubes). The shape of corridors may change dynamically and should be constructed to account for the idiosyncrasies of the vehicles that they are meant to support; for example, smaller aircraft in windy environments may require a larger corridor radius than a heavier vehicle with better control dynamics. Further design constraints can be defined in terms of the headway – or safe separation distance – between UAS. The combination of headway and corridor design can support a range of vehicle trajectory constraints, while the directed graph (digraph) imposed on the airspace presents agents with a structured environment for computation (the lanes represent a complete model of the airspace under ideal conditions). Lanes may also have other associated properties (e.g., speed restrictions) specified by the UTM, enabling regulators to communicate requirements effectively to all agents in the system. Lanes are connected so that every vertex has either in-degree or out-degree equal to one (except launch lanes which have in-degree 0 and land lanes which have out-degree 0). This permits scheduling to be based on lanes as opposed to vertexes since all flights may be deconflicted based on one incoming or outgoing lane, and simplifies the analysis of congestion because various graph-based measures can be utilized to determine most likely high congestion parts of the network. This contrasts with zone-based deconfliction that presumes vehicles can enter and exit in any direction and the entire zone must be reserved (inefficient for large areas), and cell-based deconfliction that combines zone reservation with general motion planning within each cell (similar to the two-phase decoupled approach in [25]). The choice of the lane spatial layout is key to operational performance. As previously described, several alternatives exist: 1. airways modeled from ground road networks, 2. regular grid networks, 3. networks with specific properties (e.g., Delaunay networks). 22 We now give a more detailed account of the lane creation process. Lane creation starts with a ground network defined as a graph, G = (V, E), where V is a set of ground position vertexes, and E is a set of undirected edges between the vertexes. Figure 3.2 shows example road, grid and Delaunay networks for a small set of roads from San Francisco, CA. The grid and Delaunay vertexes are within the same geographic area; the Delaunay vertex locations are randomly generated. As will be shown later, the type of network impacts the spatial network measures of the graph. To create the two-level airways between vertexes, the ground network is duplicated as a set of airway lanes at two altitudes: one for travel in direction [0, π ), and the other in direction [π, 2π ). Since ground vertexes are road intersections, each is represented by two roundabouts in the air centered over the vertex; these ideas are demonstrated in Figure 3.3. A view of the airway network displayed over the San Francisco area is shown in Figure 3.4. Consider now, the creation of a roundabout for a vertex p with neighbors Q = {qi |i = 1 . . . n}. A set of vertexes will be placed on a circle of radius r with center p, where r is chosen so that lanes connecting the vertexes on the circle will be at least headway distance, h, apart. A vertex, vi , is created for every neighbor where the line between p and qi intersects the circle. To determine the value for r, consider Figure 3.5. Let d be the distance between two consecutive points on the roundabout. Then: d2 = r2 + r2 − 2r2 cos(θ ) (3.1) where θ is the angle between the two neighbors. In order to obtain d ≥ h, set: r= p h 4cos(θ ) (3.2) In addition to vertexes to connect to other vertexes, roundabout vertexes are needed to allow launch and/or land lanes. Finally, since there is a roundabout at each of the two altitudes of the airway, there must be a lane going up from the lower to the upper, and a lane going down from the upper to the lower. These extra vertexes are added midway between the neighbor generated vertexes (e.g., like A and B). 3.1.2 Single-Altitude Airways For some purposes a multi-altitude airway is undesirable; e.g., people in air taxis may be discomfited by frequent altitude changes. Thus, it is better to provide a single altitude 23 airway. Such an airway must still satisfy the lane constraints given at the start of the chapter. There are several ways to achieve this goal, and two of them are described here. In addition, one further constraint is added: the digraph must be strongly connected (except for landing lanes which terminate on the ground); this means that there must be a path from every vertex to every other vertex. The first option considered uses roundabouts, although they are constructed in a different way to multi-altitude airways. Given an undirected graph consisting of a set of ground vertexes and the edges between them, then a set of new vertexes is created for each. There are two cases to consider: 1. If a vertex, v1 , has only one neighbor in the undirected graph, then Figure 3.6 shows how a set of lanes can be defined which satisfies the lane constraints. This does not require a full roundabout, but must have two vertexes around each neighbor which allow lanes back and forth, as well as lanes to the interior of v1 . 2. If v1 has more than one neighbor, then Figure 3.7 shows a representative example of how this is handled. A pair of vertexes is created for each neighbor to allow back and forth travel, and lanes to and from v1 . All of the local cycles between v1 and its neighbors run clockwise. This method corresponds to the semantics of roads on the ground in that UAS travel in opposite directions in neighboring lanes in the same horizontal plane. The safety of such a structure requires adequate control of the UAS to avoid crossing into the neighboring lane. Another complication of this approach is the necessity to introduce new vertexes. An alternative approach is to take the given undirected graph, G = (V, E) as the starting point, and to define a strongly connected directed graph, G 0 = (V 0 , E0 ) where V 0 = V ∪ VL and E0 are defined below. VL is a set of ground vertexes for launch and land lanes, and are the only required additional vertexes. One way to achieve this is as follows. Assign P ← V 0 be the set of points under consideration. Initialize E0 = ∅. Next, set k = 1 and find the set of points, Hk , in the convex hull of P given in counterclockwise order; define a set of edges, Ek = {( pi , pi+1 ) | pi and pi+1 are consecutive points in Hk }. Then define Rk ← ( Hk , Ek ), as the kth ring for the airway. Now, set P ← P − Hk and k ← k + 1 and repeat the process so long as P 6= ∅. This will produce a set of convex polygon rings 24 with a path from every node in the ring to every other node in the ring. Next, a set of edges must be defined to connect the rings. The basic idea behind this is to run a ray from the center of the rings out, and create an edge between the pairs of neighboring rings using the points where the line intersects the rings. There are three separate cases to consider: the innermost ring may have (1) three or more points (i.e., it is a regular ring), (2) two points, or (3) a single point. In case 1, the mean of the locations of the innermost ring points is used to create the line. For case 2, the two points need to be connected by an edge – call it ( p1 , p2 ); then p2 , the head of the edge, is connected to the next outer ring, and some point on the next outer ring is connected to the p1 , the tail of the edge. If there is a single point, then an edge is created from the next outer ring to it and an edge from it to the next outer ring. The edges connecting the rings must maintain the lane constraints, and in particular, every node must have either in-degree or out-degree less than two; Figure 3.8 shows how this can be accomplished. The algorithm is: Using this method, a single altitude airway may be constructed (without roundabouts) and which is strongly connected. Figure 3.9 shows an airway developed from a set of vertexes from a grid, while Figure 3.10 shows an airway developed from a set random points. 3.2 Spatial Network Measures Given a specific airway network of lanes, it is important to be able to measure the effectiveness and efficiency of the network. To this end, a variety of measures have been developed (see [32] for a detailed discussion of spatial network measures for road, railway and other ground networks). Two types of measure are of interest: static and dynamic. Static measures depend directly on the structure of the graph and the spatial layout of the nodes. For example, a star graph (one node in the center with edges to a set of nodes distributed around it) has a strong point of congestion at the central node since all paths must go through it. On the other hand, dynamic properties (e.g., spatial flow - the number of UAS passing a given point per time unit) depend not only on the network structure, but also on the job mix of the flights and the scheduling algorithm. Generally speaking, static properties are useful when defining the network and dynamic properties help determine the UTM policies and scheduling algorithm parameters. 25 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 P ← V0 E←∅ k←0 while P 6= ∅ k ← k+1 Ek ← ∅ Hk ← convex hull of P Ek ← Ek ∪ {(hi , hi+1 ) | hi , hi+1 ∈ Hk are consecutive points} P ← P−H end while maxRings ← k if | Hk | > 2 p ← mean( Hk ) line1 ← line through p and [ p x − py ] T I+ ← intersections of line1 withEk in π4 directions I− ← intersections of line1 withEk in − π4 directions elseif | Hk | == 2 p1 ← first point in Hk p2 ← second point in Hk E ← E ∪ {( p1 , p2 )} line1 ← line through p1 andp2 I+ ← intersections of line1 withEk in π4 directions I− ← intersections of line1 withEk in − π4 directions E ← E ∪ {( p2 , q1 )}, where q1 is an appropriate intersection point with Hk−1 E ← E ∪ {(q2 , p1 )}, where q2 is an appropriate intersection point with Hk−1 elseif | Hk | == 1 p = point in Hk line1 ← line through p with direction π4 I+ ← intersections of line1 withEk in π4 directions I− ← intersections of line1 withEk in − π4 directions E ← E ∪ {( p, q1 )} where q1 is appropriate intersection point in Hk−1 E ← E ∪ {(q2 , p)} where q2 is appropriate intersection point in Hk−1 end E ← E ∪ {(qi , q j )} where qi , q j are intersection points on the ith and jth rings Algorithm 1: Algorithm convex hull single altitude airway. 3.2.1 Static Spatial Network Measures Static spatial network measures have been defined to evaluate the quality of a given (ground) transportation network (see [27, 32–34] for a detailed set of measures), and a set of flow measures (see [33]) as well. A subset of these have been selected to analyze the various road networks used as the basis for airways. A number of spatial network measures have been used to evaluate the quality of ground transportation networks, and they can be applied to airway networks as well. If a ground road network is the starting point for 26 the creation of an air network, then the measures can be applied to the ground network for initial consideration, and then applied to the subsequent air network as well. The results on these seem well correlated. Note that, in general, the ground network is a planar, undirected graph while the air network is a 3D directed graph. Let G = (V, E) be an undirected graph with |V | = N; then the set of static spatial network measures used here includes: • Density: | E | / | N |; the ratio of the number of edges to the number of vertexes. • Total Length: ∑i6= j dij where dij is the distance from node i to j; i.e., the sum of all edge lengths. • Minimum Path Length: (in steps) L(i, j); minimum number of edges from vertex vi to vertex v j . • Minimum Path Distance: D (i, j) (Euclidean distance); minimum distance traveled along the edges from vertex vi to vertex v j . • Graph Diameter: maxi,j L(i, j); longest path in graph. • Cyclomatic Number: τ = | E| − | N | + 1; number of reduced circuits in the graph. • Meshedness (also called α-index): τ 2N −5 ; • Density (alo called γ-index): | E| ; 3| N |−6 • Organicness (also called rn ): N (1)+ N (3) ; ∑ k 6 =2 N ( k ) number of cycles vs. total possible. percent existing routes to total possible routes N (k ) is the number of vertexes with degree k. • Route Factor: Q(i, j), i 6= j; for each minimal path from i to j, number of steps over distance. • Minimum Spanning Tree (MST) Cost: sum of all edge lengths of MST in G. • Overall Cost: • Efficiency: Total Length MST Cost . ∑i,j ( L(1i,j) ) . N ( N −1) • Detour Index: pairwise ratio of straight line distance over Minimum Path Distance. 27 • Betweenness Centrality: bc(v) = ∑s6=v6=t σst (v) σst , where σst is the number of shortest paths from s to t and σst (v) is the number of shortest paths from s to t through v. • Closeness Centrality: CiC = N −1 , ∑N j=1,i 6= j dij where i is the vertex index, and dij is the shortest path distance between vertexes i and j; measures how close a node is to the other nodes in the network. • Straightness Centrality: CiS = 1 ( N −1) ∑N j=1,i 6= j dijEucl dij ; captures how straight the shortest paths through a node are. Most of these measures can be used in a similar way to road analysis. However, the last three are some of the most useful measures (for example, see the recent work of Ahmadzai et al. [35] who proposed use of the Integrated Graph of Natural Road Network (IGNRN) and measure the three types of centrality on it to show how various hierarchies in the network can be determined). Even the application to the 2D road network which gives rise to the 3D airway provides insight into possible congestion points (betweenness centrality), centrally located launch centers (closeness centrality), and nodes through which fewer turns (due to lanes) are necessary (straighness centrality). In addition, these measures provide a visual mechanism to compare different network layouts. For example, Figure 3.11 allows comparison of grid and Delaunay layouts: brighter node are likely more congested. Also, note that the grid network betweenness values are about doulble those of the Delaunay network. Table 3.1 gives the values for some of these measures for three Salt Lake City, Utah graphs; the values are indicative of the performance using the graph. In particular, the betweenness centrality (BC) is useful in locating bottlenecks. Now consider the betweenness centrality of the airways over the East Bench area of Salt Lake City, UT. Figure 3.12 shows this measure for the airway constructed from the underlying road network. Figure 3.13 shows the betweenness centrality for the grid network, and Figure 3.14 displays that of the Delaunay triangulation. These measures provide clear insight into how the graph affects performance; the Detour Index can help a user select a path, and the last three provide useful information about congestion and flow through the graph. A high measure of betweenness centrality (BC) indicates that a node is prone to congestion since many shortest paths pass through 28 it; high closeness centrality reveals a good site for a launch or land site since the node is close to many nodes; finally, straightness centrality means that shortest paths through this node do not require many turns which can be important for UAS platforms. For example, Figures 3.12, 3.13 and 3.14 show the BC measure for three lane networks and high BC measure corresponds to higher congestion parts of the network. 3.2.2 Dynamic Spatial Network Measures We have also defined a number of network flow measures (adapted from a standard ground transportation framework – see [27, 33, 34]). These include the max UAS per lane: nmax = b d c hd (3.3) where d is the length of the lane, and hd is the headway distance; the time occupancy of a lane: Θ= nk tmax (3.4) where nk is the total number of UAS through the lane, and tmax is the max time considered; spatial occupancy per lane: ks = µ d (3.5) where µ is the average number of UAS in the lane per time unit; and spatial flow: qs = k s · s (3.6) where s is the average speed of the UAS. For the simulations performed here, most of these measures are low given the traffic patterns for supply delivery; however, one interesting result is time occupancy shown in Figure 3.15. As can be seen, this measure correlates rather directly with the betweeness centrality of the spatial network. The behavior of requests and the strategy for scheduling can have a significant impact on the average density of lanes. Consider a single lane system of length x, with one entry and one exit. Further assume that vehicles consume a one-unit spatial interval within the lane, and requests arrive over time independently for a uniformly random unit interval. In the first scenario, assume that each vehicle either obtains the requested reservation or 29 drops out, a “failure.” This scenario mirrors a 1-dimensional sequential interval packing problem, also known as Renyi’s parking problem [36]. Renyi showed that as the length of the lane approaches infinity, the mean filling density approaches 0.7476. This property also holds for the lane scheduling approach given here. The dynamic measures described above may be used to predict the behavior of lane networks and aid in the design of structured airspaces. However, to compare the lanebased approach with unstructured airspace approaches, such as that proposed by the FAA and NASA, some direct metrics can include: • Delay Time: absolute difference between desired and actual launch times • Deconfliction Time: wall-clock time required for deconfliction • Failures: number of flights that could not be scheduled due to conflicts. 30 Figure 3.1: A simple 2x2 grid network. Figure 3.2: Three types of road layouts over the same locale: actual San Francisco roads (left); grid layout (middle); Delaunay triangulation (right). 31 Figure 3.3: An example two-level grid lane layout of San Francisco roads. Figure 3.4: UTM airways over the San Francisco downtown area. 32 Figure 3.5: Geometry for determination of minimum required radius. A is the point of intersection of the circle and the line between the vertex and one neighbor; B is the point of intersection of the circle and the line between the vertex and the other neighbor; r is the circle radius; θ is the angle between the neighbors. Figure 3.6: Single altitude lane connections for a vertex with one neighbor. 33 Figure 3.7: Single altitude lane connections for a vertex with more than one neighbor. Figure 3.8: Rings must be connected so as to satisfy the lane constraints. 34 Figure 3.9: A single altitude airway developed from a set of grid points. Figure 3.10: A single altitude airway developed from a set of random points. 35 Figure 3.11: Example spatial measures; (a) The Betweenness Centrality of the grid network; (b) Delaunay network (right). Note that the nodes in the center of the grid are likely to be pinch points. Figure 3.12: The Betweenness Centrality of an airway created from the ground road network. 36 Figure 3.13: The Betweenness Centrality of an airway created from a grid. Figure 3.14: The Betweenness Centrality of an airway created from a Delaunay triangulation of a set of random points above the same area. 37 Figure 3.15: The time occupancy measure for the supply delivery scenario. Table 3.1: Some measures for the three network types used in the earthquake scenario simulation. Measure Density Total Len Graph Diam Cyclomatic No. Meshedness rn LT L MST Cost Efficiency Detour Index Betweenness Cen. Closeness Cen. Straightness Cen. Delaunay 2.95 31.73 30 978 0.98 0.006 4.5915e5 9.1041e4 5.04 0.1074 0.94 see graph see graph see graph Grid 1.91 32.59 45 483 0.46 0.160 2.525e5 1.3175e5 1.92 0.0840 0.80 see graph see graph see graph GIS 1.23 31.01 73 109 0.12 0.786 1.006e5 7.7962e4 1.48 0.0642 0.73 see graph see graph see graph CHAPTER 4 STRATEGIC DECONFLICTION The Federal Aviation Administration (FAA) and NASA have provided guidelines for Unmanned Aircraft Systems (UAS) to ensure adequate safety separation of aircraft, and in terms of UAS Traffic Management (UTM) have stated[37]: A UTM Operation should be free of 4-D intersection with all other known UTM Operations prior to departure and this should be known as Strategic Deconfliction within UTM ... A UTM Operator must have a facility to negotiate deconfliction of operations with other UTM Operators ... There needs to be a capability to allow for intersecting operations. The latter statement means that UTM Operators must be able to fly safely in the same geographic area. The current FAA-NASA approach to strategic deconfliction is to provide a set of geographic grid elements, and then have every new flight pairwise deconflict with UTM Operators with flights in the same grid elements. Note that this imposes a high computational burden in resolving these 4D flight paths, and has side effects in terms of limiting access to the airspace (e.g., if a new flight is deconflicted and added to the common grid elements during a proposed flight’s analysis period, then the proposed flight must start the deconfliction analysis all over). In prior research the lane-based approach was proposed to manage large-scale UAS traffic management [11, 38] which uses one-way lanes, and roundabouts at lane intersections to allow a much more efficient analysis and guarantee of minimum separation. Presented here are the results of an in-depth comparison of FAA-NASA strategic deconfliction (FNSD) and Lane-Based Strategic Deconfliction (LBSD) and demonstrate that FNSD suffers from several types of complexity which are generally absent from the lane-based method. Small Unmanned Aircraft Systems (sUAS) are to be integrated into the low altitude (Class G) airspace, and initial concepts have been provided by the NASA UAS Traffic management (UTM) project [1]. A set of four Technical Capability Levels (TCL) have 39 been defined, and TCL 4 addresses “an urban environment and includes handling of high density environments, large-scale off-nominal conditions, vehicle-to-vehicle communications, detect-and-avoid technologies, communication requirements, public safety operations, airspace restrictions, and other related goals.” UAS Service Suppliers (USS) provide key functions for managing the airspace, and in particular, they are charged with ensuring strategic deconfliction (SD) of flights, and other services usually provided by the Air Navigation Service Provider (ANSP) in manned aviation. In addition, USS are charged with monitoring flight operations. All of these functions are to be achieved in a distributed, coöperative manner. NASA’s vision is that USS perform SD by making sure that any proposed flight has no 4D (space and time) conflict with any scheduled flight in its area of operation. This also permits arbitrary flight paths. There are clear problems with the FAA-NASA approach to strategic deconfliction, including the the high computational complexity of arbitrary 4D path planning, as well as a susceptibility to the monopolization by organizations seeking to control airspace. An alternative approach structures the airspace using well-defined lanes [11, 38] in which a (generally) fixed set of corridors (airways) are established, and then flights are scheduled through these lanes. This means that strategic deconfliction is reduced to a 1-dimensional problem that is solved by a simple application of ground-delay. This chapter provides a detailed set of experimental results, allowing for the analysis and comparison of the two alternative approaches. The results indicate that the lane-based approach is superior with respect to the metrics considered. 4.1 FAA-NASA Strategic Deconfliction The FNSD is based on a gridded approach in which the area of flight operations is divided into a number of grid elements, and each flight scheduled by a USS keeps track of the grid elements over which it operates. Then when a new flight is being scheduled, it only needs to deconflict with the flights with which it has common grid elements. The current approach proposed by the FAA/NASA is shown in Figure 4.1, where USS1 and USS2 have a number of scheduled flights (USS1 flights in green and USS2 in blue). These flights have already been deconflicted for operation over some time interval. USS3 wants to schedule a flight (red dashed line), but in order to do so, must deconflict 40 flights pairwise with both USS1 and USS12 which means all flight paths must be shared. If some other USS manages to deconflict and schedule a flight in this space-time before USS3 , then USS3 must start the process all over from the beginning [16]. Note that thousands of flights a day are envisioned, thus making the complexity of strategic deconfliction overwhelming. Figure 4.2 shows an example grid layout with a 4x4 set of grid elements (i.e., about 1320 feet on a side). For FNSD analysis we consider a flight path to consist of a polyline with three line segments: 1. segment 1: [ pt1 , pt2], a segment going straight up from a launch site to a chosen altitude, 2. segment 2: [ pt2 , pt3 ], a segment going across the workspace at a fixed altitude, and 3. segment 3: [ pt3 , pt4 ], a segment going straight down to the ground. Each flight path is comprised of a user selected launch site, land site, and flight altitude. In addition, each flight has a designated start time, and fixed speed for the entire flight. Note that this may be relaxed to allow mixed speeds without much modification of the structure of the deconfliction procedure. Given a set of flights, the convention is that they are requested and deconflicted in the order of the list. That is, the first flight is scheduled as specified since there are no flights scheduled before it, the second flight must only deconflict with the first, etc. The deconfliction strategy used is based on ground delay of the flight until it has no conflicts with scheduled flights in its grid elements; this allows a fair comparison to the lane-based method which is also based on setting a conflict free launch time; moreover, this is the way standard air traffic control is accomplished. The FNSD algorithm used here is: Although this is just an example of a deconfliction method, the statistics accumulated will be somewhat independent of the particular method. This is due to the fact that the complexity is related to the number of scheduled flights which share grid elements with the flight request and the nature of the segment interactions in the grid element. This method first eliminates from consideration any scheduled flight which shares no grid elements. Next, it eliminates any which share grid elements, but none of whose flight segments are within headway distance of the flight request segments. Finally, to determine 41 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 On input: scheduled flights, flight request, delta t, headway distance On output: new flight if flight request shares no grid elements with scheduled flights then new flight is requested flight (earliest time); return if there are no flight segments in the scheduled flights within headway distance of the flight request segments then new flight is requested flight (earliest time); return pinch pts = all segment pairs of scheduled flights that are within headway distance of the flight request segments while (any pinch point segments have the two flights within headway distance during their traversal of the segment) shift the start time of the flight Algorithm 2: FAA-NASA strategic deconfliction. whether segment pairs that are within headway distance actually pose a problem, the time of passage of the two flights must be considered. For example, if the entry-exit time through the scheduled flight pinch segment does not overlap the entry-exit time interval of the flight request segment, then there is no conflict. Finally, if these time intervals overlap, then an analysis is performed to see if the flights get within headway distance while crossing their respective segments; if so, the start time for the flight request is delayed a fixed amount, and the impact re-analyzed. The pinch point segments are determined as follows. Let P0 ∈ <3 and P1 ∈ <3 be the endpoints of segment 1, s ∈ [0, 1], and Q0 and Q1 be those of segment 2. Define: P(s) = P0 + s( P1 − P0 ) (4.1) Q ( t ) = Q0 + t ( Q1 − Q0 ) (4.2) w(s, t) = P(s) − Q(t) (4.3) Then the distance squared between P(s) and Q(t) is: | w(s, t) |2 = w(s, t) · w(s, t) (4.4) When the distance is less than the allowed headway, then the pair of segments is recorded. 42 4.1.1 A Detailed Analysis of Strategic Deconfliction As described above, a framework is being developed to support large-scale (thousands) of UAS flights per day over urban areas, and NASA has proposed a UAS deconfliction strategy that requires service providers (UAS Service Suppliers or USS’s) to exchange full flight path information and to mutually find a deconflicted set of flights. This approach has high complexity and sacrifices UAS operator privacy. In contrast, a lane-based deconfliction strategy which reduces the shared information to be simply lane entry and exit times and UAV speed through the lane can serve to hide trajectory information. Then given a requested launch time interval, it is possible to determine the set of all allowable (deconflicted) time intervals within the requested interval. Techniques proposed for flight planning, include full mix and layered methods [39, 40] for which safe separation is maintained by tactical collision avoidance methods in otherwise unconstrained flights. While several heuristic methods have been developed for this problem (e.g., [41]), it is still possible that the number of conflicts may overwhelm the algorithms (see [42] for an analysis of cascading effects of conflict resolution). There has been a large amount of research into quantifying the risk of conflict in this type of system (e.g., [28, 40, 42–44]), indicating that there are numerous risk factors that an operator would need to consider in order to reduce the risk of collision. Lane-based airways were analyzed in [45], however the UAS operations were not deconflicted pre-flight and instead were simulated much like car-following models (e.g., [46]). Recently, a report published by NASA detailed the negotiations among stakeholders regarding requirements for USS SD. Furthermore, they discuss the requirement that any scheme for strategic deconfliction must be mandated by the airspace regulator. The Strategic Deconfliction Problem is to produce a set of scheduled flight paths such that no two aircraft ever get closer than a specified safety distance (specified either in space or time). Strategic deconfliction, or strategic conflict management, refers to the first of three layers of conflict management defined by the International Civil Aviation Organization (ICAO), “achieved through the airspace organization and management, demand and capacity balancing, and traffic synchronization” [14]. The next layers are applied in order of the shrinking conflict horizon, and are tactical in nature and termed “separation provision” 43 and “collision avoidance.” Broadly speaking, strategic conflict management deals with planning collision free paths, which in the most general case of planning for multiple agents is PSPACE-hard [47]. Even the more narrow problem of tuning velocity profiles is NP-hard [48]. The approach described here considers the simpler, but more realistic scenario given the UTM architecture, of scheduling UAS in real-time within lanes, reducing the configuration space of the UAS to a single dimension for each flight. The result is a practical, computationally tractable algorithm for strategic conflict management. The theoretical contribution here is an efficient algorithm for strategic deconfliction. Experimental results are provided which takes into account the capacity constraints imposed by the system and enables airspace regulators to make informed decisions about how to address user demand. The majority of motion planning algorithms rely on some form of discretization, e.g., cell-decomposition or probabilistic sampling such as Rapidly Exploring Random Trees (RRT) [47, 49]. The algorithms that don’t rely on discretization either assume a functional representation of trajectory (e.g., a spline) or are tactical because they apply to controls directly. The decisions related to discretization are vital in determining the effectiveness and complexity of a motion planning problem. For instance, in the RRT algorithm the line connecting sampled locations must be discretely sampled to determine if any conflicts exist. If the sample resolution is too fine, then computation resources suffer. If the sample resolution is too coarse, then there is the possibility that a conflict exists that would not be discovered until it was too late. The strategic conflict management problem shares characteristics with many application areas, as well as theoretical work in discrete mathematics (see [50] in the context of scheduling) and topology (see the chapter on configuration spaces in [49]). This includes the Air Traffic Flow Management Problem (TFMP) [17–19], The Job-Shop Scheduling Problem [19–23], The Multi-Robot Motion Planning Problem [24–26], The Traffic Assignment Problem [27, 28], and Optimization Problems [51, 52]. The FAA expects tens of thousands of UAS to utilize the airspace in close proximity over urban areas, therefore the problem model composition is important to ensure that safety requirements are met. There are two ways in general to represent the safety requirements: using constraints, or with an objective function. The objective is to maximize the separation (or headway) between UAS. Assuming the 44 solution is optimal, the question of whether it meets the safety requirement is determined by a threshold, e.g., “the minimum separation is at least 10 meters,” or “the minimum separation is at least 10 meters with 99.9% probability.” Lanes, as proposed here, are created by an authorized party (e.g., the Department of Transportation), and may allow USS proposed lanes which can be approved by the UTM authorities. Each lane has an entry point and an exit point and allows one-way travel from entry to exit. Where lanes intersect, we introduce an airspace structure inspired by roadway roundabouts. In addition, we provide a computationally tractable trajectory scheduling algorithm for UAS Service Suppliers (USS) within this structure. A capacity analysis follows the description of the airway structure to provide a baseline for further research. Prior research into the capacity of airspaces does not simultaneously consider the complexity of planning the operations; however, both concepts must be considered together since the airspace regulator is expected to manage both. Over dense urban areas which are of primary concern here, there will most likely be a limited set of lanes possible, and understanding the capacity of the lane system is important to urban planners. The lane-based method proposed here can be seen as an extension of Victor and Jet Airways used in manned air traffic management [53]. However, these were rigidly defined off of VOR systems (Very High Frequency Omnidirectional Range) in the 1960’s. Moreover, such routes were under visual flight rules, and at intersections required human deconfliction. The innovation in our approach is the dynamic nature of lane creation and deletion, as well as the introduction of roundabouts which permit efficient strategic deconfliction. Finally, we note that in the following, it is not assumed that UAS have the same speed in a lane, multiple levels (altitudes) are used in the lanes, and although lanes may be above roadways, this is not required; on the latter subject, it should be noted that NASA has stated (emphasis added) [54]: With regard to the routes that UAM will traverse between two vertiports, a natural starting point for emergent UAM operations is to fly along defined helicopter routes ... These helicopter routes tend to overlay highways and freeways on the ground to mitigate societal concerns In the experiments described here, shortest route lane sequences are generated over urban areas (although arbitrary lane sequences may be used), but these are not large distance interstate routes, and the altitudes of lanes are somewhat arbitrary but are safely 45 separated. 4.2 Lane-Based Strategic Deconfliction Given a set of lanes created as described in Chapter 3, then UAS Service Suppliers (USS) will receive requests from UAS operators who want to obtain authorization to travel from a specific launch location through the lane network to a specific land location. The launch and land locations will be the entry and exit points of the launch and land lanes, respectively. An approved flight plan for the UAS will consist of a sequence of lanes and corresponding lane entry and exit times. Given a sequence, L = {L1 , L2 , . . . , Ln }, then the exit point of Li is the entry point of Li+1 , and the exit time from lane Li is the entry time to lane Li+1 . L must be specified as part of the UAS request; however, it is possible that UAS Traffic Management (UTM) systems may handle lane sequence selection differently. In order to understand the interaction of flights in a lane, the Space Time Lane Diagram (STLD) provides a concise description of the trajectories through lanes; this is similar to those used for ground road networks. Figure 4.3 shows a simple example of an STLD with two flights indicated by sloped line segments. The x-axis is for time while the y-axis gives the distance along the lane. Another STLD is shown in Figure 4.4, this time with endpoints marked by the the time they are expected to leave the lane. It is assumed that flights through the lane can be represented by their average speed through the lane. Thus, a flight, f i , through the lane is indicated by a line segment with endpoint1 at (ti1 , 0), and endpoint 2 at (ti,2 , dk ), where ti,1 is the time flight f i enters the lane, ti,2 is the time it exits the lane, and dk is the length of lane k. The other main issue is the determination of whether a proposed flight conflicts with any scheduled flight. An airway lane constrains the trajectory of the UAS to the center-line of the airway, referred to as the longitudinal direction of the aircraft trajectory in prior research (e.g. [42]). The vertical and lateral directions are assumed to be under control to remain inside the lane. Uncertain altitude and lateral movements should be compensated for in the design of the width and height of the airway; this is a subject of ongoing research. Also, a constant velocity is assumed within a segment; this constraint will also be relaxed in subsequent research. Suppose that there exists a set of scheduled flights which are represented in terms of 46 lane enter-exit times and speed through each lane (the speed of a UAS is assumed constant along a lane, but speeds may differ across UAS’s). 4.2.1 The Label Method Let F (c) be the set of scheduled flights through lane c defined as: F (c) ≡ {rt11 , rt12 , s1g ; ...; rtn1 , rtn2 , sng } (4.5) where rti 1 is the lane entry time for flight i and rti 2 is the lane exit time, and sig is the speed of the flight through the lane. Furthermore, let a flight request interval be specified as: R ≡ [q1 , q2 , srg ] (4.6) where q1 is the first possible launch time, q2 is the latest possible launch time, and sr is the proposed speed. What must be determined is the set of (possibly disjoint) intervals in R which are possible launch times (i.e., strategically deconflicted). In order to determine this, the requested launch time interval is put in the Lane 1 STLD as shown in Figure 4.4(a), where d1 is the length of Lane 1, q3 is q2 + d1 srg , and q4 is q1 + d1 srg . Each flight in Lane 1 is considered separately to ensure that the time headway, ht , is respected. To determine safe launch time intervals, first consider the labeling of the STLD shown in Figure 4.5. The labels are defined as follows: • Label 1: The interval [0, q1 ) • Label 2: The point q1 • Label 3: The interval (q1 , q2 ) • Label 4: The point q2 • Label 5: The interval (q2 , ∞) • Label A: The interval [0, q1 + • Label B: The point q1 + d1 srg d1 srg ) 47 • Label C: The interval (q1 + • Label D: The point q2 + d1 srg , q2 + d1 srg ) d1 srg • Label E: The interval (q2 + d1 srg , ∞ ) The two trajectories arising from the scheduled flight are labeled according to where their endpoints lie with respect to the requested launch interval. For example, if pi,2 < q1 and rti 1 + d1 /sig < q1 + d1 /srg , then the label for that trajectory is 1A since both start and end points are in the first intervals at distances 0 and d1 , respectively. The relation of a previously scheduled flight in a lane to the requested launch time interval is determined by the labels of the two scheduled flight headway trajectories; the requested launch interval is shown in red. Figure 4.6 shows the first 13 possible combinations. For example, 1A,1A is the case where both headway trajectories are completely to the left (i.e., before in time) the first possible launch time trajectory through the lane. Note that in the figures, p1 is rti 1 − ht , p2 is rti 1 + ht , p3 is rti 2 + d1 , sig and p4 is rti 1 + d1 sic , and ts is the time for the requested flight to cross the lane. Also, the square brackets ([]) in the figure indicate the empty interval. Although there are 625 total label combinations, only 139 are physically possible; for example, no start time can be greater than the end time (see Appendix A for a complete enumeration and Appendix B for a Matlab program which determines the correct combination and returns the correct interval set). For each combination, it is possible to give the safe launch intervals contained in the requested interval (see the figure for some examples). In some cases, there is no possible safe launch time (e.g., 1A,1E in the figure). For other combinations, the resulting safe intervals depend on the relative speeds of the two UAS’s. An example of this is 1A,3C where a scheduled flight slower than the requested flight has a different interval as when the scheduled flight is equal or greater in speed. It can also happen that multiple intervals result as shown by the 2B,3C case in Figure 4.7. To determine the viability of a flight through the complete sequence of lanes, each lane is considered in order as described by the Label Method Algorithm. The key computational cost of this algorithm is the determination of f int; each instance of this can be done in constant time; call it operation I . Then given n lanes, f k flights in lane k, and f is the total number of flights in the lane sequence, then the total number of I operations is less than or equal to: 48 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 On input: lanes: lane sequence for requested flight [q1 , q2 ]: requested launch interval nc : number of lanes flights: flights per lane ht : maximum required headway time On output: Safe time intervals to launch begin possible intervals ← [q1 , q2 ] for each lane c ∈ lanes time offset ← time to get to lane c possible intervals ← possible intervals + time offset for each flight, f , in lane c new intervals ← ∅ for each interval in possible intervals [t1 , t2 ] ← interval i f f 20 22 23 24 25 f f f interval ← get interval(label,rt1 , rt2 , s g , t1 , t2 , srg , ht ) new intervals ← merge(new intervals,f intervals) 19 21 f f label ← get label(rt1 , rt2 , s g , t1 , t2 , srg , ht ) 18 end end possible intervals ← new intervals end possible intervals ← possible intervals - time to last lane Algorithm 3: The label method. n ∑ k =1 fk + ∑ fi f j (4.7) i6= j f The second sum dominates the complexity, and assuming f k is on average n , and since n there are terms, then the big O complexity is O( f 2 ). 2 An important point is that the lane-based formulation can apply in all cases where the trajectory is reduced to a single dimension. It is not required that lanes follow the road network on the ground; lanes may be organically created by operators and reused by other operators. The critical aspect of this formulation is that there are no crossing-conflicts. In general, we assume that the headway distance, hd , is a known quantity. However, there are some constraints on how it is determined. Note that in the scheduling problem, the headway distance is converted to a headway time. The relation between the two is given by: ht = hd /s (4.8) 49 where ht is headway time and s is the speed of the UAS under consideration. The value for hd is further complicated by the angle formed by two merging lanes (see Figure 4.8). If UAS1 is moving along lane i and UAS2 along lane j, and they are to stay at least hd apart, then there is a distance from the merge point, Q, of the two lanes that the other craft should remain outside. This distance, d, is given by: d= hd 2sin( 2θ ) (4.9) The UAS speed, s, to use is the slower of the speeds in its current or previous lane. 4.2.2 The Backprojection Method The kinematic motion of the UASs may be described as follows: uas1 : x1 = s1g (t1 − rt11 ) (4.10) uas2 : x2 = s2g (t2 − rt21 ) (4.11) y where x y is the longitudinal position (meters) within an airway segment for uasy , s g is y the ground speed (meters/second), ty is the time along the segment (seconds), and rt is the release time, i.e., the time at which the UAS begins its trajectory across the segment. Also note from Figure 4.4 that x0 and x f represent the start and end of the segment, so that x f − x0 = length(segment). The time headway (distance between UAS in time) and the space headway (distance between UAS in space, sometimes referred to as spacing) are given by ht and h x , respectively. The error bars in Figure 4.4 represent the required spacing between UAS, also known as well-clear in the UAS literature. The vertical distance from a point on the line to the error y y bar is the well-clear and is denoted h x for uasy . Due to the linear nature of the problem, ht y and h x are related by: y ht = y hx s g,y (4.12) This equation mirrors the relationship between density (or occupancy in space), speed, and flow (or occupancy in time) described in the Highway Capacity Manual [55]. This is important because it connects the concepts of road capacity, well known in road-traffic engineering, to airway capacity, which is explored in the following sections. 50 The separation constraints for any two UASs may be described as follows: y y h x = | x1 − x2 | > max (h1x , h2x ), ∀ x y : x y ∈ [ x0 , x f ] ht = |t1 − t2 | > max (h1t , h2t ), ∀t y (4.13) (4.14) y Since ht and h x are linearly related, it suffices to consider only one constraint. These separation constraints are more general than the one considered in [42] to describe the capacity analysis in a foundational way. UAS operators may prescribe a required headway as needed by their vehicle and other operational considerations. Consider the case where uas2 is already scheduled and now a USS is presented with uas1 to schedule. Since v1g is considered constant, rt1 (the release time for uas1 ) is the only decision variable. Let ht,max = max (h1x , h2x ) and rt1 < rt2 ; we can describe the first position at which well-clear is violated by the following equation, xv (s1g − s2g ) + s1g s2g (rt21 − rt11 − ht,max ) = 0 (4.15) where xv is the position along the segment where a violation first occurs. When the velocities are equal, then this equation reduces to the simple relationship, rt11 = rt21 − ht,max (4.16) The corresponding constraint for planning purposes is then, rt11 < rt21 − ht,max (4.17) This assumption of uniformity of velocities is assumed in the experimental section to make network capacity constraints more visible. In the general case, however, when s1g > s2g , then xv is negative for all rt11 < rt21 and therefore the only constraint is the same as Eq. 4.17. When s1g < s2g , then the violation point may lie within the segment (this is the case in Figure 4.3). The constraint is therefore: rt11 < rt21 − ht,max − −s1g s2g xf , m= 1 m s g − s2g (4.18) 51 4.2.3 Backprojection Algorithm The algorithm based on backprojection given here is a greedy scheduler: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 input rd , re , rl , path, s g rd ← desired release time re ← earliest release time rl ← latest release time path ← requested segment ids s g ← speed seats ← available time slots ls ← 0 % The segment length for each segment in path seatssegment ← seats on segment at t ∈ [re , rl ] + ls sg seats ← seatssegment | seats % Binary OR ls ← segment length rt ← open seat closest to rd return rt Algorithm 4: The backprojection method. It is called “greedy” because the scheduler only considers the currently requested operation and minimizes the distance between the scheduled and desired release time. In other words it is locally optimal with respect to the desired release time. It is not globally optimal, in the sense that there may have been a better solution if all operations were considered simultaneously. In the UTM system, where operations are scheduled online and desired release times are unknown to the scheduler until the request is made, a globally optimal algorithm may not exist. To see why, this problem may be cast in terms of what Pinedo would describe as an online job-shop scheduling problem with no-wait constraints [20]. Specifically, this is an online-over-time problem because the scheduler “does not know at any point in time during the process how many more jobs are going to be released in the future and what their release dates are going to be” [20]. It is also classified as clairvoyent because all relevant information, such as speed, are available to the scheduler. It may be possible that a USS knows when its operations will be requested, however it is still true that it will not know when another USS’s operations will be requested (at least not in the currently envisioned UTM system). The no-wait constraint refers to the fact that, in the scenarios considered in this system, UASs cannot wait (park or hover) between successive segments. The problem of minimizing maximum lateness (a measure of the worst violation 52 of due-dates), for a single machine with requested release dates (in Pinedo’s nomenclature 1|r j | Lmax ), is NP-hard [20]. A polynomial-time online algorithm therefore represents an approximation of the optimal algorithm. This algorithm applies equally well to homogeneous and heterogeneous velocities, however only the homogeneous setup is considered here. The heterogeneous version of this algorithm applies additional time-headway as required by the term 4.2.4 xf m in Eq. 4.18. FAA-NASA versus Lane-Based SD Comparison Given the FNSD and LBSD approaches, the goal is to perform simulation experiments to better understand their respective advantages and disadvantages. To achieve this, some performance measures are defined. In addition to providing comparison metrics for the two methods, a set of measures for evaluation of different lane-based strategies is also described. Both of these are studied in terms of the following framework. A 100x100 unit area is considered, where 1 unit corresponds roughly to 10 feet. For the lane based system, a 6x6 grid of ground locations is defined, and the subsequent airways based on that; nearest eight-neighbors are connected, and every ground vertex has both launch and land lanes. For the FAA-NASA flights, a 5x5 grid is defined (i.e., grid elements are 20x20 square units). UAS speeds are in the interval [0.1,0.31], as these correspond to 3-10 mph. The altitude for the lanes is between 10-12 units, while for the FAA-NASA flights, it is set to 11 units. All flights are specified as between two ground vertexes, and lane-based flights take a shortest route through the lanes, while the FAA-NASA flights follow a 3-polyline trajectory of up, over and down. Note that this makes these latter routes shorter than the lane-based routes. A set of 1,000 flights is scheduled in each scenario; however, if the deconfliction takes more than 30 seconds for some flight, then data from the first 75 flights is used to interpolate a result for all 1,000 flights. Within this context, we consider three scenarios: • Scenario 1: The launch and land ground vertexes are selected to be the two most distant (i.e., 1 and 36). Figure 4.9 shows the lane-based and direct routes for this. The speed of every flight is fixed to be 0.12, and all flights follow the same trajectory. The start times interval for these flights is [0,2000]; that is, a flight should be assigned the earliest possible launch time in this interval. Finally, the minimum headway distance 53 is set at 1 unit. • Scenario 2: This is the same as Scenario 1, except that the launch and land ground vertexes are chosen randomly. • Scenario 3: In this scenario, the launch and land ground vertexes are chosen randomly, as are the UAS speeds, and the start times interval. Each UAS has its own speed which us constant across the whole flight, and the initial start time is randomly selected in the interval [0,1000]. Note that Scenario 3 is closest to a real situation. Table 4.1 gives the data collected from the experiments. As can be seen, the lane-based method does better in Scenarios 1 and 3, while the FAA-NASA approach performs better on average delay, but not maximum delay, on Scenario 2. Scenario 1 represents the scheduling problem on a heavily used route while Scenario 3 is more representative of a random arrival process; thus, we believe that these are more reflective of actual operational situations. Another observation is that these results are achieved in the context of all flights being nominal, that is, no contingencies occur. Since the lane-based approach has a distinct advantage with respect to contingencies, then these results indicate the overall superiority of the lane-based approach. Also, note that cost of deconfliction in the lane-based approach in Scenarios 1 and 3 is two to four orders of magnitude lower. Another advantage of the lane-based method is that it is possible to easily visualize the flight schedules through the lanes. Let’s consider an example from each scenario. First, consider flight 10 in Scenario 1. Figure 4.10 shows the complete lane sequence for the flight with all other scheduled flights. Flight 10 is shown in red, and since all the flights follow the same lane sequence and have the same speed, they are all represented as parallel line segments where the lower end point represents the launch time, and the upper end point represents the landing time. It is also clear that this representation makes it easy for a flight operations center controller to visually determine if flights are off course by overlaying telemetry data on top of this graph. Figure 4.11 shows the corresponding Space Time Lane Diagram for Scenario 2. Here it can be seen that Flight 10 is the only flight scheduled along this specific route, but that other flights are scheduled at various times on some of the lanes in the route. Finally, Figure 4.12 demonstrates how readily system-wide type information 54 is made evident by these graphs; note that the number of segments in the upper lanes (later part of the flight) indicate that there may be some congestion in that region, and it might be wise to find alternate routes so as to avoid that. The variety of slopes in the graph indicates the different speeds of the flights through the lanes. 4.2.5 Lane Stream Properties Properties specific to the lane-based approach may be defined given a set of lanes. To do so, assume an airway lane of length d and consider a time interval of length tmax , call it [0, tmax ]. Also assume that all UAS fly through the lane with a constant speed, s. A flight scheduler assigns start times for flights to go through the lane; let S be a set of such start times. Then, to satisfy constraints, it must be the case that no two start times are closer than headway time, ht , of each other. This is equivalent to packing segments of length ht into the lane (time) interval. Note that h x = s · ht is the headway distance. The maximum number of UAS possible in the lane at one time, ntmax , is then: ntmax ≡ b d c+1 s · ht (4.19) Clearly, achieving ntmax depends on obtaining a perfectly packed requested start time sequence. Suppose that flight request start times are sampled from a uniform distribution across the given time interval [0, tmax ]. The time occupancy, Θt (A), is a function of the scheduling algorithm A and is defined as: Θt (A) ≡ µA ntmax (4.20) where µA is the mean number of flights through the lane during the time interval [0, ds ] of several trials with algorithm A. If the scheduler has no choice but to assign the requested start time if possible and otherwise reject the request (call this algorithm A0 ), then this is an example of Renyi’s Parking Problem [36, 56, 57], and Θ(A0 ) → 0.74759 as tmax → ∞. In the experiments below, we compare algorithms and lane parameter sets by means of their observed time and space occupancy measures. Next consider standard ground traffic stream properties: density, occupancy and flow (see [34] for a detailed discussion). The spatial density of the lane at time t, k s (t), is defined as: 55 k s (t, A) ≡ µA d (4.21) that is, the average number of vehicles in the lane over the length of the lane. Spatial occupancy can then be defined as: Θs (t, A) ≡ Θt (A) · ndmax d (4.22) Finally, spatial flow, qs (t, A), is defined as: qs (t, A) ≡ k s (t, A) · s (4.23) These traffic stream properties are used to characterize the performance of a set of algorithms compared in the experimental section. These measures are given as a means of comparing the effectiveness of alternative lane scheduling algorithms. Since that problem is not addressed here where we compare the FAA-NASA approach to lanes, we simply give the values for these measures for Scenario 1, where the flights are most densely packed. The following values result for Senario 1 for the launch lane: ntmax = 8 (4.24) since h x = 1.41 Θt (A) = 8 =1 8 looking at the time interval [0,83.333] (since µA = 8 and 83.333 = k s (t, A) = Θs (t, A) = (4.25) 10 0.12 ). 8 = 0.8 10 (4.26) 1·8 = 0.8 10 (4.27) qs (t, A) = 0.8 · 0.12 = 0.096 (4.28) A direct comparison of performance characteristics has been made between the FAANASA and lane-based UAM approaches. A variety of scenarios were examined, and measures defined on the computational and other requirements over a set of flights. The lane-based method was found to outperform the FAA-NASA approach in the most likely actual conditions which will be encountered in large-scale UAS traffic management. Although the lane-based method requires flights of slightly longer route, there are multiple 56 advantages in terms of management. Finally, the incorporation of manned drones in this system is possible so long as the human pilots follow the assigned flight path; in addition, human pilots would be in a better position to handle contingencies. In Chapter 8 we describe the use of Agent Based Modeling and Simulation (ABMS) to determine more optimal UAM parameters related to lane properties and their layout, as well as lane speeds, and auxiliary lane support structures (e.g., emergency lanes alongside regular lanes, emergency landing lanes, etc. In addition, Chapter 6 looks into real-time adaptive lane scheduling by the UAS themselves. This may be particularly useful locally in contingency situations. 57 Figure 4.1: Sketch of current FAA-NASA proposed strategic deconfliction. Figure 4.2: The basic experimental layout with a 4x4 grid. 58 Figure 4.3: Space-Time Lane Diagram (STLD) for two UASs in a lane. The abscissa is time and the ordinate is distance along the lane. ht is the time headway (distance between UASs in time in lane), and h x is the space headway (distance between UASs in lane). Note that ht and h x are linearly related due to the constant speed. The two trajectories in this scenario intersect at t = 4 and x = 2, however, they violate space-headway before then. 59 Figure 4.4: Space-Time Lane Diagrams: (a) trajectory boundaries for requested launch time interval [q1 , q2 ]. (b) The headway boundary trajectories for a scheduled flight which enters the lane at time p and exits at time p0 . Figure 4.5: Space-Time Lane Diagram Labels. 1,2,3,4,5 indicate intervals and times at the entry to the lane, and A,B,C,D,E indicate times at the lane exit. 60 Figure 4.6: Space-Time Lane Diagrams for the first 13 possible label combinations. Figure 4.7: Space-Time Lane Diagrams for possible label combinations 57 through 71. 61 Figure 4.8: Required distance along lane to avoid violating headway constraint. Figure 4.9: The lane-based (blue) and FAA-NASA (red) routes from ground vertex 1 to 36. 62 Figure 4.10: The stacked set of space time lane diagrams for flight 10 in Scenario 1. Figure 4.11: The stacked set of space time lane diagrams for flight 10 in Scenario 2. 63 Figure 4.12: The stacked set of space time lane diagrams for flight 10 in Scenario 3. 64 Table 4.1: Results of comparison experiments. Scenario 1 Avg Delay Max Delay Avg Flight Time Avg Comparisons Avg Decon Time Scenario 2 Avg Delay Max Delay Avg Flight Time Avg Comparisons Avg Decon Time Scenario 3 Avg Delay Max Delay Avg Flight Time Avg Comparisons Avg Decon Time LSD 494.97 989.95 1,527.00 1,249.50 0.0063 LSD 19.87 67.72 879.49 61.20 0.0212 LSD 1.65 30.39 532.88 49.84 0.0014 FAA-NASA 583.61 1,167.20 1,318.00 1.49x107 138.53 FAA-NASA 10.73 917.60 705.68 27.40 0.2524 FAA-NASA 16.85 325.20 451.65 28.62 0.3579 CHAPTER 5 AIR TRAFFIC OPERATIONS CENTER The AAM community, including UAS service providers, operators and relevant government authorities, aims to provide a wide number of services (e.g., package delivery, air taxi, etc.) by means of robust and safe UAS Traffic Management (UTM) systems which achieve large-scale (i.e., thousands per day) operations in urban areas without human control, but with reliable communications and contingency planning (see [37])1 . UAS Service Suppliers, for example AirMap [58] have dealt with the operational interfaces, integration of Geographic Information Systems (GIS), registration of flights, UAS communications, and monitoring UAS activity. These capabilities are all developed to operate as described in the strategic deconflicton context defined by NASA [1] which is defined in terms of a geographic grid (a set of cells). Each new flight must be deconflicted pairwise in terms of grid cells which have other scheduled flights. The trajectory of a UAS flight is a curve in 4-dimensional space (x,y,z,t). Given set of such curves, strategic deconfliction (i.e., make sure that no two flights are ever within a specified minimum distance called spatial headway) necessitates a pairwise comparison of the curves with motion constrains, and this is in general PSPACE hard. Moreover the FAA and NASA have yet to specify contingency protocols, and some research suggests that all flights simply return to base in these scenarios (e.g., lost-link). In previous chapters, we have proposed and studied various aspects of a lane-based approach [11, 33, 38, 59–61]. This lane-based approach reduces strategic deconfliction complexity (to 1D from 4D) and makes the handling of contingencies a spatially local problem [11]. The use of lanes for commercial flights (Victor and Jet Routes) has a long-standing history [53]. However, human air traffic controllers manage commercial airway lanes, and this management function which must be automated if a large number of autonomous 1 This chapter was co-authored by Vista Marston. 66 flights are to take place daily over major metropolitan areas. Given a lane-based UTM system, then lanes are created as a static structure (much like ground road networks) and all scheduled flights will follow a sequence of assigned (reserved) lanes from launch to landing. Figure 5.1 shows a lane-based airway over downtown San Francisco, CA. If all UAS flights performed in the lane-based system as planned, and no unexpected events occurred (e.g., unplanned flights entering), then there would be no need to monitor the airspace. But since unexpected things do happen, some sort of traffic management center is necessary to help mitigate the effects of unforeseen events. In addition, an air traffic control operations center (ATOC) can serve to apply UTM policies and enforce the rules as necessary. ATOC operators can monitor activity in the lanes and use existing conditions as part of the flight plan approval process. For example, if there are strong winds in part of the airspace, the ATOC can provide advisories and route flights away from the impacted area. In addition, the ATOC can gather airway performance data and modify policies accordingly to improve things. It is also possible that existing ground transportation infrastructure can be leveraged to support ATOC operations. For example, if road intersections already provide power (say for cameras at traffic lights) and network access, then upward looking radars can be placed there to observe the airways. It may also be useful to have close coördination of the ATOC and ground traffic management operations, especially if UAS fly over roads. This allows rapid dissemination of disruptive events like an emergency landing on a road. It is most likely that ATOC’s will improve transportation performance. Certainly, this has been the case with ground transportation management systems (GTMS). For example, the TransStar system in Houston, Texas helped reduce traffic delay by over eleven million hours per year. In Utah, the CommuterLink system decreased intersection delay by 27% and increased freeway speed by 20% [62]. A full-blown GTMS for a large metropolitan area can be a costly proposition, and the cost for an ATOC would presumably be similar. A ball-park cost projection given by [2] is about $3M for a building, and another $3M annually for operation and maintenance. Of course, monitoring or provisioning hardware infrastructure imposes more cost. The same study gives a Business Plan Development Guide which shows the steps necessary 67 to develop a Traffic Management Business Plan (see Figure 5.2). Planning for an ATOC would have a similar set of considerations. However, one major additional requirement would be for some form of interaction with both the FAA and also directly with UAS flights. Working with the FAA is necessary to maintain the safety of simultaneous manned and unmanned air activity. Communication with UAS operators in necessary to receive real-time telemetry data from UAS flights. 5.1 Example ATOC Requirements by Utah Department of Transportation The Utah Department of Transportation2 (UDOT) has been studying the issues of Urban Air Mobility since 2018 with their prime focus on the airspace of the Provo to Ogden corridor. The main two use cases considered here are small package delivery and air taxis. Traffic reduction is a major goal for departments of transportation, and there is potential for UAS to offload some of the work performed by ground transportation. Other pertinent factors include the push by industry (e.g., Airbus, etc.) to have full integration of autonomous vertical take off and landing vehicles for human transportation. The hardware infrastructure deemed necessary for a reliable urban airway system includes: • Automatic Dependent Surveillance-Broadcast (ADS-B) with links to unmanned aircraft, • GPS Real-Time Kinetics (RTK) for precision vertical operations, • Dedicated Short-Range Communications (DSRC) for seamless connectivity, and • Active Radar to provide detection and avoidance capabilities. UDOT has previously proposed radar coverage over the Salt Lake Valley, shown in Figure 5.3. Other infrastructure considered by UDOT and stakeholders during working group meetings held in 2019 and 2020 include GPS (Figure 5.4) and fiber networking (see Figure 5.5 that shows the current advanced traffic management system (ATMS) and fiber 2 We would like to thank Jared Esselman for this material. 68 network), which could serve an important role for navigation and communication for Advanced Air Mobility vehicles. Finally, UDOT proposed the acquisition of (1) several more GPS-RTK reference stations for safety redundancy, (2) Mini-radar/ADS-B with an estimated 40 stations needed covering about 1,267 km2 with power and network connections already available through UDOT, and (3) DSRC wireless. The estimated cost for this infrastructure system was estimated as about $200K for GPS reference stations and $12.7M for mini-radar/ADS-B hardware. In terms of ATOC UTM capabilties, UDOT established goals to: • manage low-altitude air traffic, • provide real-time notification of flight plan changes, • communicate hazards or priority routing, and • account for traffic load, impact of skyport placement, weather, rogue and emergency air traffic. UDOT also proposed that UAM traffic be routed over existing roads and along planned flight corridors with pre-approved flight plans, with assigned altitudes related to direction of flight and with speed and location constraints. Figure 5.6 shows the proposed air layout. Advanced Air Mobility and supporting infrastructure will necessarilty be built up incrementally as technology, demand and resources converge to make it possible. Current UAS platforms require a great deal of improvement and demonstrated robustness in order to meet the stringent safety requirements of air transportation. UAS management software is still in its infancy, the package and air taxi industries are not yet at scale, and automated air monitoring and enforcement mechanisms have yet to be developed and demonstrated for these types of vehicles. Thus, the development of Advanced Air Mobility (AAM) systems requires not only a robust and safe approach to planning flights, but also a way to monitor UAS flights in real-time to determine whether flights are deviating from their expected flight trajectories and tools for diagnosing their behavior. The lane-based approach a method of trajectory 69 monitoring called Nominal vs. Anomalous Behavior (NAB), an efficient and effective way to monitor flight trajectories to determine normal versus anomalous behavior. 5.2 Lane-Based Monitoring The basic problem addressed here is how a lane-based UTM system supports the recognition of rogue flights of a variety of sorts: amateur recreational hobbyists, UAS operators making an unscheduled up, over and down flight, malicious operators, etc. In order to detect such rogue flights, we propose the analysis of trajectories based on their deviation from the lane structure, including both location in space and direction of flight at that location. The basic idea is that a model can be produced directly from the lane structure and compared to any flight individually. The alternative FAA approach would require knowledge of all 4-dimensional flight trajectories, as well as multi-target tracking to monitor the flights along those curves and a comparison of an unidentified flight to all of those curves. Thus, the proposed lane-based method is much more efficient and effective. 5.2.1 NAB Modeling Given a set of UTM lanes, a convenient model is just a set of point samples on the lanes, each with an associated direction of the travel in the lane. Figure 5.7 shows a lane-based airway over the East Bench area of SLC, UT. Figure 5.8 shows a set of sample points from the lanes. These provide a good model since any nominal flight (i.e., following its assigned lane sequence) should be near a lane and headed in the direction of the (one-way) lane. As part of the model, the direction vectors can also provide significant information about a flight. Figure 5.9 shows a subset of the trajectory direction vectors used in the model. 5.2.1.1 The Lane-Based Trajectory Model A trajectory is a sequence of 4-tuples (x,y,z,t) that provides a representation of the actual UAS motion through space and time. Trajectories can be divided into two basic types: expected and unexpected. An expected trajectory is one which results from a planned flight trajectory combined with an expected error due to contextual influences (e.g., weather, power, etc.). An assigned trajectory in terms of the lane structure is a sequence of straight line segments (nominally), but when exiting one lane to enter another, the actual flight path follows a smooth curve. Wind and precipitation may also cause a flight to be off-course. To 70 account for this, lanes are viewed as having an enclosing 3D volume (e.g., a tube structure) containing the straight line segment, and this allows room for error. In addition, the spatial headway constraint provides more cushion for speed changes, etc. Trajectories may arise from various flights, but a scheduled UAS is required to provide telemetry data at a set update rate; this data comes from on-board sensors. Alternatively, radar or other sensors may be used to monitor flights and provide an independent source for trajectory data. Next, consider a planned flight and its associated trajectory. When the UAS sends telemetry data, it also sends its ID. This makes it possible to determine if the flight is offcourse and by how much. It is also possible to monitor the airspace and produce locations of airborne objects. We assume that it is possible to determine which objects are UAS with high probability (as opposed to birds, etc.). Given such data it is possible to corroborate UAS telemetry location data. The result is that planned flights have expected trajectories in that they are near the planned path or their reported locations are consistent with ground sensor data. On the other hand, a flight that has not been planned produces a trajectory that corresponds to the type of the flight, and, in general, the trajectory will not correspond to any planned flight, will not be corroborated by sensor data, and will not follow a sequence of connected lanes. This is an unexpected trajectory. Of course, there are ways that UAS can insert themselves into the lane structure and mimic a scheduled flight (e.g., by following a scheduled flight), but this can also be detected in that they do not provide telemetry data. Unscheduled flights in the airspace are called rogue flights, whereas unexpected trajectories are called anomalous. In practice, the ATOC needs to detect rogue flights as robustly as possible. The NAB method is one such approach and operates as described in Figure 5.10. The lane data is made available along with the UTM policy parameters, and the set of scheduled flights. Based on this, a spatial database is constructed consisting of a set of 3D points sampled along the lane, and to each of these points is associated the direction of travel vector at each of those points (recall that lanes are one-way). The lane model consists of this data organized so as to be efficiently exploited. The inter-sample distance must be selected so to keep the number of points down while at the same time allowing adequate discriminatory power to determine if a flight is near a lane and headed the right direction. For example, consider the Salt Lake City East Bench airways shown previously. If an 71 inter-sample distance of 2 meters is chosen, it produces a set of 454,331 points. In order to keep the computational complexity low, the points are organized as a kd-tree using the 3D points. A kd-tree divides the points at the median of each of the vector dimensions in turn (or picking the dimension with the greatest spread in values in turn), and thus achieves a relatively well-balanced tree which allows O(log(n)) search time to find the closest points in the spatial data to a query point. Any nominal flight should be near one of the sample points and headed in the appropriate direction. Of course, a temporal analysis can be performed by checking the associated Space Time Lane Diagram (see [11]) which specifies the position of each flight in a lane at each time instant. Also, with the FAA-NASA approach, there is no fixed set of lanes, and therefore, every existing flight would require target tracking against the set of all flights. Since this approach requires large-scale sensing and computing resources, it is not considered further here. Next a set of NAB measures are determined which allow the discrimination of the different types of flights, both nominal and rogue. These are computed either by comparing the UAS trajectory to the lane data, or simply in terms of the trajectory itself. For example, two lane related measures are: 1. Mdist : minimum distance to a lane at each time step, and 2. Mdir : cosine of the angle between UAS direction of travel and the lane direction of travel at each point. These measures are applied at each point in the trajectory to produce a temporal signature to represent the flight. An example of a measure based solely on the trajectory data would be the amount of time spent hovering (i.e., staying for some minimal duration in time in one place in space). Given a characterization of the types of flights of interest, then a set of trajectory signature templates can be constructed and used as class models. Such templates can be the result of a set of simulations or produced from data sets of actual flight trajectories. Given a new trajectory, its measured features are compared to the flight signature templates and matched to the closest in order to classify the type of trajectory (i.e., nominal or anomalous). Consider a nominal flight which does not perfectly follow the lane but rather has some noise associated with it. Figure 5.11 top row shows the x values of a nominal flight 72 trajectory (with Gaussian noise of 0.16 variance), and a smoothed version of that data (in red). The middle row shows the distance to the closest lane, and the bottom row gives the cosine of the angle between the direction of flight and the lane direction. This distance and direction difference are NAB measures. For the distance measure, over 96% of the trajectory points are within 1 unit of the lane, and for the angle difference measure, 70% are within 10 degrees. The large angle differences arise at lane changes. Now consider rogue flights. Five categories are explored here: • Hobbyist Type I: Flies up from one place and makes a few moves above the launch site, then eventually lands at the same site. • Hobbyist Type II: Flies up from one place and makes a few moves above the launch site, hovers after each move, then eventually lands at the launch site. • Hobbyist Type III: Flies up in a circular motion to some highest point then flies down in a circular motion to land. • Rogue Type I: Flies up over and down as for a delivery. • Rogue Type II: Flies up to a lane, flies along the lane to the end, then flies to another lane (not necessarily connected), and eventually flies down to land. These anomalous flight patterns are representative of the types of flights to be expected. Figure 5.12 shows an example of these types. 5.2.1.2 Sensor Data to Track Rogue Flights Radar is used to scan the airspace lane network so as to make sure that scheduled flights adhere to their flight plans and to detect unscheduled flights. Here we provide an overview of the considerations necessary to employ radar sensors in airspace monitoring. Although the description addresses radar in the context of the simulation system, the actual exploitation of radar systems would require a similar analysis. In order to analyze flights in the simulation, there first needs to be a monitoring system that can detect UAS flights and report flight information back to the Air Traffic Operations Center (ATOC). The monitoring system that was selected was Radar. The structure and performance of the radar’s objects was to imitate real radar systems. One can think of the 73 system as the radar sending signals to detect objects over certain time intervals. If an object is detected, it will be reflected through broadcasting to those that are watching the scanner. This section will discuss the mechanism used to achieve this performance and structure. Figure 5.13 shows a single radar field in the grid layout lane-based simulation. One might notice that the radar’s detection field is conical in shape as the field progresses through the atmosphere, this is due to the natural occurrence of refraction. In real life this radar field would not contain distinct lines as a result of attenuation, meaning that the radar’s signal on the outer edges are weaker than those closer to the center. One can think of it similar to a flashlight shining in a dark room, where the center is the brightest and gets dimmer moving to the outer edges. However, in our simulation we excluded these weaker areas and created distinct edges for simpler computations which are discussed later in the chapter. 5.2.1.2.1 Radar placement. Radar systems can be selectively placed within city co- ordinates in order to provide the most coverage of a given air space region that surrounds the lanes. There is a lot of variability in the creation of the radar objects that can affect the coverage (see Figure 5.14). For example, some of these features are the maximum range of the radar field, the direction of the radar field, the beam angle, and the placement of the radar itself. Our goal is to find radar placements that provide the best coverage while minimizing the amount of radar field overlaps. First let’s make an assumption that the area we wish to cover is a cube that encapsulates the total area around the lane-structure. The reason we chose to maximize the radar fields’ coverage of a cube instead of the actual lane structure was because we wanted to ensure additional coverage beyond the lane system that would allow for the detection of rogue flights. In addition, turning the region of interest into a cube allows us to solve our problem by using packing optimizations for conical structures. The best solution would be to set the length of the radar’s field to the length of the cube. Then alternating the radar field direction in the vertical direction while alternating which edge of the cube to place the radar on, which is shown in Figure 5.15. However, this solution is not really practical for a real life scenario, because it has radar systems that are floating in the air. Unfortunately, it would be impractical to suspend radar systems in midair like this. A more practical solution is to mount the radar systems to other objects that are on the ground such as on 74 buildings, light posts, or the ground itself. Therefore, the arrangement of radar systems must also take into account their physical mounting requirements. The easiest solution would be to place radar systems away from each other at small increments. This would provide significant coverage, but would also come with a significant price tag if it were to be implemented in the real world. Our solution takes a different approach, instead of placing several radar systems on top of one another, we spaced them based on the height and radius of the radar fields. No matter how many lanes are stacked on top of one another, we decided to use the lower lane height as the point where the radar fields are okay to overlap, this ensures that the entire lane system will be covered. To find the distance between radar systems we take the radius of the radar field at the lower lane’s height and then double this radius. The final result is the distance between the radar systems. Figure 5.15 shows the basic parameters of a cone, and given that tan(θ ) = radius height (5.1) then we have: radius = height × tan(θ ) (5.2) Let’s shift the attention to how to measure the coverage of a set of radar systems. Getting an exact measurement is quite difficult, therefore, an approximation of coverage is calculated using Monte Carlo experiments. In these experiments, randomly generated samples, which are bound to the cube area around the lane system, will be used to count the number of points detected by the radar systems. To approximate the volume of coverage, we can simply divide the number of detected samples over the total samples in the experiment. It is through these experiments that we can determine the differing factors that help maximize coverage. Figure 5.16 shows the effects of changing the beam angle through the Monte Carlo experiments. Due to the conical shape of the radar fields, most of the coverage that is gained when using this method is in the upper altitude lane-structures; however, there are some regions that are left uncovered, which happens to correspond to areas that could contain launch and landing lanes, which must have coverage if this system is to function safely in the real world. There are two potential solutions: first, placing radar systems at every launch and land lane, or second, including more horizontally directed systems. 75 There will be a crossover point between placing radar systems at every launch and land site and simply placing horizontally directed radars. If there are a significant number of launch and landing sites relatively close to one another, there will also be more radar systems close to each other, leading to greater overlap between radar fields. In contrast, strategic placement of a few horizontally directed radar systems could cover a larger area while also reducing the amount of overlap between systems, meaning the number of radar systems needed will be lower to maintain the same amount of coverage. For example, consider the lane structure in Figure 5.16 with a radar angle of π 16 , it contains 6 launch and 6 land sites for a total of 12 sites that should be covered. Therefore, 12 additional radar systems would be needed to cover all of these sites using the first approach; whereas, the second approach would only need one horizontal radar system to cover the remaining area, if it were placed in the same location as the other original radar. Now consider how these radar systems detect objects during the simulation. When detecting objects, the main goal is to determine whether the object falls within at least one individual radar field. When determining if an object is within a radars field, one must consider a couple of factors (see Figure 5.17): distance away from the radar, the relative angle from the radar, and the radar’s maximum range. To simulate a more realistic scenario, every radar and object has to be looped over to check which object falls into which radar field. Therefore, the time complexity of this check would be O( RO), where R is the number of radar systems and O is the number of objects in flight during a time step. Therefore, since there could be a significant amount of radar systems and objects to detect, the calculation must be simple enough to ensure that the time complexity does not become excessive. To keep the time complexity manageable there are only two checks involved when determining if a given object is within an individual radar’s field. The first check is to see if the object is within the radar’s range, this is done by calculating the Euclidean distance from the object itself to the radar. The second check is to find the relative angle between the object and the middle of the radar field. For this two directional vectors are created: one to represent the radar field itself using the middle of the beam, and the other to represent the directional vector between the object and the radar itself. Then using the dot product to find the angle between the two directional vectors allows for the comparison between this angle and the radar beam angle. 76 During each simulation step, each radar object will receive a list of objects that are in flight. Then each radar will perform a detection step for each individual object. Each radar object contains a list that is clear at the the beginning of each simulation step, and then replenished with all of the objects that were detected. Once the radar object has gone through all of the objects in flight, it will then broadcast to any listeners that it has detected objects in the given simulation step. Any subscribers that are listening will have access to this list of objects, and they can proceed to handle the information according to their guidelines. Each simulation will contain an individual lane-system, radar system, number of flights, and an individual ATOC. As the simulation is running, information between all of these components are hard to capture until the entire simulation is completed. Therefore, each of these classes contain functions that allow you to gain some insights while the simulation is running in order to prevent a complete black box of the simulation process. The radar systems method is to display the region it covers, along with a red dot to represent the location of any detected object at that given simulation step. Figure 5.18 displays the graph. This is a useful tool for many reasons, one being able to target locations that might contain a higher density, or an area that potentially contains a lot of anomaly flight behaviors. The user can then see directly what is happening, and be able to make necessary changes if possible. In summary, the main goal for the radar systems in the simulation is to gather information about what is happening in the lane system and the surrounding area through scanning and reporting detections that should have associated telemetry. The structure and performance that was designed was intended to imitate actual radar systems. In addition, these radar systems are dynamic, meaning their locations and specific features can be changed in each simulation based on their designated purpose and the goal of the simulation. 5.2.2 NAB Analysis The two NAB measures given previously allow the discrimination of nominal from anomalous flight trajectories in almost all cases. This is due to the fact that anomalous flights, generally speaking, do not stay near the lanes nor do they fly in the same direction 77 as the nearest lane. However, trajectories (i.e., x,y,z,t 4-tuple sequences) are of variable length depending on the distance of the flight and the sampling rate. Thus, in order to compare trajectories, it may be necessary to normalize the length of each trajectory to some standard length. The nominal flights can be distinguished from the anomalous flights by means of a simple feed forward neural network. First, the trajectory lengths are normalized. Next, the NAB measures are computed at every point on the trajectory, and finally, the measures are concatenated into one vector (in this case, distance measure followed by cosine measure). A trajectory generator is created for each flight type based on random launch-land sites (uniformly selected over flight area), and appropriate parameters for the type of flight. Noise is added to the trajectory as follows (the same type of noise is added to all trajectories). First, the ideal trajectory is created. Then starting with the first point and moving to the second point, the error is defined by a circle around the goal point (the circle in the plane normal to the vector from the first point to the second point). A point in the circle (uniformly selected) is chosen as the target point. Next, a point on the line between the starting point and the circle point is chosen using a half Gaussian distribution centered at the circle point; this is the next point in the modified trajectory. When the circle has radius zero, and the Gaussian has zero mean and variance, then the resulting trajectory is the same as the original. A set of 100 sample trajectories was generated for each flight type, including nominal, for a total of 600 trajectories; half of these were used to train the network to classify nominal versus anomalous flights (two classes), and half were used to validate the result. Figure 5.19 shows the training performance (from Matlab) as well as the 100% correct classification results on the test set. Once an anomalous flight has been identified, it is possible to develop more refined and model-based techniques to distinguish between the sub-classes. Some characteristics of anomalous flights are: • Hobbyist Type I: not on lanes, not in correct direction, change of altitude in nonvertical direction, launches and lands near same site. • Hobbyist Type II: not on lanes, not in correct direction, change of altitude in nonvertical direction, launches and lands near same site, hovers for short periods of 78 time. • Hobbyist Type III: not on lanes, not in correct direction, change of altitude in nonvertical direction, launches and lands near same site, makes circular motion. • Rogue Type I: not on lanes, only goes up, over and down, middle segment may not align with lane, may not be at normal lane altitude, launch and land sites may not be near lanes. • Rogue Type II: not on lanes some of the time, not in correct direction some of the time, lanes followed may not be connected in lane network, some changes of altitude not vertical. These characteristics are used to develop models of the various trajectories, and a classifier built based on them. Using the same set of simulated trajectories already described, the classification confusion matrix is given in Table 5.1 are achieved. From these results it can be seen that the trajectories of the Hobbyist Type I and the Rogue Type II are similar and require further refinement for discrimination. Now, suppose that there is another type of flight, for example, an amateur just flying circles above some location in the city. Figure 5.20 shows an example trajectory with respect to the ground, and Figure 5.21 shows it with respect to the airways. Figure 5.22 shows the distance measure for the amateur flight compared to the noisy data from a nominal scheduled flight. It is clear that the error for such an unscheduled flight is much larger, and that flying in circles introduces a noticable frequency component to the distance signature. Next, consider the trajectory direction measure. Figure 5.23 shows the signatures for the noisy trajectories of the regular flight. Although it is difficult to see, the mean angle for these are: 26, 31, and 59 degrees for variances of 0.1, 1, and 10, respectively. Now consider the signature for the amateur flight (see Figure 5.24). The values are all close to 0 indicating that the direction is usually perpendicular to the lane direction. This make sense in that for the first few hundred feet, the lane is directed up while the circular flight is mainly in the x or y direction. This particular amateur flight is always near a single lane. Figure 5.25 shows the distance signature from a more densely packed lane area as shown in Figure 5.26, and a similar oscillating distance is observed. Next, consider a rogue flight that simply takes 79 the shortest route between launch and land sites (see Figure 5.27). Figures 5.28 and 5.29 show the distance and direction errors signatures, respectively. 5.3 Next Steps The lane-based UAS traffic management approach supports efficient and effective trajectory analysis of UAS flights in the airspace. This allows the straightforward detection of unplanned flights through the airspace without having to compare to every existing flight at the time of occurrence. In addition, it is possible to distinguish different types of rogue flights according to the trajectory distance and direction measures. Several new avenues of investigation are under consideration: • model updates due to dynamic lane creation and deletion, • how to exploit knowledge of UTM parameters (e.g., UAS speed limits, lane network topology, 3D corridor constraints, etc.), • any influence on trajectory measures due to weather, congestion, or other environmental or contingency effects, • the constraints on sensor data to ensure effective identification of anomalous flight patterns, and • the role of communications in UAS flight trajectory analysis. 80 Figure 5.1: Lane-based airway lanes above downtown San Francisco, CA. Figure 5.2: ATOC business plan (from [2]). 81 Figure 5.3: Radar and other sensing capabilities proposed by UDOT. Figure 5.4: GPS coverage and example hardware. 82 Figure 5.5: ATMS and fiber network for Salt Lake Valley airway corridor. Figure 5.6: Proposed airspace layout for the Salt Lake Valley airway corridor. 83 Figure 5.7: Set of airway lanes created over Salt Lake City, UT. Figure 5.8: Trajectory point set model of airway lanes over East Bench of Salt Lake City, UT. Red circles are lane endpoints; blue points are samples along lane. 84 Figure 5.9: Trajectory direction vector model of airway lanes over East Bench of Salt Lake City, UT. Figure 5.10: The Nominal versus Anomalous Behavior (NAB) method. 85 Figure 5.11: NAB measures for a nominal flight. 86 Figure 5.12: Examples of the five anomalous flight trajectories. Figure 5.13: Demonstration for how a vertical radar field could be placed in a grid layout lane system simulation. 87 Figure 5.14: Displays a couple of different packing arrangements for conical shapes. The image on the left is the optimal conical packing but not the most realistic solution. The middle image is the easiest solution to optimize coverage but it is too costly of a solution to be implemented. The rightmost image is our solution, which provides a high percentage of coverage with minimal overlaps. Figure 5.15: The basic components of a conical shape; these are used to establish the required distance between radar placements. 88 Figure 5.16: Difference comparison between varying beam angles in the Monte Carlo experiment of 100,000 randomly generated sample points. This displays the inverse relationship between the angle size and the number of radar systems that would be needed for coverage. The image on the left uses a significantly smaller angle, resulting in more radars to cover the given area; whereas, the image on the right uses a larger angle of detection. The green dots represent the sample points that were within the radar fields. The empty spaces are areas in the lane-system that are not covered by the radar systems that have been placed. Figure 5.17: Properties that are used to determine if an object is within the radar’s field of view. These properties include the radar’s maximum range, and the angle between the object and the radar. The left drawing shows a vertical radar field, and the right drawing shows a more horizontal radar field. 89 Figure 5.18: Visualization of the radar detection process during a simulation. The colorful circle represents the radar field, and the red dot represent the location of the detected object within the field. The first number in the title represents the time step and is followed by the radar identification number. Figure 5.19: Results of feed forward neural net classification of flight trajectories into two classes: nominal (first fifty) and anomalous (remaining 250). On the left is the network learning performance data, and on the right is the classification result. 90 Figure 5.20: Amateur trajectory flying in circles with respect to the ground. Figure 5.21: Amateur flight with respect to airway network. 91 Figure 5.22: Amateur flight distance signature compared to those of noisy flights. Figure 5.23: Noisy flights vector cosine wrt lanes. 92 Figure 5.24: Amateur flight vector cosine wrt lanes. Figure 5.25: Amateur flight distance signature in a more dense lane area. 93 Figure 5.26: Amateur flight location in a more dense lane area. Figure 5.27: Example rogue flight path. 94 Figure 5.28: Rogue flight distance error. Figure 5.29: Rogue flight direction error. 95 Table 5.1: Classification results. The 6 trajectory types are (1) nominal, (2) hobbyist type I, (3) hobbyist type II, (4) hobbyist type III, (5) rogue type I, and (6) rogue type II. 1 2 3 4 5 6 1 100 0 0 0 0 0 2 0 92 0 4 0 14 3 0 0 100 0 0 0 4 0 3 0 83 0 0 5 0 2 0 10 100 0 6 0 3 0 3 0 86 CHAPTER 6 UAS BELIEF-DESIRE-INTENTION AGENT ARCHITECTURE The coming wave of autonomous, semi-autonomous, and human-operated vehicles in low-altitude airspace over densely populated areas requires a new system to automate air traffic control (ATC). The current system relies heavily on the intuition of human pilots and controllers who benefit from over one-hundred years of recorded trials and errors. Even after millions of test cases per year, contingencies occur that confound experts and result in disastrous outcomes (e.g., the failure of coördination that resulted in a mid-air collision over Uberlingen, Germany in 2002 [63]). The computational intractability of enumerating all possible sequences of actions and outcomes that lead to contingencies is the root-cause for these disasters; if engineers had known the Uberlingen scenario was possible they would have avoided it (after the accident, TCAS software was patched to handle it, albeit at great expense due to the complexity of the software and subsequent testing [64]). However, the density and dynamism of the anticipated low-altitude air traffic mandates an automated approach; it is difficult to imagine human controllers managing the separation of thousands of flights per hour. This is the conclusion of most professionals in aerospace across the United States, particularly the Federal Aviation Administration (FAA) and the National Aeronautics and Space Administration (NASA), as well as in Europe and Asia, where UAS Traffic Management systems are being developed rapidly. However, if current ATC systems still experience contingencies after a hundred years, and millions of flights per year, what hope do engineers have in constructing a safe automated traffic management system? This question lies at the heart of this research, and it not only applies to the safe coördinated access of airspace (i.e., maintaining safe separation between aircraft), but also to a plethora of other issues that air traffic controllers face. The issues that require human intervention, and that make experts nervous about 97 automated air traffic systems, are typically contingency scenarios. Situations that require safe-separation and unplanned priority, for example medical rescue, temporary flight restrictions (TFR) due to wildfires, low-fuel, electronics failure, etc. The scenarios, and the combinations or sequences of events that lead to them, are difficult to enumerate, and therefore difficult to plan for. They are difficult because in a concrete sense, the determination of whether a solution exists to return to a nominal state may be as difficult as, or likely more difficult than, the Satisfiability (SAT) problem. In the language of computer science, the public relies on pilots and controllers to heuristically search for solutions that maintain our safety. It is likely that humans perform these searches at a high-level, using abstraction (and perhaps analogies) to reduce the space of possible solutions. So far, no one has shown that the brain, or any part of the nervous system routinely and exactly solves NP-complete problems [65]. For this reason, the problem of air traffic control, when considering the automation of what human operators are currently responsible for, falls squarely within the purview of computer science. This problem is fundamentally an issue of cognition and computation. Considering the cognitive and computational nature of the UTM problem, a good strategy for constructing a system to replace human pilots and controllers becomes clear: reduce computational complexity on all fronts. A direct effect of this strategy is the reduction of the number of possible contingencies because by definition there are fewer states to consider (and by implication, fewer undesirable states). It is our contention that this strategy should be executed via two channels. First is through structure and deconfliction; since safe separation is a constraint that must be satisfied in any contingency scenario, it serves to reason that this problem should be easy to solve, and hence low complexity. This is the foundation that the lane-based approach provides. Second, complexity in cognition should be reduced via abstraction, which we explore using agent based modeling and simulation (ABMS), and the Belief-Desire-Intention (BDI) architecture. With this strategy, we expect that the resulting system design will be more robust in the face of contingencies than anything else currently proposed. 98 6.1 Knowledge Representation and Inference A UAS agent must analyze a combination of heterogeneous information expressed in logical form (as sentences or statements), computational form (as numerical models of physics or other processes), and sensor data (as measurements from transducers). Each of these forms has its own way to describe uncertainty or error: e.g., frequency models, algorithmic truncation, floating point round-off error, Gaussian distributions, etc. We use our Probabilistic Sentence Satisfiability (PSSAT) method (called NILS) which receives information as logical sentences from humans, simulations (e.g., weather or environmental predictions), and sensors (e.g. cameras, weather stations, microphones, etc.), where each piece of information has an associated uncertainty; NILS then provides responses to user queries based on PSSAT which determines a coherent overall response to the query and the probability of that response; this new method avoids the exponential complexity of previous approaches. In addition, NILS can be used to identify concrete mechanisms (proposed actions) to acquire new data dynamically in order to reduce the uncertainty of the query response. The basis for this is a novel approach to probabilistic argumentation analysis [66]1 . Most knowledge-based systems do not incorporate a broad notion of uncertainty quantification (UQ), although such a capability would allow a UAS to make more informed decisions, or to acquire more data before coming to conclusions. In addition, it would be better if system responses provided an explanation of how they were derived, as well as how the uncertainty was determined. This can be the result of sensor error, computational error, human error, etc., and the best models should be selected at each time step in order to reduce the variance on quantities of interest. In addition, UAS operating in a lane-based UTM should generate dynamic path planning solutions which can include constraints on time, energy, or uncertainty reduction. The automatic generation of constraints arising from the various models can be used to inform the deployment of data measurement systems. The application studied here is UAV (Unmanned Aerial Vehicle) surveillance and reconnaissance in urban areas. Some work has been done in this general area (e.g., see [67] for a novel guidance law in windy urban environments combining pursuit and 1 This material is based upon work supported by the Air Force Office of Scientific Research under award number FA9550-17-1-0077 (DDDAS Geospatial Intelligence.) 99 line-of-sight laws, and [68] for a multi-cost UAV mission path planner). We describe here two major novel research results: (1) the combination of formal probabilistic logic methods with state-of-the-art physics-based uncertainty quantification methods, and (2) uncertainty driven active information data acquisition, demonstrated by UAV path planning, to optimize performance or to resolve contradictory information. The probabilistic logic method is a re-formulation of the approach described in [69] (although Boole [70] first proposed it); see [71] for details. Basically, Nilsson’s method requires first solving the SAT problem (i.e., find all consistent truth assignments) to set up a set of linear equations, then the use of numerical methods to solve them. We, on the other hand, create a set of nonlinear equations, and solve them directly [71]. We have previously applied this framework to geospatial intelligence systems in an implementation called BRECCIA [72]. BRECCIA is designed using a well documented multi-agent, Belief-Desire-Intention (BDI) framework called Jason [73]. Jason includes an interpreter for an extended version of the AgentSpeak(L) [74] language, which provides a Prolog-like grammar. Both the style, resembling a natural language application, and the operational semantics of the extended language that enable a data-driven architecture, fit well with the proactive and reactive goals of BRECCIA. Each agent in the BRECCIA system is composed of a backward chaining inference module (see [75] for a formal justification of modularity in BDI programming languages) with a probabilistic logic component. This module, and probabilistic component, form the most abstract fusion implementation in the BRECCIA system. To elaborate, this system may include many functional fusion modules that agents specialize for a particular application. For example, an agent that specializes in managing a particular unmanned air vehicle (UAV) requires a fusion module for estimating target detection. However, the human agent, a “user” in BRECCIA, requires information in more abstract terms to support the decision process. For example, a user may query the system by asking for the probability of mission success. Among the many bits of information that support this assertion, with their associated probabilities. Once the list of probabilities has compiled, the non-linear probabilistic logic algorithm is executed to calculate the probability for the inferred belief. Finally the inferred belief is added to the agent’s knowledge base. The process of belief-revision (when probabilities of antecedents change) currently utilizes the same algorithm, except first a list of justifications, stored 100 in the belief annotations, is compiled until the most abstract inference is located. Beliefrevision in the BRECCIA system is an active area of research. 6.1.1 Probabilistic Logic Here we address the problem of finding a suitable representation for uncertainty associated with logical sentences. Although several approaches have been proposed in the past (see [76–83]), they generally have some significant drawbacks. Usually, these have to do with the computational complexity of the semantics of the sentences (i.e., finding the set of consistent truth assignments is exponential in the number of sentences, or for Domingos, exponential in the number of cliques in the Markov graph [84]). We have developed a new approach which computes the probabilities of the atoms in the sentences, and in terms of these, provides a solution for Pr ( Q | KB), where Q is the query and KB is the knowledge base set of sentences (see [71] for details). Moreover, the knowledge of the probabilities of the atoms allows us to determine where the most uncertain part of the argument lies, and to allocate resources to lower that uncertainty, thus decreasing the uncertainty of the query. This is done by exploiting the probability of a disjunctive clause, and developing a set of equations from the sentences and their probabilities, and then solving those equations (the number of equations equals the number of sentences). Our approach to probabilistic logic starts with an analysis of Nilsson’s method [69]2 . Given a set of n sentences (conjunctions from a Conjunctive Normal Form well-formed formula), S = {S1 , S2 , . . . , Sn }, in the propositional calculus, where {S1 , . . . , Sn−1 } is the KB and Sn is the query, he first finds the set of models of the sentences (i.e., the set of truth value assignments to the sentences that are consistent using the general semantic tree [86] for a set of sentences). In our new approach [71], we trade the exponential complexity of most other algorithms for the convergence cost of solving for the logical variable (atom) probabilities directly as follows. First, we assume that the sentences are given in conjunctive normal form. This means that each sentence is a disjunction of literals (a literal is an atom or its negation). Our 2 Note that Nilsson’s method for propositional calculus is the same as that proposed by George Boole in the 1800’s [70, 85]. 101 second assumption is that Pr ( P ∧ Q) = Pr ( P) Pr ( Q)(i.e., they are independent variables); note that if this assumption is violated, our methods also allow the bounds on the probability to be determined. Next, we find the set of logical atoms (i.e., variables) in S; let A = { A1 , A2 , . . . , Ak } be this set. In this case the probability of a sentence can be computed from the probability of its literals as follows: Pr ( L1 ∨ L2 ∨ . . . ∨ L p ) = Pr ( L1 ) + Pr ( L2 ∨ . . . ∨ L p ) − Pr ( L1 ) Pr ( L2 ∨ . . . ∨ L p ) (6.1) where the probabilities of clauses on the right hand side are computed recursively. Assuming that the logical (random) variables are independent, each sentence gives rise to a (usually) nonlinear equation defined by the recursive probability of the disjunctive clause as defined above. The resulting set of equations can be solved using standard nonlinear solvers (e.g., fsolve in Matlab), and a set of consistent values for the probabilities of the atoms determined. Of course, one problem with the nonlinear solver approach is that it may not find a solution, even when one or more exist. Thus, our current approach is to solve all equations that have a single unknown (recursively), and then use an iterative method to find a set of atom probabilities which produce the correct sentence probabilities. 6.2 Reinforcement Learning Reinforcement learning provides engineers with a tool for generating complex decision making programs from high-level requirements. Traditionally, engineers receive high-level requirements, such as “UAS should avoid rain,” then proceed to define all the behavioral rules necessary to fulfill that goal. The effort required to define this program logic is a complex function of the software technology, development cycle, and requirements. Estimating the effort required is itself a large topic of software-engineering research [87], and changes to requirements or logic errors can have dramatic costs. Reinforcement learning, however, does not require a manual development of the program logic for the desired behavior. It does require a careful definition of possible states, available actions, and rewards, but it is not necessary to consider all the possible combinations of states and actions. Consider a scenario in which ground infrastructure supporting UAS communications is disrupted during normal operations (see Figure 6.1). Currently, the published protocol for handling this contingency is for every UAS to fly back to base if communications cannot 102 be re-established within a given amount of time [88]. Since this is a pre-defined policy, it is worth considering whether such a policy is robust. For example, depending on how many UAS communications are disrupted, the number of conflicts that result from the simultaneous replanning of multiple agents may have negative cascading effects [42]. As the complexity of the UTM system increases, it becomes harder for experts to enumerate all the failure modes and effects; assigning liability and performing post-failure diagnoses will also be difficult. A comparison between Traffic Alert and Collision Avoidance System (TCAS), which is currently in widespread use by airlines, and a new system called the Airborne Collision Avoidance System X (ACAS X) offers a compelling analogy. TCAS has been described as an “ad hoc rule-based specification” [64]. Limits to its robustness arise primarily because programmers are unable to anticipate the spectrum of operational scenarios, one of which caused a collision over Uberlingen, Germany in 2002. ACAS X, in contrast, adopts a process of modeling and optimization that improves robustness. Kochenderfer describes an early prototype of ACAS X, in which the anti-collision problem is formulated as a partially observable Markov decision process (POMDP) [64]. In this way, collision and alert preferences are treated as inputs, and the system logic as an output. 6.2.1 Complexity and Cognitive Structure When the number of possible states and actions is large it forces engineers to carefully create abstractions of states and actions (e.g., object-oriented software), otherwise the program logic becomes fragmented across a large and flat organizational structure. A fragmented program is undesirable because the conceptual links between a high-level requirement and low-level actions are buried in the program logic. On the other hand, a hierarchical structure of states and actions encodes conceptual links explicitly and enables efficient searching through a categorical index (e.g., Desires in the BDI architecture). The Belief-Desire-Intention architecture [89] is a hierarchical organization of states and actions (grouped into plans) that was designed specifically for agent models. The architecture not only defines the conceptual structure of a program, but also a process structure that enables dynamic planning, similar to a Markov Decision Process (MDP) ([90–93]). Organizing the program in this way supports both reasoning by the autonomous agent as 103 well as reasoning by human operators. The structure of desires, intentions, beliefs, and plans coincides well with the reasoning of the human operator. For example, a human observer could ask a BDI agent directly, “What is your current plan?”, and the agent could respond, “Correcting my heading to get back in the lane.” 6.2.2 Complexity and Airspace Structure In previous work, we outlined the structure and analytical capabilities made available by the lane-based airspace structure [11, 38]. From a cognitive perspective, the environment in which autonomous agents operate is dynamic and uncertain, and agents are resource-constrained and have only a local view of the world. The lane-based structure provides autonomous agents with more information about the state of the airspace, while requiring less computational effort to reason about it. The primary basis for this is that agents only need to consider the schedule on each lane, as opposed to the entire trajectory of other aircraft in a free-flight airspace structure. In the free-flight model, where any trajectory is allowed, agents must sample other trajectories and their own to ensure safe separation. The sampling resolution necessary to ensure safe separation, i.e., the discretization of trajectories, depends on how trajectories are specified and the available time and resources to perform the sampling. Each agent must perform this computation every time it considers a new or altered path. The lanebased approach, however, pre-calculates safe-separation in the spatial domain, so agents only need to consider the schedule. 6.2.3 Learning and BDI UAS agents have a Belief-Desire-Intention architecture which functions as shown in Figure 6.2a. The BDI cycle involves updating the beliefs, determining the desires, choosing an intention (goal), and then selecting an appropriate plan to achieve that goal. A belief is represented as a disjunction of logical variable literals, and the entire belief set is a conjunction of such beliefs (in Conjunctive Normal Form (CNF)). A desire is a belief that the agent would like to make true, and an intention is a belief that is a current goal (of which only one is active at a time). The selection of a plan is the action at the cognitive level, and optimal cognitive policies pick the best plan for a given state. Focus 1 in Figure 6.2b is where the cognitive learning takes place; i.e., the actions are a set of possible plans to 104 achieve a specific goal, and an optimal policy chooses the best plan for a given state. The selected plan is then executed until either completion or preemption. Focus 2 concerns policies at the physical level (i.e., moving through space). For example, given cognitive states S = {S1 ≡ GoToDestination − NoDri f t, S2 ≡ GoToDestination − Dri f t, S3 ≡ Fail, S4 ≡ Succeed}, where there may be several plans to achieve a goal (e.g., shorter or safer routes), and the action is to select one of these plans as shown in Figure 6.3. Rewards are associated with states and actions, and reinforcement learning is applied to find optimal policies. This is done over a large number of environmental and UTM conditions. Actions are parameterized by considerations like estimated required time, risk, communications connectivity, etc. At the physical level, a plan may consist of a sequence of lanes with associated entryexit times. Alternatively, a plan may consist of a sequence of GPS waypoints and times. We have already performed a preliminary study of this aspect of UAS plan optimization (see [38]), and shown how optimal policies (for moving through space) can be determined in the context of environmental conditions (e.g., wind). Cognitive-level reinforcement learning follows the same process as traditional reinforcement learning. 6.2.4 6.2.4.1 Experiments Learning at a Low Level The goal here is to determine an optimal action selection policy for a UAV with a given destination goal and a set of specific environmental conditions which defines the state space. The agent must operate successfully in this environment by learning a utility function on the state of the world and from those utilities determine an optimal action policy for each state. We consider an agent in a fully observable environment. Once a policy, π, is learned to maximize utility, U (s), then it deterministically specifies the action for each state, and the agent will always choose action π (s) for state s. The goal is to maximize the expected utility (see [94] for details). The utility for each state is defined by the Bellman equation: U (s) = R(s) + γmax a∈ A(s) ∑ P(s0 | s, a)U (s0 ) (6.2) where U (s) is the utility of state s, a is an action, A(s) is the set of actions possible for state s, and P is the probability of state s0 given state s and action a. We use value iteration to 105 solve for the state utilities; i.e., the above equation is iterated, updating the utility of each state until convergence is achieved. Once the utilities are known, the optimal policy at each state corresponds to the action which maximizes the expected utility from the action: π ∗ (s) = argmax a∈ A(s) ∑ P(s0 | s, a)U (s0 ) (6.3) s0 6.2.4.1.1 State representation. In order to solve for the optimal policy for UAV con- trol, we define the state space as: S = Z 3 × <3 × < + × < (6.4) where the space is composed of three integer grid coördinates, a real-valued 3D wind vector (whose magnitude is the wind speed), a precipitation value, and a temperature value. Note that although the wind, precipitation and temperature values have different dimensions, their values are represented by indexes that designate intervals in the appropriate range. For the study here, the grid consists of a 4x4x4 set of voxels (representing airspace volumes where the specific dimensions of the air volumes are determined by the problem under consideration; here we assume reasonably large volumes), 2 values are used to indicate the wind (none, high), precipitation is binary (raining or not), and 3 values for temperature (cold, normal, hot). Thus, the state vector is a 6-tuple, where the number of values for each element is [4,4,4,2,2,3], resulting in a total of 768 distinct states. 6.2.4.1.2 Possible actions. The action set for this problem is simply the selection of a neighboring air volume; in particular, one of the 6 orthogonal direction cells. These actions will be labeled { X, − X, Y, −Y, Z, − Z }, so that these actions align with a standard frame in the center of the cell (see Figure 6.4). 6.2.4.1.3 State transition function. Next, a probabilistic state transition function, P(s0 |s, a), is defined which provides the probability that state s0 results from choosing action a while in state s. The particular function used here is based on the physics of the state transitions, accounting for the impact of motion based on wind, temperature and precipitation. 6.2.4.1.4 Reward function. Finally, a reward function is defined as follows:   −0.04 s 6= goal,excluded state −1 excluded state R(s) =  +1 goal state (6.5) 106 The goal state has grid location [4,4,4] and reward value 1, while excluded cells have a value of -1. 6.2.4.1.5 Results. In order to better understand the method, a specific example will be considered here. First, a 4x4x4 grid of 64 air volumes as shown in Figure 6.5 will be indexed by either their grid coördinates or by a simple index. E.g., cell [3,2,4] will also be identified by the index 31. In addition to the grid location, each volume also has temperature, wind and precipitation information. This latter aspect will be considered below. The actions available are directly tied to moving to one of the neighboring (closest) six-neighbors. Figure 6.5 shows the states. Note that when a cell is on the boundary, then the UAV is not allowed to exit the 4x4x4 grid, and so if that direction is chosen, the UAV will remain in the same cell with some probability. In order to solve the value iteration problem, the neighbors of each cell in each action direction is first determined. A few of these are given in Table 6.1. Next it is necessary to define the probability of moving into each neighboring cell given the desired action. This is provided in terms of Table 6.2. Note that we assume it is more likely that motions in the X-Y plane will have a certain probability, and that moving up is more uncertain than moving down. Although we have assigned likely values here, these are also parameters that may be learned over time. Given this information, it is possible to run the value iteration algorithm and find the utilities for the states. Figure 6.6 shows the utilities produced at each state as well as the path through the highest probability sequence of states. Note that this may not match the optimal actions selected since it does not take into account the maximal expected utility of the action. It does show however, that the UAS will most likely move up and then over in such a way as to avoid the excluded air volume (index 60, cell [4,4,3]). Figure 6.7 shows how the utility values converge for this problem. An optimal policy can then be determined for each state and produces the result shown in Table 6.3. These results are also shown in Figure 6.8. Note that the red arrows indicate a move in the Y direction. Of course, the actions are not deterministic, and in order to better understand the impact of this policy, 1000 trials were run with start location cell [1,1,1], index 1, and with 107 goal location cell [4,4,4], index 64. One cell is excluded, cell [4,4,3], index 60; if the UAS enters that cell it must land and terminates the mission. Figure 6.9 shows number of times each cell in the airspace was traversed by the 1000 trials. As can be seen, much information may be gleaned from these results as to the probability that a UAS will be in the assigned air volume (we have not included temporal aspects, but that is readily available, if desired). Also, note that a half a percent of the trials resulted in failure. Next, consider the impact of a stiff wind blowing in the Y axis direction. This information is easily added to the model by simply providing the state transition probability for motions impacted by the wind. This is called context-based probabilistic state transition. In the case of a strong wind in the Y axis direction, say produced by afternoon canyon winds in Salt Lake City, a set of state-action probabilities are provided (either by learning over time, or by physics-based simulation). Figure 6.10 shows the values used here. Running value iteration with these probabilities gives rise to the convergence values shown in Figure 6.11. The policy determined for these utilities is shown in Figure 6.12. Some interesting observations may be made. For example, the policy never chooses a Y-axis action. Figure 6.13 shows the number of times each air space volume is traversed over 1000 trials. Note that there are a significantly higher number of failures (39) due to the strong wind. Of course, this policy is the result of a certain level of tolerance for failure vs energy expenditure. It is possible to vary these parameters and produce policies less likely to fail by entering excluded zones. Finally, it is important to note that the method can be considered having a learning part and an application part. 6.2.4.2 Learning at a Cognitive Level To demonstrate the BDI reinforcement learning at the cognitive level, a simulation was run to determine the optimal policies for UAS agents in an environment that contains rain and wind. The cognitive state of each UAS before the simulation begins is represented by a knowledge-base containing the clauses in Table 6.4. The initial goal is for the UAS to be assigned a mission, so the intention stack is started with the desire to be ASSIGNED. Once assigned, the UAS selects intentions from the precedence list shown in Table 6.5 (i.e., after being assigned, the next desire is to remain IN LANE). A combined state and sequence diagram outlining a single reasoning cycle is shown in 108 Figure 6.14. When the UAS agent receives a percept from the simulator, containing both state information as well as messages from other agents, it parses the data and updates its knowledge-base. In the Analyze state, the agent considers its desires and updates its current intention. The Filter state involves selecting the optimal plan for the current intention, and in Execute Plan the required low-level actions are taken. The process for generating and assigning flights follows the general design proposed by NASA where a UAS Service Supplier (USS) is responsible for deconflicting flights with other USS in the system. To accommodate the ABMS setup, the USS is also responsible for generating random flight requests and auctioning them to UAS agents. This process is diagrammed in Figure 6.15. 6.2.4.3 Environment Model The environment used to train a policy is shown in Figure 6.16, where solid circles mark areas of rain and dashed circles mark areas of wind. Rain is modeled as a scalar intensity value that decreases with distance from the center of the feature. Rain affects the speed of UAS proportional to the amount of rain. Wind is similarly defined, except it affects both speed and heading of a UAS in a direction tangential to the radius of the wind feature. 6.2.4.4 Actions UAS agents in this simulated framework have the ability to set their velocity after each reasoning cycle. In a scenario without wind or rain, the desires are selected in order of precedence (lower precedence happens first). However, due to the dynamics in the environment brought in by wind and rain, agents are affixed with logic that requires replanning when the situation is not NOMINAL. In this case, the agent must choose a contingency plan that returns the cognitive state to NOMINAL. There are a number of plans that may achieve this, given in Table 6.6. The selection of plans is determined by a policy obtained through reinforcement learning. 6.2.4.5 Transition Probabilities and Rewards To generate the transition probabilities, a Monte Carlo simulation was run and a threedimensional state-action transition matrix, representing the probability P(s0 |s, a), was generated from the data. The reward model, R(s, a), considers only the eight states generated 109 by the Cartesian product of NOMINAL, RAIN, and WIND, and the four high-level plans in Table 6.6. A state reward (Rs ) of +6 was assigned to any state that was NOMINAL, and −2 for any state that was not. Plan rewards were set as follows (reflecting their cost to execute): R a (FOLLOW LANE) = −1 R a (CORRECT SPEED) = −3 R a (CORRECT HEADING) = −5 R a (GO TO LANE) = −8 6.2.5 Policy Selection A policy was selected by running value iteration and generating state utilities. A trace of the state utilities after each iteration (Eq. 6.2) is shown in Figure 6.17. A slice of the transition probability matrix for the plan FOLLOW LANE is depicted by the digraph in Figure 6.18. The UAS agent then combines the state utilities and the transition probabilities using Equation 6.3 to select the optimal plan. A graphical depiction of the behavior of a single UAS mission without rain or wind contingencies is shown in the plan and state graph in Figure 6.19. Figure 6.20 shows the behavior of a UAS navigating the same trajectory through a rain feature with a learned policy. Finally, in Figure 6.21, a behavior trace of a UAS navigating the rain feature using a programmed policy that deals with the rain contingency directly by correcting its speed. 6.2.6 Discussion The experiments demonstrate that reinforcement learning at the cognitive level is a viable option for programming agents in a UTM system. The program in this instance was comprised of a number of plans that could be engineered independently, in contrast to the currently proposed strategy of enumerating risk factors and developing contingency plans in concert across the industry. In this simple experiment, the resulting policy is guaranteed optimal with respect to the rewards because dynamic programming was used. In a large-scale system, the number of possible states and actions may be too large to pre-calculate utilities. However, a plan can be engineered in which the UAS agent performs dynamic programming over a narrowed 110 set of states provided by the Analyze step in the BDI architecture. Since the BDI architecture supports replanning when a plan fails or when desires change, the agent can avoid overcommitting to a plan that did not consider a particular contingency. This fact is demonstrated in the experimental output of Figures 6.19, 6.20, and 6.21. The explicitly programmed policy is brittle in the face of a rain contingency because the speed correction causes the UAS to overshoot lane waypoints. The learned policy selected a different plan, one that corrects heading and speed concurrently, and proves to be more robust. 6.2.7 Conclusion and Future Research We have shown how states and actions at the cognitive level can be combined with reinforcement learning to generate optimal policies for UAS agents. Additionally, the BDI architecture provides a convenient structure for defining high-level states and actions, while reducing the engineering complexity by allowing plans to be designed independently. The benefit of this approach is that it reduces the amount of programming logic required to build robust policies, and utilizes a logic structure that supports operational insight since decisions are expressed in a human-understandable form. In upcoming research, our intention is to demonstrate the performance of the combined cognitive-level reinforcement learning BDI architecture with respect to contingency scenarios where a large number of states exist. In these scenarios the agent must replan when optimal plans do not produce the correct outcome due to unforeseen states. Also, the effects of replanning and contingency handling on the aggregate state of all UAS agents in a UTM system must be understood. If it can be shown that the best available policy for UAS agents includes dynamic replanning and the proposed cognitive structure, then this strategy will enable a more rapid adoption of autonomous agents due to the decreased engineering complexity. 111 Figure 6.1: Example scenario: ground communication disrupted for multiple UAS. Figure 6.2: (a) BDI architecture (taken from [3]). (b) Reinforcement learning Focus 1: cognitive level plan selection to achieve goal, and Focus 2: actions in the individual plan (this figure is adapted from [4]). 112 Figure 6.3: MDP representation of BDI architecture (this figure was adapted from [5].) Figure 6.4: Actions for UAS low-level reinforcement learning. 113 Figure 6.5: States for UAS low-level reinforcement learning. Figure 6.6: The state utilities and path through the highest utility states. 114 Figure 6.7: The convergence of the state utilities. Figure 6.8: Optimal policies for the states (pictorially). 115 Figure 6.9: The number of times each cell is traversed in 1000 trials. Figure 6.10: The context-based probabilistic state transition probabilities for the case of a strong wind in the Y axis direction. 116 Figure 6.11: The convergence of value iteration with the context-based method. Figure 6.12: The policy produced by the context-based state transition method. 117 Figure 6.13: The number of times each space volume is traversed over 1000 trials. Figure 6.14: Combined sequence and state diagram showing agent architecture. 118 Figure 6.15: Flight path generation and assignment. Figure 6.16: Wind and rain placement for training. 119 Figure 6.17: Value iteration trace. N-Nominal, W-Wind, R-Rain. Figure 6.18: Transition probabilities for the Follow Lane plan. 120 Figure 6.19: Nominal behavior without contingencies. Figure 6.20: Learned policy with rain contingency. 121 Figure 6.21: Programmed policy with rain contingency. 122 Table 6.1: State transitions for UAS low-level reinforcement learning. Table 6.2: State transition probabilities for UAS low-level reinforcement learning. Table 6.3: Optimal policies for the states. 123 Table 6.4: UAS KB at initialization. ID 1 2 3 4 5 6 7 8 9 10 11 Clause IN LANE ∧ ON HEADING ∧ SPEED OK → NOMINAL LAST LANE ∧ AT NEXT WAYPT → AT FINISH ¬IN LANE ¬ON HEADING ¬SPEED OK ¬ASSIGNED ¬IN FLIGHT ¬AT START ¬AT NEXT WAYPT ¬ADVANCE LANE ¬WRAP UP Table 6.5: UAS desires and precedence. Desire ASSIGNED IN LANE ON HEADING SPEED OK AT NEXT WAYPT ADVANCE LANE WRAP UP Precedence 10 20 30 40 50 60 70 Table 6.6: UAS high-level plans. Plan CORRECT HEADING CORRECT SPEED FOLLOW LANE GO TO LANE Description Heading Optimized Controller Speed Optimized Controller Main Lane Following Control Take Immediate Action and Fly to Lane Segment CHAPTER 7 CONTINGENCY HANDLING NASA engineers have published a number of system requirements in an effort to enable dense operations of Unmanned Aircraft Systems (UAS) in urban environments [95, 96]. These requirements describe a free-flight model, where operators are afforded the maximum flexibility to design individually optimal trajectories, with the caveat that all operations must be strategically deconflicted prior to flight. Strategic deconfliction reduces the probability of having to perform tactical deconflicton using onboard sensors and real-time algorithms. Such approaches require a common protocol to guarantee that UAS do not collide, but do not scale well. Thus, UAS Service Suppliers (USS) must deconflict their planned trajectories pairwise prior to flight in order to achieve strategic deconfliction. However, sometimes flights are not able to follow their plan and must handle some contingency; a contingency is a future possible event, usually causing problems or making further plans necessary. We propose a communication-based protocol to coördinate airspace during flight. This work was done as part of the Air Force Office of Scientific Research program on Dynamic Sata-Driven Applications Systems (DDDAS). We present a dynamic distributed protocol for reactive conflict management that serves a similar purpose, albeit functioning at a time-horizon in between strategic deconfliction and sensor-based conflict management. This DDDAS inspired approach obviates the need for any centralized control by having each UAS maintain a model of its environment, and exploiting sensing and communication resources as dictated by the lane-based model. In a seminal article describing the purpose and scope of dynamic data-driven applications systems, Darema [97] describes a motivating example where injecting experimental data into a long-running computation (informing oil exploration decisions) could be performed in an online manner to produce better results. An online program in the DDDAS paradigm accepts data whenever it is available and could also inform the measurement 125 process to improve system efficiency. The computational effort required to produce good decisions is also a motivating factor for the development of a DDDAS approach to traffic management described here. NASA and the FAA are making a concerted effort to develop an Unmanned Aircraft System (UAS) Traffic Management (UTM) system to enable large-scale UAS exploitation in urban environments. The UTM is organized in terms of UAS operators who manage their flights through UAS Service Suppliers (USS). These service suppliers must declare the geographic region of their flights (in terms of 4D trajectories of space-time), and moreover, must strategically deconflict their flights pairwise with all other UAS flights in the region (we call this method FAA-NASA Strategic Deconfliction or FNSD). This can easily lead to quite complex path planning and coördination problems, and also requires USS to share data which would best be kept private. We have introduced a lane-based organizational structure for a UTM in which a set of lanes are defined (much like a ground road network), and then a USS simply reserves a sequence of lanes from takeoff site to destination site [11, 38]. In that work, we demonstrated a lane reservation system that efficiently guarantees strategic deconfliction, however that only applies to flights that have yet to be active in the airspace. Active flights experience a more dynamic situation, where contingencies can occur. Contingencies are communicated to agents in an online fashion, either by tactical avoidance sensors such as radar and sonar, or as information from authorities and other agents. Both sources can result in undesirable system responses, for example cascading effects due to high-density operations [42] and unstable control response due to the structure of the information flow [98]. We described earlier the Lane Strategic Deconfliction algorithm (called LBSD), and showed that it has very low complexity, and allows for quite acceptable lane stream properties. Overall, contingencies that lead to a violation of safe separation represent the most critical element to consider in the design of a large-scale traffic management system. Safe separation requires agents to plan collision-free paths, which in the most general case of multiple-agent planning is PSPACE-hard. Even the more narrow problem of tuning velocity profiles is NP-hard [48]. We have given a lane-based airspace model that enables the propagation of contingency information in a well-defined manner. UAS plan locally in real-time within lanes, 126 broadcasting contingencies (as deceleration events) to neighboring lanes that are likely to be effected. Unlike car-following models [46], information from a contingency can reach multiple agents at the same, yet enabling agents to react in a similarly predictable way. The theoretical contribution here provides an efficient real-time algorithm for strategic deconfliction and applies a solution in terms of ground-delay (delaying access to the airspace network) or air-delay. The experimental section below demonstrates the ability to resolve conflicts within a simulated environment. 7.1 Lane-Based UTM A central issue concerning the DDDAS paradigm is the choice of model, and how information is represented, distributed, and consumed. The lane-based airspace structure is a model for the configuration of UAS in space and time and contrasts with other proposed models, such as the grid-based structure proposed by NASA. For example, in a grid model UAS share position information (through a USS as a proxy) within cells of a grid, and it is incumbent on USS to determine whether changes to trajectories could impact operations in neighboring cells. In other words, the flow of information between cells is not explicit in the model and represents a major point of uncertainty in the system. This contrasts to the lane-based approach, where impacts of trajectory changes (the dynamic data in this system) within a lane propagate in a well-defined manner throughout the lane network. The lane-based approach imposes a clear downstream and upstream direction to the information flow because lanes form a graph structure that mirrors the possible paths by UAS. The representation of trajectories in the lane-based approach is simple, as described below, and limits the amount of information that must be shared between aircraft to ensure safe separation. Finally, utilities can be defined in a straightforward way for both the UTM and UAS; e.g., the distance between all flights is important for the UTM, while maintaining desired speed and distance to destination characterize the utility of a configuration for a UAS. Given a set of ground launch and land sites, a set of one-way lanes is defined which provides a path from any launch to any land site. A lane is a directed 3D vector with its tail as the entry point to the lane and its head as the exit point. A flight path is a sequence of lanes starting with a vertical launch lane and ending with a vertical land lane. 127 A crucial constraint on lanes is that every vertex (entry or exit point) has either in-degree 1 or out-degree 1; this allows the deconfliction of flights by considering lanes as opposed to nodes in the network. In order for two UAS to be safe, they must at no time be closer than some minimal Euclidean distance, called dS . We assume that lanes are defined so that no two lanes have points closer than dS unless the two lanes share an endpoint. Figure 7.1 shows the simple lane layout used in the set of experiments described below. There are 51 lanes, along with 10 launch lanes and 10 land lanes. 7.2 Contingencies Both approaches (FNSD and LBSD) are subject to the problem of contingencies when a UAS flight departs from its nominal plan (e.g., slows down, goes off course, etc.). Due to the complexity of the UTM system, predicting the effects of contingencies is a major impediment to the wide-spread integration of UAS into the urban airspace. The currently published protocol for mitigating many contingencies requires the UAS to try to return directly to its launch site [88]. However, this trajectory may not be strategically deconflicted and requires obstacle detection and avoidance along the way. The lane-based model, together with the coördination protocol proposed here, offers a method to mitigate such a contingency and also provides techniques to analyze the possible outcomes of different contingencies. The well-defined structure of lanes suggests that only a restricted set of contingency trajectories need to be considered, those that follow the lane structure and those that do not. For example, addressing contingencies where UAS must exit a lane could include designating emergency side lanes where a UAS can wait, or dynamic landing lane creation to go to the nearest safe landing site. In the case that the UAS can still follow lanes, the simulations demonstrated in the experimental section below offer a method to understand the possible outcomes. In [11] an analysis of the impact of lane density on the delay of a requested lane reservation was shown to be an instance of a process of random space filling, sometimes referred to as Renyi’s parking problem [36]. The lane-based structure imposes constraints on the network that make this analysis possible and could inform what a safe operating density for the UTM should be. The proposed real-time tactical deconfliction method described in this paper simply modifies UAS speeds throughout the network in such a way as to avoid conflict. This 128 method effectively absorbs contingencies when the UAS agent is still capable of following lanes. In the event of a contingency where a UAS cannot still follow lanes, the impact is minimized because non-contingent operations remain within the lane structure. As long as all flights are strategically deconflicted and stick to their assigned flight paths, then there will be no problems in the airspace. However, situations can arise which cause a flight to deviate from its nominal path. Example include: • weather conditions: wind, rain, snow, etc. can cause a flight to slow down in the direction of the lane or move off-course, • platform failures: power, navigation, fuel or structural damage, etc., can prevent a flight from maintaining its speed, altitude, etc., • priority preemption: emergency or other aircraft may be given authorization to use a lane during the time allocated for a lower priority regularly scheduled flight. Note that these and other conditions may be statistically predictable, but it is not possible to know exactly when they will occur. There are a couple of ways to handle contingencies: pre-planned versus dynamic mechanisms. Pre-planned mechanisms involve either modifying the design of the airway lane structure so that flights may address contingencies by using those lanes, or by establishing UTM parameters or policies which allow specific contingencies to be overcome. Dynamic mechanisms are those which create structures on the fly or provide for UAS to interact and modify their flight paths in order to stay safely deconflicted. 7.3 Pre-Planned Contingency Mechanisms Consider how ground road networks provide ways for automobiles to handle contingencies. On most roads if a vehicle breaks down or runs out of gas, etc., there is a shoulder or emergency lane where the car can pull over and take care of the problem. Moreover, in town there are usually many less-traveled side streets where a car can pull off the main road. Similar to ground road networks, it is possible to create emergency air lanes alongside every regular lane so that a UAS which has a problem can pull into that emergency lane and address its problem. Such emergency lanes can be defined when the airway lane network is constructed. 129 If a UAS is unable to continue its flight to destination, then it would need to land. This can also be addressed when the air lane network is defined. For example, every emergency lane can have an associated landing lane. In this case, the emergency side lane could be comprised of two sub-lanes with opposite directions of travel towards each other. A landing lane can have its entry point where the two emergency sub-lanes meet. All this structure can be pre-defined. Of course, multiple aircraft in the same lane may have problems, and therefore their use of the emergency and landing lanes must be coördinated. If they retain adequate platform control, then the reservation system can be used to schedule them from their current positions through the emergency lane and down the landing lane. Such coördination can take into account the remaining individual capabilities of the aircraft involved. If some UAS is seriously incapacitated, but can still fly (i.e., it is not deploying a parachute and falling from the sky!), then it can be cleared to descend as needed. A multi-craft emergency in the same lane can also be resolved dynamically (see below) by having the flights coördinate between themselves the usage of the emergency and landing lanes. It is also possible to implement UTM policies to help manage certain types of contingencies as opposed to defining emergency lane structures. For example, if a UTM requires all flights to fly at the same speed, say s, then if some flight slows down to speed s0 , then the UTM can require all flights to slow to speed s0 . At this reduced speed the flights will remain deconflicted. To see this, consider first if two flights are in the same lane when the speed reduction occurs, then the distance between them will not change. Now consider two flights which are in lanes which merge to a single lane. Let flight f 1 be at distance d1 from the merge point (i.e., the exit point for the current lane for f 1 and the entry point for its next lane), with speed s and scheduled arrival time t1 . Similarly, let flight f 2 be at distance d2 in its lane from the merge point and moving with speed s and scheduled to arrive at the merge point at time t2 . Suppose that at time t both flights must reduce their speeds to s0 < s. It must be shown that the headway distance is maintained at the lower speed. For the scheduled flights we have: d1 = s ( t1 − t ) (7.1) d2 = s ( t2 − t ) (7.2) 130 with the reduced speeds we have: d1 = s0 (t10 − t) (7.3) d2 = s0 (t20 − t) (7.4) which yields: t10 = t + d1 s0 (7.5) t20 = t + d2 s0 (7.6) t10 = t + s ( t1 − t ) s0 (7.7) t20 = t + s ( t2 − t ) s0 (7.8) Substituting the definitions of d1 and d2 : Subtracting the first from the second and rearranging gives: s0 (t20 − t10 ) = s(t2 − t1 ) (7.9) Since the following holds for the headway distance dh : d h ≤ s ( t2 − t1 ) (7.10) dh ≤ s(t2 − t1 ) = s0 (t20 − t10 ) (7.11) then and, in fact, the same headway distance is achieved. There are other ways to pre-define solutions for contingencies. As one more example, consider a flight which is informed that one of the lanes it is scheduled to traverse in its flight path is no longer available. It is possible that the reservation system actually reserves alternate paths to the destination so that the flight can choose one of these alternates if necessary. In this case, the flight will choose a reserved flight path that does not include the unavailable lane. As the flight passes turning points to take alternate routes, it can inform the UTM which releases those reservations and makes the lanes available for use by other flights. 131 7.4 Dynamic Contingency Mechanisms Dynamic contingency handling allows more flexibility in responding to problems. It also generally requires more communication or sensing capabilities for the platforms (or agents) involved. The advent of 5G and beyond communication techniques is opportune for the development of dynamic contingency handling because of the increased bandwidth and the lower delay times. Consider first the case in which a UAS cannot follow its assigned schedule, and it communicates this to the UTM. At one extreme the UAS may be forced to land, and the UTM can dynamically create a landing lane suitable for the specific UAS and location. According to the type of UAS and the roads and buildings below it, a landing lane may be created to allow the UAS to land on a building top, or in the emergency lane of a road on the ground below it, or in an empty part of a nearby parking lot, etc. Moreover, it is also possible for the UTM to monitor cell phone usage in the landing area and choose a site with the fewest number of people. Alternatively, if the flight is able to continue to its destination, but at a reduced speed, then it may be possible to allow other UAS to continue on their scheduled paths and for them to dynamically interact with the slowed flight in order to avoid it. For example, if emergency side lanes are available in the air lane network, then when a nominal UAS approaches the slowed flight, they can make sure that the minimal headway distance is maintained by detect and avoid measures (assuming the emergency air lane is close enough to the regular lane to warrant that). Negotiation of a passing maneuver can be handled by direct communication, intermediate communication through the UTM, or by using an established protocol. As part of our work on UAS dynamic tactical deconfliction we developed the Closest Point of Approach Deconfliction Algorithm (CPAD)[33]. CPAD was developed in the context of the Dynamic Data-Driven Applications Systems program from the Air Force Office of Scientific Research. This approach is described in detail below. 7.5 Real-Time Tactical Deconfliction Each lane has a set of connected lanes with which it shares an endpoint. A flight in a given lane is tactically deconflicted if there is no point in its trajectory along the lane such that it is within distance dS of any flight in a connected lane. This can be efficiently checked using the Closest Point of Approach (CPA) algorithm as follows. Let two lanes, L1 and L2 , 132 −−−→ −−→ be defined by vectors S¯1 and S¯2 , where S¯1 ≡ P̄1 P̄2 and S¯2 ≡ Q̄1 Q̄2 . The trajectories of flights f 1 and f 2 in lane L1 and L2 , with velocities v̄ and w̄, are defined as P̄(t) = P̄1 + tv̄ and Q̄(t) = Q̄1 + tw̄. Since the velocities are v̄ = s1 ( P̄2 − P̄1 ) | P̄2 − P̄1 | and w̄ = s2 ( Q̄2 − Q̄1 ) , | Q̄2 − Q̄1 | where s1 and s2 are the respective speeds of f 1 and f 2 , then the time, tmin , when the two flights are closest in their trajectories is: tmin = −( P̄1 − Q̄1 ) · (v̄ − w̄) | v̄ − w̄ |2 (7.12) If tmin is found for t ∈ [tcurrent , tmin TOA ], where tmin TOA is the minimum time of arrival at the end of the lane for flights f 1 and f 2 , then the minimum distance, dmin , between the flights across these intervals is just | P̄(tmin ) − Q̄(tmin ) |. If dmin < dS , then a conflict exists between the two flights. Figure 7.2 illustrates the CPA method. If a flight, f 1 , has a conflict with flight f 2 , then the two flights can be deconflicted as follows: 1 2 3 4 while conflict( f 1 , f 2 ) reduce speed, s1 , of f 1 if s1 < smin then flight f 1 fails Algorithm 5: Deconflict pair. This allows the definition of the Closest Point of Approach Deconfliction (CPAD) algorithm: 1 2 3 4 5 6 7 ∀ active flight, f if f enters a new lane OR a neighboring flight has slowed OR f has reduced speed on its own then call Deconflict Pair for all flights in neighboring lanes if f has reduced speed then f broadcasts this information. Algorithm 6: Closest point of approach. 7.5.1 Approximate Global Deconfliction Using CPAD Global tactical deconfliction is achieved by having each UAS run the CPAD algorithm. CPAD does not guarantee strategic deconfliction (i.e., that no two flights get within distance dS across the entire set of current flight plans), however, it does guarantee that no two flights are ever within distance dS of each other at any time. The benefits of this approach include that that there is no centralized flight planning, no sharing of detailed flight info 133 between USS, and robustness in the face of contingencies. The cost of the approach is that some flights may be forced to fail; however, this can be mitigated by choosing appropriate lane structure, controlling the number of flights, and eventually by dynamic flight route selection (currently the lane sequence is fixed). Certain communication requirements are imposed, however, the data shared between flights is essentially their telemetry data which the FAA-NASA UTM requires broadcasting anyway. 7.6 Experiments A discrete event simulation is run which allows specification of the simulation time interval, [0, tmax ], and the number of flights, n f . One unit distance corresponds to 50 feet, and one unit time corresponds to 10 seconds. Two maximum speeds are considered: 5 and 9, which correspond to about 17 and 31 mph, respectively. Each flight has randomly selected launch and land sites, as well as a random desired launch time. A fixed 3x4 grid of lanes at altitude 10 units is serviced by 10 launch lanes and 10 land lanes (see Figure 7.1). When a flight plan is created for a flight, it consists of a sequence of lanes and for each a specific Time of Departure (TOD: departs entry point to lane) and Time of Arrival (TOA: arrives at exit point of lane). The next event is just the flight with the earliest TOA in its current lane, unless it has not yet launched in which case it is the current launch time. The launch times of the flights are uniformly distributed across the simulation time interval. Note that if a flight cannot launch at its desired launch time due to conflicts in the launch lane, then it is rescheduled to a later time (with fixed delay). Once an event is selected, all flights are advanced according to their respective speeds in their current lanes. Next, the flights are deconflicted. We consider two aspects for study: (1) maximum simulated time (set to 100 and 200 units), and (2) maximum UAS speed (set to 5 and 9 units distance per unit time). These correspond to about 17 and 33 minutes, and 17 and 31 mph, respectively. The number of flights is chosen to equal the maximum time since this represents on average one launch per launch site every 50 seconds. Given a max time, UAS max speed, and number of flights, the simulation is run using the CPAD algorithm. Table 7.1 gives the data for five representative runs, as well as the means. As can be seen, these results indicate that the CPAD algorithm works well in these sce- 134 narios with only one flight failure in all of the experiments (3000 flights overall). Moreover, the average speed is quite near the maximum allowed speed, and there are very few delays (68 out of 3000). The most critical parameter for algorithm performance is the maximum speed of the UAS. Other trends revealed in the data include that the longer the time period, the more flights complete their mission, and the fewer flights are delayed or in the air (on average). 7.7 Conclusions and Future Work The lane-based approach provides a viable model for large-scale urban air traffic, and CPAD closes the symbiotic DDDAS feedback loop to update the model based on measurements and communication as required by the model. The results here lay the foundation for a further study into the role of DDDAS in large-scale unmanned traffic management. System designers must consider the impact of airspace structure on information flow as well as the accessibility of the network (measured as delay). This demonstrates the importance of considering the structure of the descretization of the configuration space and how a real-time dynamic flight deconfliction algorithm can operate under strong assumptions about the space/time structure of the environment. Future issues to be explored include: (1) a broader set of experiments will be run to study the role of the number of lanes, the distribution of flights over lanes, etc., as well as a sensitivity analysis of the experimental parameters, (2) flights are assigned a complete sequence of lanes in this study, but we intend to explore the application of the software defined networking paradigm to dynamically select the lane sequence, (3) the structural properties of the airway network also play a role in facilitating flight deconfliction, and those parameters will be studied, (4) experiments will be conducted on realistic airways scenarios; e.g., the Utah Department of Transportation is exploring the use of the lane-based approach in Utah, where the airways are located above roadways, and (5) CPAD imposes communication requirements on the aircraft, and this aspect will also be studied in terms of the likelihood of failure to communicate correctly and its impact on deconfliction. 135 Figure 7.1: Set of UAS on airways during discrete event simulation. Red dots represent UAS in flight; blue lanes are launch lanes. Figure 7.2: CPA Algorithm: two flights at closest points Ptmin and Qtmin . 136 Table 7.1: Delays and failures in experimental simulations. tmax 100 nf 100 smax 5 100 means 100 9 200 means 200 5 200 means 200 9 means Wait 1 2 0 0 1 0.8 0 1 0 0 0 0.2 0 0 0 1 0 0.2 0 1 0 0 0 0.2 Fly 18 12 15 11 18 14.8 11 8 12 6 11 9.6 14 11 17 13 6 12.2 7 6 8 7 4 6.4 Done 81 86 85 89 81 84.4 89 91 88 94 88 90 186 189 183 186 194 187.6 193 193 192 193 196 193.4 Fail 0 0 0 0 0 0 0 0 0 0 1 0.2 0 0 0 0 0 0 0 0 0 0 0 0 Avg Speed 4.98 4.98 4.99 4.98 4.96 4.98 8.98 8.94 8.99 8.99 8.98 8.98 4.96 4.97 4.98 4.99 4.96 4.97 8.96 8.97 8.97 8.98 8.97 8.97 Delays 2 2 1 2 4 2.2 1 2 0 0 0 0.6 6 8 6 10 9 8.6 4 2 4 3 2 3 CHAPTER 8 AGENT BASED MODELING AND SIMULATION Various Agent Based Modeling and Simulation software frameworks exist (see [99– 101]), however a specific framework was developed here in order to find a set of symbiotic UAS behaviors and UTM policies. The framework is instrumented to allow measurement of crucial features, including local statistics and flow metrics, contingencies (and if possible their causes), and higher-level system features and emergent behaviors. Our previous work on the BRECCIA system ([9, 10]) included a BDI-agent based framework built from a Java library called Jason [73]. However, the ability to rapid-prototype different models of communication, or create complex agents, is inhibited by having to switch between Java and the Jason domain-specific language. To better enable rapid prototyping an objectoriented framework was developed in Matlab. 8.1 Lane Systems and Sensitivity Once the airway network is defined, a lane-based strategic deconfliction algorithm is required to schedule flights into the lanes so as to maintain the required minimum separation at all times during flight; this assumes that every flight follows its approved flight plan. The Lane-Based Strategic Deconfliction (LBSD) algorithm is given which allows computationally efficient scheduling. It is shown that this algorithm is O(k2 ), where k is the number of flights in the lane sequence of the proposed flight during its flight. Alternatively, in-flight planning arises due to contingencies, i.e., possible future events, usually causing problems or making further plans and arrangements necessary. Contingency handling may occur at different time-horizons and require different mechanisms, for example tactical (sensor-based) deconfliction. For these scenarios the Closest Point of Approach (CPA) algorithm is defined so that UAS can exploit the lane structure to continue their flights while avoiding collisions. This protocol may be based on either individual 138 UAS sensor data or on local inter-UAS communication. 8.2 Lane Systems and Robustness The layout of the lane system can also have significant effects on the behavior of the system. A common refrain among air mobility enthusiasts is that the ability to travel point-to-point in a straight line, Figure 8.1 for example, should be maintained and decreases the desirability of structured airspaces. However, a system of agents performing individually optimal trajectories in an unstructured airspace is unlikely to produce an efficient system. This is true in the case where agents can make decisions dynamically based on system-wide conditions, for example, Braess’ paradox demonstrates where additional route options can result in an increase in travel time [102]. This also appears to be true when considering conflict counts for a simple cell-based deconfliction experiment (point-to-point flights deconflicted using ground-delay, we call the FAA-NASA approach). Figure 8.2 show histograms for cell traversals (how many times a flight crossed a cell) and intersections (how many flight paths intersect) for an experiment with 1000 UAS flying point-to-point in an unstructured airspace with uniformly distributed land and launch sites. These graphs show an increased density of conflicts focused in the center of the area of interest. An ABMS framework that simulates AAM should model the roles and responsibilities of the real UTM framework. However, the separation of responsibilities between the USS (or PSU) and the operator can be merged for the purposes of analyzing the resulting traffic and system behavior, since this separation mainly serves regulatory requirements. Therefore a reasonable organization for an ABMS includes the following organization: • LBSD: This class encapsulates the Lane-Based Strategic Deconfliction (LBSD) Supplemental Data Service Provider, responsible for approving reservations into the airspace • ATOC: An instance of this class represents an Air Traffic Operations Center, providing users with visualization capabilities • UAS: Unmanned Aircraft System is a mobile agent that operates within the UTM • KB: A database for knowledge storage and access 139 • SIM: This class encapsulates all the simulation functions, including mocking gps and radar sensors and updating agents • RADAR: An encapsulation of a reduced-order radar sensor model 8.3 ABMS Optimization The complexity of determining good plans for handling contingencies is a central problem for large-scale autonomous systems. Different adaptations of the lane-based approach can represent the majority of proposed airspace structures, from the least structured (every UAS creates its own lane, i.e., the free-flight model), to the most structured, where regulators completely determine the layout of the airspace. How these structural decisions affect the behaviors of individual UAS, and ultimately the system dynamics, are captured by the ABMS approach. ABMS is ideally suited to the analysis of complex, large-scale systems with interacting heterogeneous agents, and can incorporate cognitive models which mediate how individual agents perceive and react to the environment. By alternately optimizing individual behaviors and system-level policies, a symbiotic design is found that balances individual preferences, which include the ability to make good decisions for handling contingencies, and system-level metrics, such as network accessibility and efficiency. Figure 8.3 shows the iterative process which uses ABMS to assess policies and behaviors to demonstrate the effectiveness of the lane-based approach. The goal is to optimize: 1. UTM Policies: UTM policies over a given set of UAS behaviors, and 2. UAS Behaviors: UAS agent behaviors given a set of UTM policies. In each iteration, a set of policies is selected for a given environment, then UAS behaviors are adapted to improve individual and system-level measures (compromises are explicitly documented). For the next iteration, the UAS behaviors are held fixed while the policies are adapted. In addition to manually adapting the behaviors and policies, hierarchical reinforcement learning (RL) techniques may be applied to determine whether more robust systems can be found automatically. The goal is to understand how UTM structure and constraints interact with UAS agents to produce emergent behaviors that im- 140 pact system performance. In particular, it is essential to avoid cascading non-conforming flights arising from contingencies. 8.3.1 Contingency Handling as a Measure of Effectiveness Contingencies are important because they represent the safety and logistics issues that arise in real-world systems, serving major design and operational constraints. Contingencies also represent a considerable computational challenge, since the representation and planning for all possible effects is generally intractable. In fact, contingencies are the primary impediment to the large-scale integration of autonomous systems. Therefore, to demonstrate that the lane-based approach is an effective organizational strategy for UTM, it must be shown that the lane-based approach provides a low-complexity foundation for contingency analysis and mitigation. As an example of how the lane-based approach supports contingency analysis, consider the single-lane example in Figure 8.4 (adapted from [103]). In this model, the lane is a one-dimensional curve represented by an array of length L. Each element of the array can be in one of seven states: it may be empty, or occupied by a UAS having an integer speed of zero to five. The speed value represents the number of array elements that UAS moves forward in the next step of the simulation. The behavior of each UAS is defined as follows, calculated at each step of the simulation simultaneously for all UAS (gap is the number of unoccupied array elements in front of the UAS): 1. Acceleration: Each vehicle with speed v < vmax and gap ≥ v + 1 gets speed v ← v + 1; 2. Deceleration: Each vehicle with gap ≤ v − 1 gets speed v ← gap; and 3. Move: Each vehicle moves forward v elements The simulation in Figure 8.4 begins with 85 UAS placed randomly across the lane. Here, a contingency is defined as a deceleration event (item 2 in the behavior described above). At iteration 500 a contingency is forced on a number of UAS in the lane. Figure 8.5 shows aggregated contingency events for 10 runs of the simulation with 85 and 100 UAS. Figures 8.4 and 8.5 show two dramatically different system responses: in (a) the number 141 of contingencies returns to a settled value before the forced contingency, while in (b) each run produces a different, and hence unpredictable 1 , outcome. The lane-based approach provides a single control variable to account for these two scenarios: the density of UAS in the lane structure. A more complex individual behavior could potentially guard against such unpredictable effects, but with this approach the trade-offs are made explicit for the policy maker. 8.3.2 UAS Behaviors for Contingency Handling Consider a scenario in which ground infrastructure supporting UAS communications is disrupted during normal operations. Currently, the published protocol for handling this contingency is to fly back to base if communications cannot be re-established within a given amount of time [88]. Since this is a pre-defined policy, it is worth considering whether such a policy is robust. For example, depending on how many UAS communications are disrupted, the number of conflicts that result from the simultaneous replanning of multiple agents may have negative cascading effects [42]. As the complexity of the UTM system increases, it becomes harder for experts to enumerate all the failure modes and effects; assigning liability and performing post-failure diagnoses will also be difficult. Table 8.1 provides a relevant set of contingencies, a contingency response, and the likely required communication capabilities for the response. 8.3.2.1 The Role of Cognition in Contingency Handling To develop robust policies for UAS in the UTM system, one can frame the problem in a similar manner as designers did for ACAS X. Another method is to use a hierarchical decomposition of states and actions, and a semi-Markov decision process model for statetransitions [5, 105, 106]. A benefit of the lane-based system is that a concise description of surrounding vehicles, their intentions and locations, is provided by the space-time lane diagram (lanes, reservations, headways, and speeds). The STLD representation of system-level state can be fed to each UAS agent for the purpose of contingency planning, potentially leading to higher preference behaviors. For example, in a communications-loss 1 As noted by Nagel and Rasmussen [103], this model can be treated analytically [104], but the analytical results are “more difficult to obtain” than measurements from simulation. A result that would support this dissertation would show that free-flight systems, the least structured of airspace designs proposed, are more difficult to analyze than lane-based systems and therefore less ideal for contingency handling. 142 scenario, a UAS may decide to seek a different emergency landing pad if it knows that other agents in the lane are likely to execute the same contingency plan. Another possibility is that two UAS with communications loss continue past the disruption because other UAS farther along appear to be functioning nominally, especially if a goal encoding relay-communications were included. Such a scenario-specific policy would be difficult to program explicitly, but such robust solutions are feasible through policy optimization. A hierarchical reinforcement learning technique can automatically determine actions, as well as policy. More recent developments have shown promise in learning games by creating abstractions of the actions available to agents. Both the Option-Critic Architecture [5] and FeUdal Networks [107] take advantage of semi-Markov processes to model different time-scales in the hierarchy, but FeUdal networks benefit from an explicit representation of a high-level goal. The BDI architecture is a convenient way to structure and analyze agent behavior, and includes a hierarchical decomposition of actions in the form of plans. The different decompositions of actions for operators, USS, and airspace controllers in the UTM system are compared in Section 5 on Experiments. For example, two competing high-level goals in the UTM system are to maximize headway and lane density. Actions available to agents include Replan, Land, Cruise, etc. Additionally, since NASA has published the required application programming interfaces (APIs) for USS, the possibility of automatically generating low-level actions from these APIs is also available. 8.3.3 UTM Policies for Contingency Handling An intelligent airspace provides a robust and redundant basis for contingency handling, such as the one-way lane structure, i.e., by eliminating, in the nominal case, any necessity for UAS to coördinate through intersections. However, the lane is more than a simple curve in 3D Euclidean space. Some properties which must be specified include the lane boundary surface (e.g., a tube), minimum and maximum speeds, geometry (straight, curved), length, etc. Another issue to be considered is whether regular lanes between two intersections going in opposite directions should be aligned left-right at the same altitude (as standard ground roads) or one above the other. In addition, lane placement should be conditioned on several crucial operational aspects, some of which include: (1) ground structures: buildings, towers, etc., (2) population density which dynamically varies with time, (3) roadways on the ground: these may be 143 avoided or followed, (4) communications ground stations: which may also provide network repair if ground stations fail, (5) landing sites: availability of emergency landing sites, and (6) monitoring infrastructure: radar, microphones, etc. Some of these choices are not dichotomous. For instance, placing a lane above a road will help to reduce aerial obstacles and improve communications as UAS can use the cellular wireless infrastructure which has full coverage in most of the roads and communicate with ground vehicles and road side units below them using the unlicensed dedicated short-range communications (DSRC) channels [108] which might lead to better safety; however, such a lane assignment increases the population density below the flight, and thus, the risk of human injury. There are four basic types of lanes that the trajectory of a UAS can encounter: (1) launching/landing: allow ingress, egress to/from airspace, (2) regular: allow travel through airspace (intersection to intersection), (3) roundabout: provide way through intersection, and (4) ramp: transition between regular and roundabout lanes. There are several parameters which constrain the placement and connectivity of lanes. For example, a regular lane may be at a different altitude from roundabouts, and therefore ramp lane placement must accommodate the individual lane parameters of each other type. 8.4 ABMS Test: FAA versus Lane-Based Approach To schedule a flight, launch and land sites are selected, as well as a sequence of lanes going from each one to the other, along with a desired speed, and a launch time window. The set of lanes may be selected however desired; for example, to minimize distance or weather constraints, or other relevant factors. The launch time window gives the earliest and latest possible launch times (line 1 of LBSD algorithm). Lanes are scheduled individually by flights, and every new flight must respect the headway distance not only in each lane, but also when moving from one lane to another (i.e., with respect to all merging or diverging lanes). The Lane-Based Strategic Deconfliction algorithm used in the following experiments were described in Chapter 4, Algorithm 1: Label Method. A requirement of the Label Method is that a complete database of flight reservations must be maintained and used by the algorithm; however, this will generally be required by the flight authorities anyway to allow informed monitoring of airspace usage. The original idea of the FAA was to allow a decentralized approach where each USS maintained its own 144 flight info and shared as necessary; the drawback of this is that if any USS fails, the system fails, and there is the possibility of semantic mismatch in terms of trajectory definition (e.g., meters vs. feet). The Space Time Lane Diagram (STLD) that is enabled by the lane-based approach also provides a straightforward way to visualize the traffic through a lane for monitoring UTM operations. Figure 8.6 shows a set of planned flights through a lane, where reservations represent a reduced-order model (speed and headway) for the actual or planned trajectory. and their trajectories reflect the accelerations necessary to turn between lanes. Lanes also allow real-time comparison of the UAS’ planned flight path and the actual trajectories (e.g., provided by telemetry data). 8.4.1 Experiments to Determine Parameter Impact on Scheduling Algorithms In a complicated system like a UTM, analytic solutions may not exist, and therefore, simulations are used to explore UTM performance with respect to parameters of interest. The experiments performed here are designed to allow both inter-UTM (e.g., LBSD vs. FNSD) and intra-UTM (e.g., grid vs. Delaunay) structural analysis, as well as a cursory system/behavioral analysis (relating the agents flexibility in scheduling to the overall system performance). The parameters studied here include: 1. Launch Frequency (flights per hour): comparable to an arrival rate of flights into the system [values: 100 and 1000] 2. UAS Speed (m/s): Average UAS speed through lane [values: 5, 10, 15] 3. Headway Distance (m): Minimum distance allowed between UAS [vaues: 5, 10, 30] 4. Flex Time (sec): Interval of possible launch times for flight [values: 0, 300, 1800]. The simulation covers an area of 5 square km (roughly the size of the Salt Lake Valley) with the FAA cells spaced as a 10x10 cell structure. The LBSD grid was chosen to correspond to this as an 11x11 node grid. The 121 launch (land) sites are located near the ground node points in both layouts. The Delaunay networks are generated with the same number of nodes, but they are distributed randomly (sampled from uniform distribution) in the given area. Road-based networks include an area over San Francisco and an area over Salt Lake 145 City. Ten simulation trials were run for each of the 54 parameter combinations (note that for the Delaunay networks an additional ten trials were run for each due to the random nature of the node locations). The simulation period was set to 4 hours simulated time. The FAA flights are up, over and down trajectories scheduled between randomly selected launch and land sites; the flight altitude was randomly assigned between the min and max altitudes of the LBSD network. For both UTM methods, given the flight frequency, a random set of desired flight times are generated which are uniformly spread across the total simulation time. Figures 8.7 and 8.8 show the mean statistics for launch frequency of 100 flights/hour and 1000 flights/hour, respectively. The means of the maxima over all trials are also given in Figures 8.9 and 8.10. The statistics include: delay (calculated as the time between the requested launch time and the assigned launch time), failed flights (flights that could not be accommodated due to time or space constraints), and deconfliction time (the amount of wall-clock time that the computer required to schedule a flight). This data indicates that all six categories of structures have response characteristics that are most undesirable when the flex is low, the speed is low, and the headway is high. However, the unstructured FAA airspace and the road-based San Francisco networks are particularly sensitive to these inputs with respect to the mean statistics. The max statistics in regard to delay show a somewhat different story where the FAA structure responded similarly to the others and the San Francisco graph performed the worst. These results indicate that small changes in the policies and behaviors may have dramatic effects on what the average UAS agent experiences accessing the unstructured (FNSD) airspace and complex road networks. Conversely, all the structured airspaces had relatively subdued effects related to these inputs (note that Salt Lake City has a grid-like road system). 146 Figure 8.1: Sample straight-line paths between launch and land vertices. (a) Cell traversal counts. (b) Path intersection counts. Figure 8.2: Unstructured airspace density and path intersections. 147 Figure 8.3: Agent-based modeling framework for learning UTM policies and UAS behaviors. The UTM policies are set, and then UAS agents learn both at the BDI cognitive level (beliefs) as well as at the platform control level (plans). UAS behaviors are fixed and UTM policies are then optimized with respect to global lane stream properties as well as safety measures. (a) UAS after one iteration (b) UAS after 91 iterations Figure 8.4: Single lane contingency effects. 148 (a) 85 UAS Showing Predictability (b) 100 UAS Showing Unpredictability Figure 8.5: Contingency effects on system state and individual behavior in a lane. Figure 8.6: Lane diagram for a single lane, showing six flight reservations, planned trajectories, and simulated telemetry. 149 Figure 8.7: Simulation results: averages for launch frequency of 100 flights per hour. The upper row describes the parameter combination enumeration in the lower three rows which give the mean number of failed fights, mean delay, and mean deconfliction for those combinations of parameters. 150 Figure 8.8: Simulation results: averages for launch frequency of 1000 flights per hour. 151 Figure 8.9: Simulation results: maxes for launch frequency of 1000 flights per hour. 152 Figure 8.10: Simulation results: maxes for launch frequency of 100 flights per hour. 153 Table 8.1: Contingency table; V2V (Vehicle to Vehicle V2V), V2G (Vehicle to Ground), and V2X (Vehicle to Everywhere). Contingency UAS Nav Loss UAS Speed Loss UAS Energy Loss UAS Comms Loss UAS Falling Lane Obstacles Lane Closure Responses Get position from other UAS UAS Operator Manual Control Tactical Landing Move to Emergency Lane Alternate Flight Path Tactical Landing Plan Alternate Route to Destination Alternate Destination Nearest Destination Tactical Landing Wait in Emergency Lane Tactical Landing Deploy Parachute Tactical Avoidance Move to Emergency Lane Alternate Route Create lanes Communication Requirements V2V, V2G V2G V2X, V2G V2V, V2G V2G V2X, V2G V2V, V2G V2V, V2G V2V, V2G V2X, V2G V2V, V2G V2V, V2G V2V, V2G V2G CHAPTER 9 STRATEGIC DEPLOYMENT OF DRONE CENTERS AND FLEET SIZE PLANNING FOR DRONE DELIVERY This chapter is taken from a study performed for the Utah Department of Transportation in which the use of drones was compared to ground vehicles for package delivery. For more details, see [109]. The research described here produces simulations for state-wide drone delivery considering a truck/drone fleet mix. Such airspace network is optimized to ensure that drones are strategically deconflicted as required by NASA and the total energy over that day is minimized. Such an airspace network is realized in a web-based platform, named the Utah Advanced Air Mobility Simulator (UAAMS). UAAMS enables researchers, planners and practitioners to record and update assumptions about the distribution of vertiports, traffic, population, and other requirements that may affect the operation of the transportation network. 9.1 Introduction The parameters of a specific UTM application are important not only for considering the feasibility of Unmanned Aircraft Systems (UAS) package delivery within the state, but also for determining the impact and ultimately the efficient operation of drone delivery. UAAMS allows the Utah Department of Transportation (UDOT) to assess different assumptions of the model and run “what-if” scenarios by generating an animation of the optimized airspace network. The platform provides the state with more clarity about the energy impacts of large-scale drone delivery, as well as a viable airspace network. The tool can further inform the UDOT Division of Aeronautics to develop policies and negotiate with industry stakeholders. Through this UTRAC research project, the implementation and deployment of the UAAMS was successful and will be available for continuing research in the development 155 of advanced air mobility in the state of Utah. This type of tool is critical for research in this area, as it incorporates the latest software and infrastructure development techniques available. Also shown was the viability of considering the large-scale impacts (e.g., environmental) of advanced air mobility on specific communities by using micro-simulation technology. In contrast to micro-simulations performed for human-controlled ground and air traffic, in this case a model for the autonomous agents has the potential to be exact, since their algorithms must be documented. Future improvements will consider optimizations to the object function in the optimization procedure – this would allow a more thorough exploration of the possible configurations of vehicle and vertiport parameters. 9.1.1 Problem Statement In a 2017 report by the RAND corporation [110], analytical methods for calculating the total energy consumed by a mix of delivery trucks and drones were developed and shown to be highly dependent on the layout of distribution centers as well as distance traveled by delivery vehicles. This suggests that the city layout, i.e., street connectivity and other network parameters, are important considerations for energy-conscious policies. While industry stakeholders must determine the market viability of drone delivery, they are not required to calculate the external and indirect costs that may be associated with this burgeoning industry. The web-based platform developed for this report, called the Utah Advanced Air Mobility Simulator (UAAMS), enables researchers, planners and practitioners to record and update assumptions about the distribution of vertiports, traffic, population, and other requirements that may affect the operation of the transportation network. These parameters are important not only for considering the feasibility of Unmanned Aircraft Systems (UAS) package delivery within the state, but also for determining the impact and ultimately the efficient operation of these new transportation technologies. Furthermore, additional analysis and what-if scenarios may be developed using this simulator. Example Jupyter notebooks (python) are provided to help guide development, however users are not limited to any particular programming language. To facilitate the iterative process needed for the development of Advanced Air Mobility in the State of Utah, UAAMS is a web-based software and can be accessed from any web browser. Additionally, the simulator is delivered with the open source web-map server, GeoServer [111], which con- 156 tains the geospatial data used or generated by the simulations. This enables multiple agencies within the state, who often use Geographic Information Systems (e.g., ArcGIS), to communicate planning efforts and incorporate data from, or provide data to, these simulations. Figure 9.1 shows the main features of the software developed for this project: 1. Simulator Form: The simulator form enables the client to enter assumptions about the vehicles and the environment, and then execute the simulation. 2. 3D Map Interface: A viewer for map and simulation data. 3. File System Explorer: Input and output files for simulations (e.g., vertiport locations and result figures) may be stored in the filesystem. Additionally, all the source code for simulations are accessible through this file system explorer. 4. Simulation Output Viewer: Simulations produce a number of figures that can be viewed here. An initial simulation implementation was developed for this report to demonstrate the workflow for running simulations, as well as developing new ones, and is described in the sections that follow. The UAAMS is deployed to US data centers on Google Cloud, and is accessible by anyone with authorized credentials by visiting the url https://utrac.georq.io. 9.1.2 Objectives The motivation for this work is to provide regulators, policymakers and industry stakeholders with a data-driven framework for assessing the energy costs and trade-offs of large-scale drone delivery in the state of Utah. The primary objective is to produce a web-based platform that takes inputs of state-wide road network, the total number of (drone-deliverable) packages to deliver on a given day and their destinations, and energy and cost assumptions per vehicle, and produces a state-wide airspace network, delivery schedule, and truck/drone fleet mix. A secondary goal is to optimize the network to ensure that drones are strategically deconflicted as required by FAA/NASA and the total energy over that day is minimized. Overall, this program will provide the state with more clarity about the energy impacts of large-scale drone delivery, as well as a viable airspace network. 157 9.1.3 Scope This research involves three major components: data collection, optimization model development, and web-based platform development. We first gathered data on the statewide roadway network, population data (year 2025), Traffic Analysis Zone (TAZ) boundary, and post office locations. The dataset enabled the creation of delivery zones to simulate package delivery coverage area for each drone center. In addition, the air delivery network was created by lifting the virtual highway network into the sky. This includes generating airways, deconflicting zones for Utah considering the delivery scheduling, launching/landing lanes, etc. An optimization model was developed to determine the schedule and deployment of drones within the study area. The developed system model with the optimization component was then implemented onto a web-based platform where people can assess different assumptions of the model and run what-if scenarios by generating animation of the optimized airspace network. The rest of the discussion is structured as follows. Section 9.2 summarizes the research methods. Section 9.3 illustrates the web-based platform along with a case study for a medium-sized simulation with 33 TAZs. Section 9.4 presents the results and findings, and outlines the lessons learned for follow-up research. Appendix D of the UDOT Technical Report [109] includes the user guide of the developed web-based platform. 9.2 Research Methods Large-scale drone delivery is on the horizon nationwide, as it has the potential to decrease pollution and help alleviate road congestion. Up until now, industry is mainly concerned with the market viability of drone deployment, and very little attention has been paid to external costs such as energy tradeoff. The benefits of drone deployment, however, are largely dependent upon layout of distribution centers and distance traveled. To this end, we employ a data-driven approach to strategically replace ground-based delivery networks with air-based drone deployment. In this project, we structured the air space by regulating and treating it as lanes, and optimize drone dispatch. Figure 9.2 shows the methodological framework of our research. 158 9.2.1 Data Source The main data sources utilized to be fed into the airspace network construction consist of four parts: 1. Population projection: this is retrieved from the Utah Geospatial Resource Center (UGRC) [112] and is collected at the TAZ level. Year 2025 data was retrieved as the demand input. The population projection was further converted into parcel demand estimation, where on average 21 parcels/capita/year was assumed initially [113]. 2. Post office locations: this is retrieved from UGRC [114] to serve as potential truck delivery centers. Figure 9.3 shows the overlay of post office locations with the TAZ boundary. 3. Utah road network: this is retrieved from UGRC [115] and the layout is used to create the elevated virtual airways in the sky for the drone fleet. 4. Utah building footprints and addresses: this is retrieved from UGRC [116] and provides the locations of possible drone deliveries for demand modeling. 9.2.2 Optimization Setup The optimization procedure considers possible locations for vertiports to answer the question: Does there exist a set of vertiport locations that minimizes the total energy requirements? The spatial distribution of vertiports affects which addresses are served by them and therefore the distance travelled by UAS. Other parameters, such as vehicle characteristics, the proportion of packages that are delivered by UAS, and the routes of UAS and trucks have important effects, but were not considered in this procedure. To gain some intuition about how the locations of drone ports can affect the deliveries of packages, consider the relative locations of a drone center, truck center, and two TAZs in Figure 9.4. At the start of the optimization, the drone center and the truck center are co-located. Package deliveries are scheduled according to the procedure which assigns aircraft to addresses that are within range and capabilities (according to the simulation parameters), and then fills the resulting demand with truck deliveries. For more efficient simulation, the truck distances and energy are calculated by solving a probabilistic traveling salesman problem [110]. In contrast to the regular shaped cells used for this calculation in [110], 159 this simulation calculates the resulting convex hull that encompasses the truck-assigned demand. At each iteration of the optimization, the drone centers are moved to a new location, the simulation is re-run, and the resulting total energy is calculated. The optimization is complete when further updates to the locations of the drone centers results in an increase in the total energy consumption. Several methods for optimizing vertiport locations were considered for this project, with the settled approach being the Covariance-Matrix Adaptation Evolution Strategy (CMAES) [117]. The objective was to minimize the total energy required by both UAS and trucks to deliver a set of packages in an area. Only the locations of the vertiports were considered variables in the optimization, while the job mix, number of vertiports, and vehicle parameters were held constant. Therefore, the number of variables in the optimization equals the number of vertiports under consideration. Figure 9.5 shows a plot of a sample optimization run for a single TAZ. 9.3 Web-Based Platform A simulation that extends what was described in the RAND report is provided along with UAAMS. This simulation incorporates all the parameters listed in Table 9.1. In contrast to the RAND report, this simulator considers the real locations of addresses in Utah [114], as well as the projected population densities in order to simulate projected demand that is more accurately distributed throughout the state. Figure 9.6 shows a cross-section of these datasets. The simulation implementation progresses along the following steps: • Load Vertiport Locations • Load Truck Depot Locations • Load Population Projections (TAZ Zones) • Load Building Locations • Initialize UAS Model • Filter TAZ Based on Requested City Areas • Create Demand (Parcel Requests for a Day) 160 • Source Parcel Requests to Nearest Vertiport • Calculated Parcel Requests within UAS range and Capability • Calculate Truck/UAS Job Mix • Generate UAS Trajectories • Estimate UAS Round-Trip Times • Estimate UAS Energy Requirements • Estimate Truck Energy Requirements • Generate Animation • Generate Results 9.3.1 Case Study A medium-sized simulation was executed that included 33 TAZs as a case study to demonstrate the web-based platform. 9.3.1.1 Simulation Form A simulation form is displayed whenever a file with the suffix.sim is clicked in the workspace explorer. As this form is edited by the user, the parameters are written to the .sim file (the raw json format can be viewed by opening the file in the code editor as shown in Figure 9.7. When the Run Simulation button is clicked at the top of the form, the python program defined in the Simulation Runner field is executed and the .sim file is provided as an argument to the program. 9.3.2 Simulation Procedure The following section describes each step of the simulation in detail. The complete source code of the simulation is delivered with UAAMS and accessible from the file explorer. 9.3.2.1 Load Input Location Data This step includes loading the following location data: 161 1. Vertiport Locations 2. Truck Depot Locations 3. Population Projections (at the TAZ level) 4. Building Locations Depending on the size of the dataset, the location data can be loaded either from the filesystem or from a web map server (one is provided with UAAMS). For example, the population projections dataset was small enough to have loaded from a shapefile located on the filesystem into memory, while the building locations dataset is too large to handle in this way. The buildings dataset is loaded into the web map server offline, then during the simulation the server is queried within constrained areas. 9.3.2.2 Initialize UAS Model The UAS model considered for this simulation is derived from a study conducted by D’Andrea [118]. The parameters considered are listed in Table 9.1. 9.3.2.3 Filter TAZ Zones Based on Requested City Areas The simulation can consider a subset of areas in Utah, defined by the “City Areas” column in the Population Projections dataset [112]. 9.3.2.4 Create Demand (Parcel Requests for a Day) For each TAZ under consideration, uniformly sample the building centroids within that TAZ from a binomial distribution with intensity given by the estimated number of parcels per person per day. 9.3.2.5 Source Parcel Requests to Nearest Vertiport For each parcel request in the demand dataset (from the last step), calculate the nearest vertiport that can serve that request. 9.3.2.6 Calculate Truck/UAS Job Mix Fixed by user input into simulation the form. 162 9.3.2.7 Generate UAS Trajectories For each parcel request that is served by a UAS, generate a trajectory based on the vehicle parameters provided in the simulation form. 9.3.2.8 Generate Animation To help visualize the distribution and density of drone flights that meet the demand specified in the simulation, a 3D animation is generated that can be viewed within the workspace. The animation is specified by an open-source human/machine readable file called CZML [119]. Figure 9.8 shows an example frame from this animation. 9.3.2.9 Generate Results Results are generated and stored in figures as described in Figure 9.9. 9.4 Conclusions and Future Work The implementation of a cloud-based collaborative simulator represents the bulk of this UTRAC project. A major unexpected hurdle was that the scale of the simulation required significant memory and compute resources to execute. In particular, the building dataset represented a challenge and required the use of a web map server rather than simply loading the entire shapefile into memory. During the demand creation step in the simulation implementation, small areas were queried to avoid overwhelming memory resources. With regard to the test simulation, the total energy required by both UAS and trucks combined was slightly more than the required energy if trucks were to meet the same demand alone. This could be a result of the chosen vehicle parameters, as well as the location distribution of vertiports. Despite this result, it may still be desirable to take advantage of the short distance and round-trip times that UAS has to offer. Another challenge was encountered with the optimization objective. It was found to be computationally difficult to consider all the vertiport locations, as well as the various input parameters that could have been considered. The algorithm that was chosen to perform the optimization is called the Covariance-Matrix Adaptation Evolution Strategy [117] because it has a good reputation for handling highly non-linear problems with many variables. However, the algorithm proved to be rather slow, likely due to the random sampling and 163 database querying that occur during the demand creation step. Consequently, in the time allotted for optimization, the algorithm did not successfully converge to a solution. Overall, the implementation and deployment of the UAAMS was successful and will be available for continuing research in the development of advanced air mobility in the state of Utah. This type of tool is critical for research in this area, as it incorporates the latest software and infrastructure development techniques available. Also shown was the viability of considering the large-scale impacts (e.g., environmental) of advanced air mobility on specific communities by using micro-simulation technology. In contrast to micro-simulations performed for human-controlled ground and air traffic, in this case a model for the autonomous agents has the potential to be exact, since their algorithms must be documented. Less certain are the emergent effects that may result from the parallel execution of these algorithms, and the environmental impacts of their collective behavior. This UTRAC project has resulted in feedback from stakeholders that can be used to improve the simulation tool, as well as the resulting analysis. For example, simplifications to the workspace are necessary for non-developer, or non-engineer, stakeholders (e.g., planners). It is crucial to enable a frictionless communication between them and a tool like UAAMS has the potential to fill this role. Future improvements will also consider optimizations to the object function in the optimization procedure - this would allow a more thorough exploration of the possible configurations of vehicle and vertiport parameters. 164 Figure 9.1: The Utah Advanced Air Mobility Simulator (UAAMS). Figure 9.2: Methodology framework of proposed drone network optimization. 165 Figure 9.3: TAZs with post office locations. 166 Figure 9.4: The location of drone centers affect which addresses trucks and drones service. Figure 9.5: Optimization iterations showing divergent behavior. 167 Figure 9.6: Utah Buildings dataset overlaid on population projections dataset. 168 Figure 9.7: The simulation form. 169 Figure 9.8: One frame of the simulation animation results. 170 Figure 9.9: Simulation-produced plots. 171 Table 9.1: The simulation parameters. Form Field Simulation Parameters Name Simulation Runner Simulation Year Population Projections Map Initial Vertiport Positions Parcels/Person/Year Ktsp Average Wind Speed Init. Deliv. by UAS Ratio Init. UAS Per Package Areas to Analyze Truck Parameters Truck Efficiency Parcels Per Truck UAS Parameters UAS Payload Mass Lift-to-Drag Ratio UAS Vehicle Mass Power Transfer Efficiency Cruise Speed Power Consumption Climb Rate Maximum Range Package Load Time Cruise Altitude Description simulation name python script name Pop. projection year shapefile/geojson file post office locs.K avg # packages TSP constant avg wind speed felt from trucks to UAS UAS/package at vertiport. areas from TAZ dataset Truck efficiency mpg Truck package load payload mass (kg) of UAS UAS mass (kg) motor to propeller UAS cruise speed of electronics UAS climb rate of UAS onto a UAS UAS cruise altitude (m) REFERENCES [1] J. Rios, I. Smith, P. Venkatesan, D. Smith, V. Baskaran, S. Jorcak, S. Iyer, and P. Verma, “UTM UAS service supplier development,” NASA, Hampton, VA, USA, Tech. Rep. NASA/TM-2018-220050, Dec. 2018. [2] Booz Allen Hamilton Inc., Kimley-Horn and Associates Inc., “Transportation management center business planning and plans handbook,” Federal Highway Administration, Washington, DC, USA, Tech. Rep. TMC Pooled Fund Study, Dec. 2005. [3] Institute of Informatics. Belief, desire and intention agents. (2019) Federal University of Rio Grande do Sol, Porto Alegre, Brazil. Web Page for Research Group. [4] P. Caillou, B. Gandou, A. Grignard, C. Q. Truoung, and P. Taillandier, “A simple to use bdi architecture for agent based modeling and simulation,” in Conf. Euro. Soc. Sim. Assoc., Groningen, The Netherlands, Sep. 2015, pp. 15–28. [5] P.-L. Bacon, J. Harb, and D. Precup, “The option-critic architecture,” in AAAI Conf. Art. Intell., San Francisco, CA, USA, Feb. 4–9, 2017, pp. 1726–1734. [6] National Aeronautics and Space Administration (NASA), “Urban air mobility market study executive summary,” National Aeronautics and Space Administration (NASA), Hampton, VA, USA, Tech. Rep., 2018. [7] Federal Aviation Administration, “FAA Aerospace Forecast Fiscal Years 2020-2040,” Federal Aviation Administration, Washington, DC, USA, Tech. Rep., 2020. [8] C. Reiche, R. Goyal, A. Cohen, J. Serrao, S. Kimmel, C. Fernando, and S. Shaheen, “Urban air mobility market study,” National Aeronautics and Space Administration (NASA), Hampton, VA, USA, Tech. Rep., 2018. [9] D. Sacharny, T. C. Henderson, A. Mitiche, R. Simmons, T. Welker, and X. Fan, “BRECCIA: Unified probabilisitic dynamic geospatial intelligence,” in IEEE Conf. Int. Robot. Sys. (IROS 2017 Late Breaking Paper), Vancouver, Canada, Sep. 2017. [10] D. Sacharny, T. Henderson, R. Simmons, A. Mitiche, T. Welker, and X. Fan, “BRECCIA: A novel multi-source fusion framework for dynamic geospatial data analysis,” in IEEE Conf. Multisensor Fusion Int., Daegu, South Korea, Nov. 16–18, 2017, pp. 390–396. [11] D. Sacharny and T. Henderson, “A lane-based approach for large-scale strategic conflict management for UAS service suppliers,” in Int. Conf. Unman. Aircraft Sys., Atlanta, GA, Jun. 11–14, 2019, pp. 937–945. [12] M. Johnson and J. Larrow, “UAS traffic management conflict management model,” FAA-NASA UTM Research Transition Team: Sense and Avoid Working Group, Hampton, VA, USA, Tech. Rep., 2020. 173 [13] L. Martin, C. Wolter, K. Jobe, M. Manzano, S. Blandin, M. Cencetti, L. Claudatos, J. Mercer, and J. Homola, “TCL4 UTM (UAS traffic management) Nevada 2019 flight tests, airspace operations laboratory (AOL) report,” National Aeronautics and Space Administration, Hampton, VA, USA, Tech. Rep., Mar. 2020. [14] International Civil Aviation Organization, “Doc 9854 AN/458 - global air traffic management operational concept,” ICAO, Montreal, Canada, Tech. Rep., 2005. [15] ASTM. ASTM UTM protocol (API and sequence diagrams). (2021) https://github. com/astm-utm/Protocol. (accessed Jun. 20 2020). [16] “dss/assets at master · interuss/dss,” https://github.com/interuss/dss/tree/ master/assets, (accessed Dec. 20 2020). [17] D. Bertsimas and S. S. Patterson, “The air traffic flow management problem with enroute capacities,” Oper. Res., vol. 46, no. 3, pp. 406–422, Jun. 1998. [18] J. Rios and J. Lohn, “A comparison of optimization approaches for nationwide traffic flow management,” in Guid. Nav. Cont. Conf. Exh., Reston, Virigina, USA, Aug. 10– 13, 2009. [19] C. H. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity. Mineola, New York, USA: Dover Publications, 1988. [20] M. L. Pinedo, Scheduling: Theory, Algorithms, and Systems, 5th ed. USA: Springer International Publishing, 2016. New York, NY, [21] A. Arisha, P. Young, and M. E. Baradie, “Job shop scheduling problem: An overview,” in Int. Conf. Flex. Autom. Intelli. Manuf., Dublin, Ireland, Jul. 2001, pp. 682–693. [22] A. Jones, L. C. Rabelo, and A. T. Sharawi, “Survey of job shop scheduling techniques,” in Wiley Enclopedia of Electrical and Electronics Engineering, J. G. Webster, Ed., Hoboken, NJ, USA: John Wiley & Sons, Inc., 1999. [23] J. C. Beck and N. Wilson, “Proactive algorithms for scheduling with probabilistic durations,” in Int. Joint Conf. Artif. Intell., vol. 28, Edinburgh, Scotland, Jul. 30 – Aug. 5, 2005, pp. 1201–1206. [24] J. van den Berg, J. Snoeyink, M. Lin, and D. Manocha, “Centralized path planning for multiple robots: Optimal decoupling into sequential plans,” in Robot. Sci. Sys., Seattle, WA, USA, Jun. 28 – Jul. 1, 2009. [25] M. Saha and P. Isto, “Multi-robot motion planning by incremental coordination,” in IEEE Int. Conf. Intell. Robots Syst., Beijing, China, Oct. 9–15, 2006, pp. 5960–5963. [26] J. Rios and K. Ross, “Delay optimization for airspace capacity management with runtime and equity considerations,” in AIAA Guid. Nav. Cont. Conf., Hilton Head, SC, USA, Aug. 20–23, 2007, p. 10. [27] M. Patriksson, The Traffic Assignment Problem: Models and Methods. Mineola, NY, USA: Dover Publications, 2015. 174 [28] V. Bulusu, R. Sengupta, V. Polishchuk, and L. Sedov, “Cooperative and noncooperative UAS traffic volumes,” in Int. Conf. Unman. Aircraft Sys., Miami, FL, USA, Jun. 13–16, 2017, pp. 1673–1681. [29] S. Devasia, M. Heymann, and G. Meyer, “Automation procedures for air traffic management: A token-based approach,” in Proc. Amer. Cont. Conf., Anchorage, AK, USA, May 8–10, 2002, pp. 736–741. [30] S. Devasia, D. Iamratanakul, G. Chatterji, and G. Meyer, “Decoupled conflictresolution procedures for decentralized air traffic control,” IEEE Trans. Intell. Transp. Syst., vol. 12, no. 2, pp. 422–437, Jun. 2011. [31] J. Yoo and S. Devasia, “Provably safe conflict resolution with bounded turn rate for air traffic control,” IEEE Trans. Cont. Sys. Tech., vol. 21, no. 6, pp. 2280–2289, Nov. 2013. [32] M. Barthelemy, Morphogenesis on Spatial Networks. Cham, Switzerland: Springer, 2018. [33] D. Sacharny, T. Henderson, and E. Guo, “A DDDAS protocol for real-time UAS flight coordination,” in InfoSym./Dyn. Data Driven App. Sys. Conf., Boston, MA, USA, Oct. 2–4, 2020, pp. 49–56. [34] M. Treiber and A. Kesting, Traffic Flow Dynamics. Berlin, Germany: Springer Verlag, 2013. [35] F. Ahmadzai, K. Rao, and S. Ulfat, “Assessment and modeling of urban road networks using integrated graph of natural road networks (a GIS-based approach),” J. Urb. Manage., vol. 8, no. 1, pp. 109–125, Apr. 2019. [36] A. Rényi, “On a one-dimensional problem concerning random space filling.” Pub. Math. Inst. Hung. Acad. Sci., vol. 3, pp. 109–127, 1958. [37] J. Rios, “UAS traffic management (UTM) project strategic deconfliction: System requirements final report,” NASA, Moffet Field, CA, Tech. Rep. NASA Report, July 2018. [38] D. Sacharny and T. Henderson, “Optimal policies in complex large-scale UAS traffic management,” in IEEE Conf. Ind. Cyb.-Phys. Sys., Taipei, Taiwan, May 6–9, 2019, pp. 352–357. [39] L. Sedov and V. Polishchuk, “Centralized and distributed UTM in layered airspace,” in Int. Conf. Res. Air Trans., Cataloma, Spain, Jun. 26–29, 2018, pp. 1–8. [40] E. Sunil, J. Hoekstra, J. Ellerbroek, F. Bussink, D. Nieuwenhuisen, A. Vidosavljevic, and S. Kern, “Metropolis: Relating airspace structure and capacity for extreme traffic densities,” in Air Traf. Manage. Research Dev. Sem., Lisbon, Portugal, Jun. 23–26, 2015. [41] S. Balachandran, C. Munoz, and M. C. Consiglio, “Implicitly coordinated detect and avoid capability for safe autonomous operation of small UAS,” in Aviat. Tech. Int. Ops. Conf., Reston, Virginia, USA, Jun. 5–9, 2017. 175 [42] M. R. Jardin, “Analytical relationships between conflict counts and air-traffic density,” J. Guid. Control Dyn., vol. 28, no. 6, pp. 1150–1156, Nov.–Dec. 2005. [43] H. Blom and G. Bakker, “Conflict probability and incrossing probability in air traffic management,” in Proc. 41st Conf. Decis. Control, Las Vegas, NV, USA, Dec. 10–13, 2002, pp. 2421–2426. [44] M. Tra, E. Sunil, J. Ellerbroek, and J. Hoekstra, “Modeling the intrinsic safety of unstructured and layered airspace designs,” in Air Traff. Manage. Research Dev. Sem., Seattle, Washington, USA, Jun. 27–30, 2017. [45] D.-S. Jang, C. A. Ippolito, S. Sankararaman, and V. Stepanyan, “Concepts of airspace structures and system analysis for UAS traffic flows for urban areas,” in Infor. Sys., Grapevine, Texas, USA, Jan. 9–13, 2017. [46] G. Newell, “A simplified car-following theory: A lower order model,” Trans. Res. B Methodol., vol. 36, no. 3, pp. 195–205, Mar. 2002. [47] S. M. LaValle, Planning Algorithms. 2006. Cambridge, U.K.: Cambridge University Press, [48] D. Alejo, J. M. Dı́az-Bá ñez, J. A. Cobano, P. Pérez-Lantero, and A. Ollero, “The velocity assignment problem for conflict resolution with multiple aerial vehicles sharing airspace,” J. Intell. Robot. Sys., vol. 69, no. 1-4, pp. 331–346, Jan. 2013. [49] H. Choset, K. Lynch, S. Hutchinson, G. Kantor, W. Burgard, L. Kavraki, and S. Thrun, Principles of Robot Motion Theory, Algorithms, and Implementation. Cambridge, Massachusetts, USA: MIT Press, 2005. [50] H. D. Friedman, D. Rothman, and J. K. Mackenzie, “Solution to: An unfriendly seating arrangement (problem 62–3),” SIAM Rev., vol. 6, no. 2, pp. 180–182, Apr. 1964. [51] G. Neuhauser, A. R. Kan, and M. Todd, Optimization. New York, NY, USA: NorthHolland, 1989. [52] I. Zelinkaet, V. snasel, and A. Abraham, Handbook of Optimization. Berlin, Germany: Springer Verlag, 2013. [53] F. A. Administration, “Instrument procedures handbook,” Federal Aviation Administration, Washington, DC, USA, Tech. Rep. FAA-H-8083-16B, August 2017. [54] D. Thipphavong, R. Apaza, B. Barmore, V. Battiste, B. Burian, Q. Dao, M. Feary, S. Go, K. Goodrich, J. Homola, H. Idris, P. Kopardekar, J. Lachter, N. Neogi, H. Ng, R. Oseguera-Lohr, M. Patterson, and S. Verma, “Urban air mobility airspace integration concepts and considerations,” in Aviat. Tech. Int. Ops. Conf., Atlanta, GA, USA, Jun. 25–29, 2018. [55] National Research Council (U.S.). Transportation Research Board., Highway Capacity Manual. Washington, D.C., USA: Transportation Research Board, National Research Council, 2000. 176 [56] M. P. Clay and N. Simanyi, “Renyi’s parking problem revisited,” Stoch. Dyn., vol. 16, pp. 1–12, Jun. 2014. [57] M. Gargano, A. Weissenseel, J. Malerba, and M. Lewinter, “Discrete Renyi parking constants,” Pace University, New York, NY, USA, Tech. Rep., May 2005. [58] AirMap Inc., “Five critical enablers for safe, efficient, and viable UAS traffic management (UTM),” AirMap Inc., Santa Monica, CA, USA, Tech. Rep., 2018. [59] D. Sacharny, T. Henderson, M. Cline, and B. Russon, “Reinforcement learning at the cognitive level in a belief, desire, intention UAS agent,” University of Utah, Salt Lake City, UT, USA, Tech. Rep. UUCS-20-013, 2020. [60] D. Sacharny, T. Henderson, and M. Cline, “Large-scale UAS traffic management (UTM) structure,” in IEEE Multisens. Fusion Int. Conf., Karlsruhe, Germany, Sep. 14– 16, 2020, pp. 7–12. [61] D. Sacharny, T. Henderson, M. Cline, B. Russon, and E. Guo, “FAA-NASA vs. lanebased strategic deconfliction,” in IEEE Multisens. Fusion Int. Conf., Karlsruhe, Germany, Sep. 14–16, 2020, pp. 13–18. [62] Texas A&M Transportation Institute, “Traffic management centers,” https: // static.tti.tamu.edu/tti.tamu.edu/documents/policy/congestion-mitigation/trafficmanagement-centers.pdf, (accessed Dec. 20, 2021). [63] German Federal Bureau of Aircraft Accidents Investigation, “Investigation report,” Bundesstelle für Flugunfalluntersuchung, Hermann-Blenk-Str. 16, 38108 Braunschweig, Tech. Rep., May 2004. [64] M. J. Kochenderfer, C. Amato, G. Chowdhary, J. P. How, H. J. D. Reynolds, J. R. Thornton, P. A. Torres-Carrasquillo, N. K. Üre, and J. Vian, Decision Making Under Uncertainty: Theory and Application, 1st ed. Cambridge, MA, USA: The MIT Press, 2015. [65] E. Dietrich and A. B. Markman, Cognitive Dynamics: Conceptual and Representational Change in Humans and Machines. New York, NY, USA: Psychology Press, 2014. [66] T. C. Henderson, R. Simmons, B. Serbinowski, M. Cline, D. Sacharny, X. Fan, and A. Mitiche, “Probabilistic sentence satisfiability: An approach to PSAT,” Artif. Intell., vol. 278, Nov. 2020. [67] M. Kothari, I. Postlewaite, and D.-W. Gu, “UAV path following in windy urban environments,” J. Int. Robot. Sys., vol. 74, pp. 1013–1028, Aug. 2014. [68] L. Swartzentruber, “Improving path planning of unmanned aerial vehicles in an immersive environment using MetaPaths and terrain information,” Ph.D. dissertation, Dept. Mech. Eng., Iowa State Univ., Ames, IA, USA, 2009. [69] N. Nilsson, “Probabilistic logic,” Art. Intell. J., vol. 28, no. 1, pp. 71–87, Feb. 1986. [70] G. Boole, “Further observations on the theory of probabilities,” Phil. Mag. J. Sci., vol. 2, no. 9, pp. 96–101, 1851. 177 [71] T. Henderson, A. Mitiche, R. Simmons, and X. Fan, “A preliminary study of probabilistic argumentation,” University of Utah, Salt Lake City, UT, USA, Tech. Rep. UUCS-17-001, Feb. 2017. [72] D. Sacharny, T. Henderson, A. Mitiche, R. Simmons, T. Welker, and X. Fan, “BRECCIA: A Multi-Agent Data Fusion and Decision Support System for Dynamic Mission Planning,” in 2nd Conf. Dyn. Data Driven Appl. Syst., Cambridge, MA, USA, Aug. 7–9, 2017. [73] R. Bordini, J. Huebner, and M. Wooldridge, Programming Multi-Agent Systems in AgentSpeak Using Jason. Hoboken, NJ, USA: Wiley, 2007. [74] A. Rao, “Agentspeack(l): BDI agents speak out in a logical computable language,” in Proc. Mod. Auton. Agents Multi-Agent World, Valencia, Spain, Jun. 30 – Jul. 2, 1996, pp. 42–55. [75] G. Oriz-Hernandez, J. Huebner, R. Bordini, A. Guerra-Hernandez, G. Hoyos-Rivera, and N. Cruz-Ramirez, “A namespace approach for modularity in BDI programming languages,” in Eng. Multi-Agent Sys., M. Baldoni, J. Mueller, I. Nunes, and R. ZalilaWenkstern, Eds. Singapore: Springer International Publishing, 2016, pp. 117–135. [76] V. Gogate and P. Domingos, “Probabilistic theorem proving,” Commun. ACM, vol. 59, no. 7, pp. 107–115, Jul. 2016. [77] M. Richardson and P. Domingos, “Markov logic networks,” Mach. Learn., vol. 62, no. 1–2, pp. 107–136, Feb. 2006. [78] T. Alsinet, C. Chesnevar, L. Godo, and G. Simari, “A logic programming framework for possibilistic argumentation,” Fuzzy Set Syst., vol. 159, no. 10, pp. 1208–1228, Jan. 2008. [79] P. Domingos and D. Lowd, Markov Logic: An Interface Layer for Artificial Intelligence. San Rafael, CA, USA: Morgan and Claypool, 2009. [80] A. Hunter, “A probabilistic approach to modelling uncertain logical arguments,” Int. J. Approx. Reason, vol. 54, no. 1, pp. 47–81, Jan. 2013. [81] H. Li, N. Oren, and T. Norman, “Probabilistic argumentation frameworks,” in Proc. Int. Theory App. Form. Arg., Barcelona, Spain, Jul. 16–17, 2011, pp. 1–16. [82] L. D. Raedt, A. Kimmig, and H. Toivonen, “ProbLog: A probabilistic Prolog and its application in link discovery,” in Int. Conf. Art. Intell., Hyderabad, India, Jan. 6–12, 2007, pp. 2462–2467. [83] M. Thimm, “A probabilistic semantics for abstract argumentation,” in Proc. Conf. Art. Intell., Montpellier, France, Aug. 27–31, 2012, pp. 750–755. [84] M. Biba, “Integrating logic and probability: Algorithmic improvements in markov logic networks,” Ph.D. dissertation, Dept. Comput. Sci., Univ. of Bari, Bari, Italy, 2009. [85] G. Boole, An Investigation of the Laws of Thought. London, UK: Walton and Maberly, 1857. 178 [86] R. Kowalski and P. Hayes, “Semantic trees in automatic theorem proving,” in Automation of Reasoning, J. Siekmann and G. Wrightson, Eds. Berlin, Germany: Springer, 1983, pp. 217–232. [87] B. Boehm, C. Abts, and S. Chulani, “Software development cost estimation approaches – a survey,” Ann. Softw. Eng., vol. 10, no. 1, pp. 177–205, Nov. 2000. [88] J. Baculi and C. Ippolito, “Onboard decision-making for nominal and contingency sUAS flight,” in AIAA Scitech Forum, San Diego, CA, USA, Jan. 7–11, 2019. [89] M. Georgeff, B. Pell, M. Pollack, M. Tambe, and M. Wooldridge, “The Belief-DesireIntention Model of Agency,” in Intelligent Agents V: Agents Theories, Architectures, and Languages, J. P. Müller, A. S. Rao, and M. P. Singh, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 1999, pp. 1–10. [90] G. Rens and D. Moodley, “A hybrid POMDP-BDI agent architecture with online stochastic planning and plan caching,” Cog. Syst. Res., vol. 43, pp. 1–20, Jun. 2017. [91] M. Schut and M. Wooldridge, “Principles of intention reconsideration,” in Proc. Int. Conf. Auton. Agents, Montreal, Canada, May 2001, pp. 340–347. [92] M. Schut, M. Wooldridge, and S. Parsons, “On partially observable MDPs and BDI models,” in Foundations and Applications of Multi-Agent Systems, M. d’Inverno, M. Luck, M. Fisher, and C. Preist, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2002, pp. 243–259. [93] G. I. Simari and S. Parsons, “On the relationship between MDPs and the BDI architecture,” in Proc. Int. Conf. Auton. Agents Multiagent Sys., Hakodate, Japan, May 8–12, 2006, pp. 1041–1048. [94] S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach. Upper Saddle River, NJ, USA: Prentice-Hall, 2001. [95] P. Kopardekar, J. Rios, T. Prevot, M. Johnson, J. Jung, and J. R. III, “Unmanned aircraft system traffic management (UTM) concept of operations,” in Aviation Tech. Int. Ops. Conf., Washington, DC, USA, Jun. 13–17, 2016. [96] NASA, “Unmanned aircraft systems (UAS) traffic management (UTM) concept of operations, v2.0,” Federal Aviation Administration, Washington, DC, USA, Tech. Rep., March 2020. [97] F. Darema, “Dynamic data driven applications systems: A new paradigm for application simulations and measurements,” in Int. Conf. Comput. Sci., Krakow, Poland, Jun. 6–9, 2004, pp. 662–669. [98] J. A. Fax and R. M. Murray, “Information flow and cooperative control of vehicle formations,” IEEE Trans. Autom. Control, vol. 49, no. 9, pp. 1465–1476, Sep. 2004. [99] A. Cherniak, “Exploring behavioral patterns in complex adaptive systems,” Ph.D. dissertation, School Inform. Sci., Univ. Pittsburgh, Pittsburgh, PA, USA, 2014. [100] A. Namatame and S.-H. Chen, Agent Based Modeling and Network Dynamics. Oxford, UK: Oxford University Press, 2016. 179 [101] S. F. Railsback and V. Grimm, Agent-Based and Individual-Based Modeling. Princeton, NJ, USA: Princeton University Press, 2012. [102] D. Easley and J. Kleinberg, Networks, Crowds, and Markets: Reasoning About a Highly Connected World. Cambridge, UK: Cambridge University Press, 2010. [103] K. Nagel and S. Rasmussen, “Traffic at the edge of chaos,” in 4th Int. Workshop Synth. Simul. Living Syst., Cambridge, MA, USA, Jul. 6–8, 1994, pp. 222–235. [104] A. Schadschneider and M. Schreckenberg, “Cellular automation models and traffic flow,” J. Phys. A, vol. 26, no. 15, pp. L679–L683, Aug. 1993. [105] R. Alur, “Formal verification of hybrid systems,” in Proc. ACM Intl. Conf. Embedded Softw., Taipei, Taiwan, Oct. 9–14, 2011, pp. 273–278. [106] S. Russell, D. Dewey, and M. Tegmark, “Research priorities for robust and beneficial artificial intelligence,” AI Magazine, vol. 36, no. 4, pp. 105–114, Dec. 2015. [107] A. S. Vezhnevets, S. Osindero, T. Schaul, N. Heess, M. Jaderberg, D. Silver, and K. Kavukcuoglu, “FeUdal networks for hierarchical reinforcement learning,” in Proc. Int. Conf. Mach. Learn., Sydney, Australia, Aug. 6–11, 2017, pp. 3540–3549. [108] J. B. Kenney, “Dedicated short-range communications (DSRC) standards in the United States,” Proc. IEEE, vol. 99, no. 7, pp. 1162–1182, Jul. 2011. [109] D. Sacharny and C. Liu, “Strategic deployment of drone centers and fleet size planning for drone delivery,” Utah Department of Transportation, Salt Lake City, UT, USA, Rep. UT-21.33, 2021. [110] A.J. Lohn, “What’s the buzz? the city-scale impact of drone delivery,” RAND Corporation, Santa Monica, CA, USA, Rep. RR-1718-RC, 2017. [111] GeoServer, http://geoserver.org/, (accessed Jun. 10, 2020). [112] Utah Geospatial Resource Center. “Projected populations and households for TAZs”. (2021). Distributed by Utah Geospatial Resource Center. Salt Lake City, UT, USA. https://gis.utah.gov/data/demographic/projections/#populationhouseholdstaz. [113] DER SPIEGEL, “Deutsche erhalten mehr Pakete als andere Nationens,” https://www.spiegel.de/wirtschaft/unternehmen/ pakete-in-deutschland-wird-mehr-verschickt-als-in-anderen-nationen-a-1272662. html, (accessed Dec. 10 2019). [114] Utah Geospatial Resource Center. “Zip code PO boxes”. (2021) Distributed by Utah Geospatial Resource Center. Salt Lake City, UT, USA. https://gis.utah.gov/data/ location/u-s-postal-service/#ZipCodePOBoxes. [115] ——. “Utah roads”. (2021). Distributed by Utah Geospatial Resource Center. Salt Lake City, UT, USA. https://gis.utah.gov/data/transportation/roads-system/. [116] ——. “Building footprints”. (2021). Distributed by Utah Geospatial Resource Center. Salt Lake City, UT, USA. https://gis.utah.gov/data/location/building-footprint. 180 [117] N. Hansen, Y. Akimoto, and P. Baudis, “CMA-ES/pycma on Github,” https://doi. org/10.5281/zenodo.2559634, (accessed Feb. 20 2019). [118] R. D’Andrea, “Can drones deliver?” IEEE Trans. Autom. Sci. Eng., vol. 11, no. 3, pp. 647–648, Jul. 2014. [119] “CZML Guide AnalyticalGraphicsInc/czml-writer Wiki,” https://github.com/ AnalyticalGraphicsInc/czml-writer/wiki/CZML-Guide, (accessed Jun. 3 2020).