Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
An application of aerial drones in high definition mapping for autonomous vehicles
(USC Thesis Other)
An application of aerial drones in high definition mapping for autonomous vehicles
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
An Application of Aerial Drones in High Definition Mapping
for Autonomous Vehicles
by
Victoria Scherelis
A Thesis Presented to the
Faculty of the USC Graduate School
University of Southern California
In Partial Fulfillment of the
Requirements for the Degree
Master of Science
(Geographic Information Science and Technology)
December 2019
Copyright © 2019 by Victoria Scherelis
To my family
iv
Table of Contents
List of Figures ................................................................................................................................ vi
List of Tables ............................................................................................................................... viii
Acknowledgements ........................................................................................................................ ix
List of Abbreviations ...................................................................................................................... x
Abstract .......................................................................................................................................... xi
Chapter 1 Introduction .................................................................................................................... 1
1.1. Research Question and Objectives......................................................................................1
1.2. Motivation ...........................................................................................................................2
1.3. Study Sites ..........................................................................................................................4
1.3.1. The Bertrandt Parking Lot .........................................................................................4
1.3.2. The German International School Parking Lot ..........................................................5
1.4. Thesis Organization ............................................................................................................6
Chapter 2 Related Work .................................................................................................................. 8
2.1. Advancement Towards Autonomous Vehicles ...................................................................8
2.2. Autonomous Vehicles in Parking Structures ....................................................................10
2.3. Characteristics of High Definition Maps ..........................................................................10
2.4. Navigation Data Standard (NDS) .....................................................................................13
2.5. UAV Application in Remote Sensing ...............................................................................15
Chapter 3 Methods ........................................................................................................................ 16
3.1. Determination of Prototype HD Data Standards and Structure ........................................18
3.2. Collection and Preparation of Drone Imagery ..................................................................21
3.2.1. Bertrandt Parking Lot Data ......................................................................................22
3.2.2. German School Parking Lot Data ............................................................................24
3.3. Manual Delineation of Required Features on Orthomosaics ............................................26
v
3.4. Tool-based Extraction .......................................................................................................29
3.4.1. Main Model ..............................................................................................................32
3.4.2. Extension Models .....................................................................................................35
3.5. Validation Methods ...........................................................................................................39
Chapter 4 Results .......................................................................................................................... 40
4.1. Assessment of Drone-generated Images and Orthomosaics .............................................40
4.2. Manual Delineation Results ..............................................................................................47
4.3. Tool-based Extraction Results ..........................................................................................50
4.4. NDS Conversion ...............................................................................................................57
Chapter 5 Discussion and Conclusion .......................................................................................... 60
5.1. Current HD Data Standards and the Drone-generated HD Datasets ................................60
5.2. Limitations and Challenges in Drone-based Data Collection ...........................................62
5.3. Proprietary Information ....................................................................................................64
5.4. The Feature Extraction Processes .....................................................................................64
5.5. Future Research ................................................................................................................67
5.6. Final Conclusions..............................................................................................................69
References ..................................................................................................................................... 70
Appendix A: Enlarged Version of the NDS Building Blocks ...................................................... 73
Appendix B: Questions and Answers from HERE Technologies ................................................ 74
Appendix C: Bertrandt Parking Lot; Pix4D Quality Report ......................................................... 76
Appendix D: German School Parking Lot; Pix4D Quality Report............................................... 85
vi
List of Figures
Figure 1 The Bertrandt parking lot: first study site. ........................................................................ 5
Figure 2 The German International School parking lot; second study site. .................................... 6
Figure 3 Overview of the NDS building blocks ........................................................................... 14
Figure 4 Thesis project workflow ................................................................................................ 16
Figure 5 Pix4D mapper output showing the flight path for the first study site ............................ 23
Figure 6 Pix4D mapper output showing the flight path for the second study site ........................ 24
Figure 7 Before (left) and after (right) snapshots of vegetation removal. .................................... 26
Figure 8 Illustration of boundary exclusion in manual delineation. ............................................. 28
Figure 9 Illustration of the Multipart to Singlepart tool. .............................................................. 29
Figure 10 Main ArcGIS model: Orthomosaic to polygon feature classes .................................... 33
Figure 11 Bertrandt extension model: Polygon to line feature class ............................................ 36
Figure 12 German school extension model: Polygon to line feature class ................................... 37
Figure 13 “Auto-Complete Polygon” tool demonstration. ........................................................... 38
Figure 14 Image overlap results. Study site 1 (left), study site 2 (right). ..................................... 42
Figure 15 Bertrandt parking lot orthomosaic; study site 1 ........................................................... 44
Figure 16 German School parking lot orthomosaic; study site 2 .................................................. 46
Figure 17 Manual delineation; final representation of study site 1. ............................................. 48
Figure 18 Manual delineation; final representation of study site 2 .............................................. 49
Figure 19 Main model results for the Bertrandt study site 1 ........................................................ 52
Figure 20 Bertrandt extension model results; final representation of study site 1........................ 53
Figure 21 Main model result for the German School study site 2 ................................................ 55
Figure 22 German School extension model result; final representation of study site 2 ............... 56
Figure 23 Bertrandt parking lot in NDS format ............................................................................ 58
Figure 24 German School parking lot in NDS format .................................................................. 59
vii
Figure 25 Distortion in the northwest corner of the Bertrandt parking lot orthomosaic .............. 63
Figure 26 Illustration of parking spot challenge from vehicles extraction ................................... 66
viii
List of Tables
Table 1 Prototype HD data standards based on communication with Here, Inc. ......................... 19
Table 2 Prototype HD data structure for conversion to NDS ....................................................... 21
Table 3 Horizontal and vertical accuracy of ground control points........ ...................................... 25
Table 4 Name and description of all geoprocessing tools applied ................................................ 30
Table 5 Tool input parameters ...................................................................................................... 31
Table 6 Time required for orthomosaic generation by Phantom 4 drone ..................................... 41
Table 7 Summary of Pix4D processing results ............................................................................. 41
Table 8 Error assesment calculated as the difference between initial and computed positions ... 43
Table 9 Time required for manual delineation ............................................................................. 47
Table 10 Tool and model component runtime for tool-based extraction method. ........................ 50
Table 11 Data collection by mapping-vehicles vs. by aerial drone .............................................. 61
ix
Acknowledgements
I sincerely thank my thesis advisor, Karen Kemp, Ph.D, for her continuous guidance and support.
I would also like to thank Andrew Marx, Ph.D, and Robert Vos, Ph.D, for the advice and support
that helped build the foundation for this research project, as well as Steve Fleming, Ph.D, for his
participation on my thesis committee. Many thanks as well to my employer, Bertrandt Inc., who
supported this research project in more ways than one. In particular, I would like to thank my
mentors at Bertrandt, Inc., Lisa Gillman and Norman Timtschuk for their support and guidance
that made this project possible. I am grateful for the information provided to me by HERE
technologies and would like to thank Timm Kayser and Zaki Nassif Garcia for making that
possible. Lastly, I would like to thank Andreas Pehlke from Bertrandt, Inc., for his continuous
communication with me regarding NDS information and his efforts invested in this project by
converting the outcomes to the NDS format.
x
List of Abbreviations
AAT Automatic aerial triangulation
ADAS Advanced driver-assistance systems
BBA Bundle block adjustment
BMD Basic map display
EPoSS European technology platform on smart systems integration
GB Gigabyte
GCP Ground control point
GIS Geographic information system
GISP German International School of Portland
GSD Ground sampling distance
GPS Global positioning system
HD High Definition
IMU Inertial measurement unit
JSON Javascript object notation
LiDAR Light detection and ranging
NDS Navigation data standard
TB Terabyte
UAV Unmanned aerial vehicle
xi
Abstract
The future of the automotive industry continues to head towards the development of autonomous
vehicles. Without a human driver behind the wheel, the self-driving vehicle must be able to
navigate itself within the road network. This research project investigates the application of
aerial drones, also known as unmanned aerial vehicles (UAVs), as an alternative data collection
method to create HD datasets for use in autonomous vehicles. Drones may be a low-cost
alternative method to the current leading data collection method of sensor-equipped mapping-
vehicles. A Phantom 4 drone was used in two case studies to create orthomosaics of parking lots.
The drone-generated orthomosaics were processed by methods of manual delineation and tool-
based extraction to evaluate different methods of processing high-resolution data. In addition,
current HD data standards were acquired from various sources to evaluate the results of the
research project and to compare data collection methods. The results show that drone-based data
collection with GPS correction techniques can be an accurate and low-cost alternative method.
Both manual delineation and tool-based extraction techniques proved successful in extracting
desired feature classes from the high-resolution imagery.
1
Chapter 1 Introduction
The future of the automotive industry heads towards self-driving, fully autonomous, vehicles.
Many variables must be taken into consideration for autonomous vehicles to become a reality,
such as the production of high-definition (HD) maps. To navigate autonomously within a road
network, current maps within a vehicles’ navigation system do not have the precision and
accuracy needed to replace the driver’s eyes on the road (Automotive World 2018). For example,
most navigation systems symbolize the road driven as a single line segment, even if the line
segment is a six-lane highway.
Semi-autonomous cars already exist and are bristling with sensors that help navigate the
vehicle within a lane, however, these sensors only assist the driver and cannot replace the human
behind the wheel (Hyatt 2018). To support these autonomous vehicles, high definition (HD)
maps are built specifically for autonomous and semi-autonomous vehicles with high precision
and detail and, ideally, centimeter level accuracy to ensure the vehicle stays within the lane
(Vardhan 2017). Companies such as Here Inc., Lyft, and TomTom, among others, presently
provide HD mapping services to the automotive industry by the use of sensor-equipped mapping
vehicles (Kent 2015). These mapping vehicles may be the industry’s leading mapping method;
however, the services and sensors are expensive, and a driver is needed for every step of the way.
1.1. Research Question and Objectives
The main research question to be answered is: Can aerial drones be used as an alternative
data collection method to provide drone generated imagery that can be used for the development
of HD datasets for autonomous vehicles? The subsidiary research question to be answered is:
What processing method of the drone-generated imagery renders the best results? To answer
these questions, the objectives of this research study are to: (1) evaluate the accuracy and overall
2
quality of drone-generated orthomosaics from two case studies; (2) evaluate and compare manual
and tool-based processing methods of drone-generated orthomosaics in the production of HD
datasets; and, (3) compare the resulting datasets from the two case studies to current HD data
standards in terms of the time required for data collection, the resulting data structure, and the
quality and efficiency of data collection by means of aerial drones vs. traditional methods.
1.2. Motivation
Self-driving vehicles would not only allow the individual to lean back and relax while the
auto-pilot chauffeurs, but it may lead to safer roads and a decline in automotive fatalities as well.
According to the Insurance Institute for Highway Safety, 37,133 people died in motor vehicle
crashes in the U.S in 2017 with a total of 34,247 crashes involving 52,645 vehicles (IIHS 2018).
Globally, vehicle crashes account for 1.25 million deaths and 20 to 50 million injuries every year
(CDC 2017). Handing the control over to the vehicle may reduce the amount of vehicle-related
deaths as the computer in the autonomous vehicle does not experience human traits such as
drowsiness or impairments due to drugs or alcohol. In 2016, 10,497 people died due to alcohol-
impaired driving crashes, accounting for 28% of all traffic-related deaths in the Unites States
(CDC 2019).
In addition to vehicle crashes due to impaired driving, a large percent of vehicle crashes
are due to distracted driving habits. The distractions in the vehicles continuously grow as new
technologies are introduced to the vehicles and as individuals use commuting time for additional
activities such as eating food, watching movies, playing games, or texting on the cellphone.
According to the U.S Department of Transportation, nine percent of U.S fatal crashes in 2016
were distraction-affected crashes, accounting for 3,450 deaths in motor vehicle crashes by
distracted drivers (NHTSA 2018). Autonomous vehicles would make commuting time safer and
3
also more efficient as it would allow the individual to pay attention to other activities, all while
the vehicle safely transports the individual to the destination.
Implementing a fleet of drones to create high definition maps for autonomous vehicles
may also be beneficial to the environment. The mapping-vehicles used to create HD maps are
required to drive each stretch of road multiple times to create high-quality data, burning fossil
fuels in the process. Considering that the goal is to create an HD map network for all roads, the
amount of fossil fuel burned to create such an HD road network comes with a cost to the
environment. Instead of expanding our carbon footprint, drones with rechargeable lithium-
powered batteries would be able to map without burning fossil fuel.
The production of an HD map network is also expensive. According to the artificial
intelligence and industry review magazine Synced, the US rising star in HD mapping, DeepMap,
charges $5,000 per kilometer for its services in the US (Synced 2018). Creating a large HD map
at such cost would require large investments by companies, most likely only attracting large
companies that could afford it. Another option for smaller companies may be to create mapping-
vehicles themselves, instead of paying for the services of other companies. As mapping-vehicles
are bristling of sensors, structures to mount the sensors, and require high-processing powered
computers and software, ultimately the cost of creating an HD map is still very high. Utilizing
drones as an alternative HD mapping technique may save some money. A good quality drone
such as the DJI Phantom 4 pro costs about $1,500 brand new, according to the DJI sales website.
In Germany, a vehicle often used for mapping purposes is the Volkswagen Passat (Dolgov and
Thrun 2009). According to the VW sales website, a Passat costs about $25,000 brand new. Even
if all sensors would cost the same for the drone and the vehicle, the drone is still the more cost-
efficient option.
4
1.3. Study Sites
The first steps taken in the autonomous vehicle network will occur within parking lots
and highways, as these environments are less complex than city intersections and other fast-
moving environments (Dokic et al. 2015). The introduction of autonomous vehicles will start
slowly, with vehicles parking themselves in parking lots and vehicles driving down highways in
auto-pilot. The two study sites chosen for this study are therefore two parking lots.
1.3.1. The Bertrandt Parking Lot
As the European automotive industry has great interest in HD map research, the
engineering services company named Bertrandt supports this research study. To incorporate a
parking structure in Germany, the first selected study site is the main parking lot of the Bertrandt
company campus in Tappenbeck, Germany. Although the parking lot does not have lane
separations, other components such as road (drivable surface), parking area, and parking spots
(non-drivable surfaces) are included at this site. The Bertrandt parking lot is approximately
0.0183 km
2
(4.53 acres) and is located between fields and by residential houses (see Figure 1).
Three ground control points were established from previously recorded benchmarks in the
parking area. Due to Germany’s very strict UAV guidelines and regulations, a drone flight was
approved for a one-week window, January 7
th
-January 11
th
, 2019. The Bertrandt parking lot was
utilized as one of two case studies to create a drone-generated map.
5
Figure 1. The Bertrandt parking lot; first study site. Source: satellites.pro
1.3.2. The German International School Parking Lot
The second case study was the parking lot of the German International School of Portland
(GISP), located in Beaverton, Oregon (see Figure 2). The choice of location of the case studies
was rather arbitrary, as long as the chosen study sites were parking lots and accessible to drone
flights, they were suitable. Thus, the GISP parking lot was chosen due to the fact that the school
is privately-owned, and the headmaster granted permission for drone flights of the property. The
GISP parking lot was flown in May 2019 and is approximately 0.0106 km
2
(2.6 acres). To
acquire the best absolute accuracy, four ground control points were set within the parking lot.
6
Figure 2. The German International School parking lot; second study site. Source:
maps.google.com
1.4. Thesis Organization
This thesis includes four additional chapters. The next chapter provides a literature
review to understand current advancements in the automotive industry and to give a background
on high definition maps, their structure, and all they encompass. The next chapter also provides
information on current UAV use as a remote sensing technique. Chapter 3 describes in detail the
methods used to complete this research study, including the drone-generated data produced and
used for orthomosaic generation, the feature extraction methods applied to the orthomosaics, and
evaluation of the resulting datasets. Chapter 4 provides the results of the study. Chapter 5
7
discusses the conclusions drawn from the results, potential future research of this study, and also
discusses any limitations faced.
8
Chapter 2 Related Work
As the automotive industry in Europe, China, and the US is particularly eager for the emergence
of autonomous vehicles, abundant research has been done to explore the production of HD maps
and autonomous vehicles. The current research, however, focuses heavily on HD map production
by ground vehicles only. Other remote sensing methods, such as drones with imaging devices,
can be used for map production as well and there is abundant research on their use within certain
scientific fields, such as in agriculture and environmental sciences. This research study aims to
bridge the gap between the production of HD maps for autonomous vehicles and the use of
drones in certain scientific fields.
2.1. Advancement Towards Autonomous Vehicles
European automotive manufacturers and their suppliers have successfully introduced new
smart components and systems, such as advanced driver assistance systems (ADAS), to the
European high technology industry. ADAS are systems that help the driver in the driving process
and include technological components such as collision avoidance, adaptive cruise control, and
lane departure warning systems. These driver-assistant components have been technological
breakthroughs within the automotive industry as they enabled road and passenger safety, energy
efficiency, and emission reduction (Dokic et al. 2015).
According to the European Technology Platform on Smart Systems Integration (EPoSS),
the introduction of autonomous vehicles is a feasible goal for the near future with milestones set
at 2020, 2025, and 2030. By the first milestone of 2020, parking lot and traffic jam situations
should be manageable by automated vehicles driving at low velocities. By 2025, highway
autopilots should be introduced and by 2030 highly automated driving within cities with
complex traffic structures will be possible (Dokic et al. 2015).
9
Besides the timeframe, different automation levels exist and range from level 0 (human
driver has full control) to level 5 (fully autonomous vehicle) (Van Brummelen et al. 2018). As
parking lot and traffic jam navigation are milestone 2020 goals, the automation level requirement
is level 3 (conditional automation) in which the car is aware of its surroundings and can handle
independently for a certain amount of time. This automation level can be found in the Tesla
model X and S (Dokic et al. 2015; Van Brummelen et al. 2018). As the focus for the near future
is on parking lots and traffic jams, the HD map generation of those parking lots will allow faster
advancements of higher automation levels for navigation within parking structures.
Challenges facing the introduction of autonomous vehicles are many, as the higher-
automation level requires more and better sensors on the vehicle, larger data storage space, and
the maps must be upgraded frequently to provide sufficient information. A fully autonomous
vehicle requires sensors such as sonar devices, stereo camera, lasers, radars, and highly accurate
GPS to compare to the five human senses (Seif and Hu 2016). A LiDAR sensor would be an
important sensor as it senses objects in the near-environment of the car with a high accuracy up
to a range of 100 m and a rotational ability of 360 degrees. At a cost of $4,000 per sensor, the
LiDAR sensors are some of the most expensive sensors on the vehicle (Randall 2019). LiDAR
sensors used by mapping-vehicles, such as the Velodyen top-end HDL-64E, retail at about
$100,000.
Besides the financial aspect of such expensive equipment, another challenge is the data
collection from the sensors as one hour of drive time produces one terabyte of data and takes two
days to process by high computing power (Synced 2018; Seif and Hu 2016). One solution to the
current challenges of data collection, processing, and storage, is to consider all autonomous
vehicles as part of the infrastructure of a future traffic system. This future traffic system would
10
consist of the autonomous vehicles, roadside units, HD maps, and high-performance computing
and storage for cloud services (Seif and Hu 2016).
2.2. Autonomous Vehicles in Parking Structures
Current research in the field of autonomous driving focuses mostly on highly structured
environments, such as highways or cities, or on unstructured environments, such as off-road
driving. In highly structured environments, it is assumed that a topological graph, or lane-
network graph, exists over the environment to which the vehicle is constrained to drive on with
little to no deviations (Dolgov and Thrun 2009). For unstructured environments, the vehicle is
not constrained to a topological graph and can freely choose a path, considering safety and other
constraints. Parking structures fall in to the semi-structured category where a topological graph
structure exists, but maneuvers off the graph are valid (Dolgov and Thrun 2009). Current
research focuses on the use of topological graphs within these semi-structured environments to
see if they benefit the vehicle in path planning or not. Results show that predetermined lane-
networks (topological graphs) do, indeed, benefit the vehicle in path planning through a parking
structure opposed to free-space path planning (Dolgov and Thrun 2009). For largescale multi-
level structures such as parking garages, the approach of a predetermined path has also shown to
be beneficial. In one study, based on surface maps of the corresponding environment and a
calculated path through the parking garage, an autonomous vehicle was able to completely park
itself within the parking garage (Kummerle et al. 2009).
2.3. Characteristics of High Definition Maps
Traditional maps used for navigation mainly serve visualization purposes and do not
have the requirements needed for autonomous vehicles as they lack the accurate lane geometries
(Massow et al. 2009). Maps particularly built for self-driving purposes are commonly referred to
11
as high definition maps or HD maps. These HD maps are extremely precise and contain a lot of
information as the robots need precise instructions on how to maneuver within the 3D space
(Vardhan 2017). To meet the need of higher quality maps, new HD map formats are emerging
from mapping services companies such as TomTom and Here, Inc.. Some standardized map
formats for specific companies already exist, such as the HD live map from Here and the highly
automated driving (HAD) map from TomTom (Kent 2015; TomTom 2015; Massow et al. 2009).
Although HD map developers such as TomTom, Here, and Lyft, among others, are
working towards a standardized format for all HD datasets, the exact format is currently still a
fluid concept. The TomTom HD map consists of layers including lane models, traffic signs, road
furniture, and lane geometry (TomTom 2018). Lyft, an on-demand transportation company,
organized their HD map into five layers including the real-time layer, map priors layer, semantic
map layer, geometric map layer, and the base map layer (Chellapilla 2018). The map priors layer
shows locations where the behavior of objects (e.g. timing and sequence of traffic lights), people
(e.g. bicyclists in the driving lane) and other vehicles (e.g. places where left turns are common)
impact simple navigation decisions. The HD live map specifications from Here, Inc. structure the
HD map into two major models known as the lane model (group 1) and road centerline model
(group 2). These models are further split into a lane topology and geometry model (group 1),
lane attribute model (group 1), link topology-geometry model (group 2), and road attributes
model (group 2). The lane model from Here, Inc. is based on the topology of individual lanes,
lane groups, and lane group connectors. It includes lane boundaries and lane paths as well as
their lane-level attributes (Here 2018). The road centerline model is based on links and nodes,
2D geometry of polylines and shape points, and the attributes (Here 2018). With such differing
12
data structures and map specifications, an exact standard of the components an HD dataset must
include has not yet been established.
Although the exact format specifications between HD datasets from different companies
may still vary, all consider an HD dataset to comprise of multiple layers that place the vehicle
precisely in a lane with information on road signs and markings in the vehicle’s immediate
surroundings. With varying data structures between different mapping and automotive
companies, defining the data structure for an HD dataset is difficult.
To find common ground, a standard structure for the data would be ideal. As described in
Massow et al. (2016), the infrastructure of an HD map can be generalized to contain three major
layers consisting of 1) dynamic data, 2) road furniture data, and 3) road geometry data. The
dynamic data includes up-to-date information on current incidents, hazards, and events such as
construction areas or accidents (Massow et al. 2016). For example, the HD live map from Here,
Inc. includes such a dynamic layer to receive up-to-date information in the vehicle by vehicle-to-
vehicle communication. The idea of this communication technique is that all vehicles driving on
the road are connected and inform each other of changes on the road (Bonetti 2016). The road
furniture layer comprises of features that may influence the driver’s behavior, such as road signs
or traffic lights. Lastly, the road geometry layer contains detailed information about the absolute
position of the road in general, as well as lane positioning and direction (Massow et al. 2016).
With a dynamic map layer, the road geometry and road furniture layer do not have to undergo
constant reconstruction, as general roadway structures do not change very often. This allows
mapping companies to map a road for autonomous-vehicle-permitting purposes without constant
re-mapping to record changes on the road.
13
2.4. Navigation Data Standard (NDS)
To standardize more than only the general structure of HD datasets, a navigation data
standard (NDS) was developed by mapping companies, automobile manufacturers and their
suppliers. The NDS format is a physical storage format of automotive-grade navigation data. The
NDS consortium developed this standard with the aim to standardize navigation data for
effortless exchange between different systems around the world (NDS association 2019). With a
global data standard, the sharing of information and data would be instant and vehicles from
various manufacturers would have the ability to be in constant connection. To make a global
vehicle-to-vehicle connection and data standard a reality, automotive companies, suppliers, and
mapping companies have joined the NDS Association. Members include Volkswagen, BMW,
Daimler, Nissan, Hyundai, Mitsubishi electric, Bertrandt, Here technologies, TomTom, Garmin,
Bosch, Panasonic, etc. (NDS Association 2019).
Described in detail in Chapter 27 of Winner et al. (2009), the uniqueness of the NDS
format is the organization of data into so-called building blocks. The navigation database is first
divided into update region databases. For example, Germany would be its own database. The
update regions are further split in to components. The component databases are the building
blocks. For example, the Germany database would include multiple unique databases such as the
routing database and Basic Map Display (BMD) database. The USA database would include its
own databases such as the routing and BMD database. The individual building blocks hold
specific information of one kind, such as names, digital terrain models, or points of interests.
Figure 3 provides an overview of the fourteen building blocks and their names. The data within
the individual building blocks are in some form or another connected to the data within other
14
building blocks, where the most fundamental characteristic of the data is their coordinates and
name (Winner et al. 2009).
Figure 3. Overview of the NDS building blocks. An enlarged version of this Figure is included in
Appendix A. Source: archive.is/WadYB (TomTom archives).
With extensive information on the data structure from the NDS format described in
Winner et al. (2009), and the basic HD dataset structure described in Massow et al. (2016), the
study described here focused on the Basic Map Display (BMD) database. Similar to the road
geometry component described in Massow et al. (2016), the BMD building block includes areas,
lines, and polygons of the absolute road position, parking lots, parking areas, and other basic
components seen in a map. In other words, the BMD serves as a fundamental building block to
which names, points of interests, and other building blocks are connected (Winner et al. 2009).
15
2.5. UAV Application in Remote Sensing
In recent years, drones have been used as an alternative remote sensing platform to
satellites or aircraft in fields such as coastal and environmental science (Klemas 2015) as well as
in agricultural sciences (Xiang 2011). UAVs have the ability to capture high resolution imagery
suitable for ground measurements in both 2D and 3D flights (O’Neil-Dunne 2015), once the
imagery is freed of distortions by software such as Pix4D. A recent study compared data
collected from different DJI drones at different elevations, and the results show that data
collected from DJI drones can be used for linear measurements, with an average margin of error
of 1.1% for all flights. The results also show that flying at low altitudes of 66 feet (20 meters)
improved measurement accuracy by 0.35%, with an average measurement error of only 0.26 feet
(0.08 meters) for a phantom 4 pro DJI drone (Putch 2017). Although applied in multiple fields,
the use of drones has yet to be applied to the production of HD maps for autonomous vehicles.
The benefits of UAVs for various applications include the ability to deploy a UAV
relatively quickly and repeatedly at low altitude. With the miniaturization of sensors and the
abundant availability of UAVs, they have become a versatile remote sensing platform (Laliberte
2016). As UAV applications have increased considerably in recent years, clearly, their
application may extend to the automotive industry as an additional or alternative way to capture
HD maps. This research explored that possibility.
16
Chapter 3 Methods
The objective of this research study was to evaluate the potential use of UAVs in HD mapping
efforts to aid the implementation of autonomous vehicles in the near future. To investigate if
UAVs are a viable alternative mapping method, several steps were taken before the study sites
could be evaluated. The following workflow depicts all major steps taken to complete this
research study (Figure 4). Dependencies within the workflow are shown by arrows, where each
arrow starts from the dependent step and ends at the succeeding step.
Figure 4. Thesis project workflow
In overview, the workflow proceeded as follows. As both the United States and Germany
have specific regulations on the use and operation of drones, the selected study sites had to first
be approved for flight. Flight approval took several months for the Bertrandt company campus in
Tappenbeck, Germany, as image capture is prohibited on the property. On the other hand,
17
approval to fly over the German International School was readily obtained since it is private
property and the school’s owners were supportive of this research effort.
After flight approval, GCPs were established within the study sites to improve
geolocation accuracy of the resulting orthomosaic. Pix4D capture was used to create flight plans
for the study sites as this app allows route planning, locks in a specified altitude, and makes for
easy transition to Pix4D mapper. Pix4Dmapper is a professional photogrammetry and drone-
mapping software capable of orthomosaic generation based on orthorectification. According to
the Pix4D support and training website, the drone images are not simply stitched together,
instead the software computes keypoints in the images to find matches. After the initial matches
are made, the software runs an automatic aerial triangulation (AAT) and bundle block adjustment
(BBA). The orthomosaic is then created based on orthorectification which removes the
perspective distortions from the imagery.
The drone imagery and GCPs were imported and processed through the initial processing
step to compute keypoints and evaluate relative and geolocational accuracy errors. After initial
processing, the third processing step was run to generate the orthomosaics. The orthomosaics
were then exported for further use in ArcGIS Pro and ArcGIS Desktop. As described on the Esri
website, both ArcGIS Pro and ArcGIS Desktop are mapping and analytics platforms to perform
varying tasks on geospatial data. ArcGIS Desktop comprises of applications such as ArcMap,
ArcCatalog, and ArcToolbox. ArcGIS Pro is the latest professional desktop GIS software that
includes all applications in one, though since it is still evolving, it does not currently contain the
entire set of functionalities built into the older ArcMap.
The exported orthomosaics underwent two independent processing methods in ArcGIS
Desktop and ArcGIS Pro. These methods include manual delineation and tool-based extraction
18
of the orthomosaics, respectfully. After ArcGIS processing, the drone-generated HD datasets
were exported to individual geopackages and compared to known HD data standards to evaluate
the application of UAVs in HD mapping practices.
The following sections describe the work completed in detail.
3.1. Determination of Prototype HD Data Standards and Structure
The definition of an HD dataset is still fluid. However, the drone-generated datasets in
this study needed to be compared to some sort of standard to validate the application of UAVs in
HD mapping practices. Prototype HD data standards and data structure were thus compiled for
this study by examining the research of related works, information on data specifications of HD
datasets from the NDS consortium, and outreach to Here, Inc.
For logistical information on the production of HD datasets by mapping-vehicles,
contacts from the US HAD Team at the HERE technologies mapping company agreed to answer
several questions regarding the use of mapping-vehicles for the development of HD datasets. The
answers from Here, Inc. provided this research study with a standard on the logistical aspects of
the current leading method of HD map production: mapping vehicles. Based on these answers,
Table 1 summarizes the prototype HD data standards used in this study as determined from the
interview, including the questions that were asked, the answers, and answers found in related
literature. The answers from Here, Inc. found in Table 1 have been summarized and are not the
exact words from the contacts at Here, Inc. For the full unaltered answer, see Appendix B.
19
Table 1. Prototype HD data standards based on communication with Here, Inc.
Question to Here Answer from Here
Information obtained by
literature review
Prototype HD data
standards
What is the
geolocational
accuracy of an HD
map?
Absolute accuracy is below
1.0 m and for some features
below 50 cm. The absolute
accuracy is always higher
than the relative accuracy
and both are equally
important.
Absolute accuracy of 1.0
m or below, in terms of x,
y, z coordinates (Massow
et al. 2009).
Relative accuracy of 15
cm in terms of
neighboring reference
locations within the map
and their relative position
to each other (TomTom
2017).
Absolute accuracy
must be below 1.0 m
in term of x, y, z.
Relative accuracy
must be below the
absolute accuracy.
How many times
must the mapping
vehicle drive a
stretch of road to
have sufficient data
to create an HD
map?
HERE True vehicles drive a
map link only once. Re-
drives occur if there is an
indicator for change, such as
construction.
A stretch is driven 5-10
times with a 64-channel
LiDAR system (Synched
2019).
A stretch is driven
1-5 times to create
an HD dataset.
Drive frequency
varies by company
and their mapping
vehicles.
What is the data
volume obtained
from driving a
stretch of 1km?
Quite a lot, HERE True
vehicles collect 60-80
Mbyte/second of raw data
from lidar plus 80Mpixel
imagery collected at 20 Hz.
OEM vehicles collect 80-
100 kbyte/km, OEM sensor
data are segmented and
highly compressed.
One hour of drive time
corresponds to 1 terabyte
(TB) of data (Seif and Hu
2016).
The data volume
obtained is high,
from 3-5 GB per
minute to multiple
TB per hour.
During data
collection, how fast
can the mapping
vehicle go?
The speed is fairly low with
a maximum speed of 80
km/h (49.7 mph). The faster
the vehicle moves, the more
sparse the lidar point cloud
gets.
No related work found. The maximum speed
for a mapping
vehicle is 80 km/h
(49.7 mph).
20
Question to Here Answer from Here
Information obtained by
literature review
Prototype HD data
standards
How much time,
including processing
time, is needed from
data collection to the
finished HD map
that can be used by
an autonomous
vehicle?
Initial road mapping with
HERE True raw sensor data
takes several days, even
weeks. Once initial mapping
is done, meaning they
consume already aggregated
segmented content, the HAD
team targets a 24hr-
turnaround-cycle.
For a 20 km radius of a
Beijing park, a fleet of
mapping vehicles spent 5
days on fixed GPS and
one day driving the
stretch 5-10 times
(Synched 2018).
Processing 1 TB of
collected data by means
of high computing power
requires two days to
create usable navigation
data (Seif and Hu 2016).
Initial processing
time, from collection
to completed HD
dataset, takes several
days to weeks.
Does an HD map of
a parking lot exist?
If so, what is the
general data
structure of the
map?
The HAD team is focused
on Limited Access Road
network coverage, where
physical dividers exists
between roads. The area of
“Parking” is not directly
covered by HAD for now.
No related work found. Currently, no
publicly available
HD map of a
parking lot exists.
How much does the
service of Here cost?
Here, Inc. cannot be hired as
an individual contractor and
thus cannot answer this
question.
A mapping company
named DeepMap charges
$5,000 per kilometer for
its services in the US
(Synced 2018).
Mapping companies
for hire can charge
sums of $5,000 per
kilometer for their
services.
With the goal to obtain a uniform data structure, the NDS consortium requires all HD
datasets to follow the same data structure. As described in Chapter 2, the NDS data structure
constitutes multiple databases and multiple layers of geo-informational data. The focus of this
research study was on the Basic Map Display (BMD) and the required NDS data structure of the
BMD database. Based on the data structure used by Bertrandt, Inc. for conversion to NDS, Table
2 outlines the HD data structure used in this study for the drone-generated HD datasets. The
digitized feature classes had to be drawn counter-clockwise as the NDS format can only display
feature classes where the line-vertices are as such. Every feature class had to be exported to its
own geopackage. A list with all feature class names used in the NDS format was provided by
Bertrandt, from which the feature classes that occurred within the individual study sites were
21
selected. From there, the exported geopackages were converted to the NDS format by the
engineering services company Bertrandt. Validation of the output of the workflow in this study is
achieved if the data can be converted and displayed in the NDS format, such that the data can be
displayed for use in autonomous vehicles.
Table 2. Prototype HD data structure for conversion to NDS.
Layer Name
Included
features
Mandatory
Attributes
Geometry
AREA_GREEN_URBAN vegetation
fid, markCount,
areaFeatureClass
Single part
line
AREA_TRAFFIC_PARKING
the entire
parking lot
fid, markCount,
areaFeatureClass
Single part
line
AREA_TRAFFIC_PARKING
_ LOT
parking spaces
within the
parking lot
fid, markCount,
areaFeatureClass
Single part
line
AREA_TRAFFIC_ROAD
road within the
parking lot
fid, markCount,
areaFeatureClass
Single part
line
AREA_ROCK
landscaping
made of rock
fid, markCount,
areaFeatureClass
Single part
line
BMD_LINES
borders,
boundaries,
fences, walls
fid, markCount,
lineFeatureClass
Single part
line
3.2. Collection and Preparation of Drone Imagery
As a remote sensing research project, data requirements for this study include drone-
generated aerial imagery and a standard for comparison. The gathered data consisted of aerial
imagery collected from a DJI Phantom 4 drone with one 12.4-megapixel camera that captured
true color camera imaging at visible wavelengths. For both study sites, an orthomosaic was
created and exported in a TIFF file format.
22
3.2.1. Bertrandt Parking Lot Data
A total of 413 images were collected at nadir and captured at an altitude of 15 meters (49
feet). This altitude was originally selected as it allowed for vehicle and tree canopy clearance at
the study site while being low enough for detailed and clear imagery capture. The spatial
resolution of the Bertrandt Parking lot was 0.57 cm per pixel. The images were collected during
a period of continuous cloud cover to avoid glare and shadows in the images. As the battery life
of a Phantom 4 drone allows a flight duration of approximately 30 minutes, several flights were
flown to cover the entire study site. All images, including repeated flights, were imported and
processed together as one project for each study site. Processing all imagery together allowed
imagery from separate flights to be tied together and geolocated to one another.
Once all images were imported, GCPs were added to the project and marked manually
within each image where they were visible. For the Bertrandt parking lot, previously measured
benchmarks with an accuracy of 4 cm were available within the selected study site extent, made
available by the cadastral office in Gifhorn, Germany. These benchmarks were used as GCPs and
were marked on the ground with a large, high-contrast, target. A total of three GCPs were used
for this study site. They are represented by blue crosses in the Pix4D mapper output shown in
Figure 5. Here, all images used for the study site are represented as red dots, whereas the green
lines illustrate the flight path of the drone while capturing the images.
23
Figure 5. Pix4D mapper output showing the flight path (green lines), locations of captured
images (red dots), and ground control points (blue crosses) for the first study site. The width of
the mapped area shown is 120 meters.
After initial processing in Pix4D mapper, manual tie points (MTPs) were added as they
can improve the reconstruction accuracy. Similar to GCPs, the MTPs can be marked in each
image in which the selected tie point is visible. As the number of GCPs was relatively low for
the size of this study site, a total of nine MTPs were added. After all desired points were marked,
the project was reoptimized and reprocessed by rerunning the initial processing and orthomosaic
generation steps in Pix4D mapper.
24
3.2.2. German School Parking Lot Data
A total of 454 images were captured with the Phantom 4 drone for the German School
parking lot. The images were collected at nadir during light continuous cloud cover and at an
altitude of 15 meters (49 feet). The spatial resolution of the German School parking lot was 0.62
cm per pixel. A total of 4 GCPs were evenly distributed throughout the study site, as shown in
Figure 6.
Figure 6. Pix4D mapper output showing the flight path (green lines), locations of captured
images (red dots), and ground control points (blue crosses) for the second study site. The width
of the mapped area shown is 100 meters.
As no previously measured benchmark data was available, the GCPs in study site 2 were
established using a Trimble GeoExplorer XH 6000 series GPS unit, made available by the
Spatial Science Institute at USC. As described by Trimble, the GeoXH handheld GPS unit uses
25
two multipath rejection technologies to provide decimeter, 10 cm, accuracy either real-time or
after postprocessing (Trimble 2011). As shown in Table 3, submeter real-time horizontal
accuracy and approximately one-meter vertical accuracy was achieved for all GCPs.
Table 3. Horizontal and vertical accuracy of ground control points.
Horizontal
Accuracy
Vertical
Accuracy
GCP 1 0.69 m 0.87 m
GCP 2 0.77 m 0.93 m
GCP 3 0.74 m 0.99 m
GCP 4 0.64 m 0.86 m
The images for the German school parking lot were collected in May 2019, whereas the
images for the Bertrandt parking lot were collected in January 2019. As a result, the orthomosaic
generated for the German school parking lot encountered tree canopy obstruction above several
parking spaces, due to the seasonal change to spring. To remove the overhanging vegetation, the
generated orthomosaic was edited within the orthomosaic editor in Pix4D mapper. Within the
editor, areas of the orthomosaic can be selected and subsequently all available images for that
particular location are displayed. Multiple images shot from different angles or days can be
selected to replace the image used with the obstructed view. Figure 7 shows the results of this
removal of vegetation in the study site through the orthomosaic editor in Pix4D mapper. Once all
edits were complete, the orthomosaic was saved and exported as the final orthomosaic for study
site 2.
26
Figure 7. Before (left) and after (right) snapshots of vegetation removal.
3.3. Manual Delineation of Required Features on Orthomosaics
According to Bertrandt’s efforts in converting data structures to NDS, current
commercial efforts of HD dataset generation still depend heavily on the manual delineation of
desired features. Thus, one of the two processing methods of the orthomosaics was done in
ArcGIS Desktop and was comprised of the heads-up digitization of the study sites. The
orthomosaics of both study sites were digitized following the required HD data structure.
In ArcCatalog, a geodatabase (.gdb) was established for each study site. Feature classes
were created within the geodatabase for each feature class represented in the orthomosaic. For
example, study site 1 had small landscaping areas of rock, thus an AREA_ROCK feature class
was created in the geodatabase for study site 1. All areas, such as parking spaces or roads, were
27
initially set up as polygon feature classes. Borders, such as the end of the parking lot, were set up
as a polyline feature class and labeled BMD_LINES, as shown in Table 3 above.
All feature classes within the geodatabase received the mandatory attributes of
“markCount” and “line-” or “areaFeatureClass”. As Esri ArcGIS products assign every feature
within a feature class a unique identifier known as OBJECTID, no additional “fid” attribute had
to be entered as the OBJECTID was used as “fid.” The purpose of the markCount attribute was
to show how many vertices a digitized feature has. All area features were required to contain a
minimum of three vertices. The markCount attribute type was numeric and was calculated by the
field calculator with the following python formula:
!shape!.pointcount
The line or areaFeatureClass was a text field and contained the name of the feature class,
such as AREA_ROCK. Once the feature class data structure was established in ArcCatalog, the
orthomosaic was digitized manually in ArcMap.
Since the NDS database can only read area feature boundaries where the line direction is
counter-clockwise, all area feature classes were digitized counter-clockwise, tracing polygons
over the orthomosaic from right to left. All tracing efforts were done within an editor session and
saved periodically. For use in autonomous vehicles, individual parking space polygons cannot
share a border with one another as the outer boundaries of each parking space must be unique.
Curbs and other physical obstructions count as non-drivable surfaces and were therefore
excluded in the tracing efforts, as shown in Figure 8. By excluding curbs, parking space lines,
and other barriers, all polygons were given a unique boundary.
28
Figure 8. Illustration of boundary exclusion in manual delineation.
To convert the geopackages to the NDS format, all feature classes had to have a single
part line geometry. Thus, once all tracing efforts were completed, the polygon feature classes
were changed to polyline feature classes by use of the tool “Polygon to Line.” The tool converted
the polygon boundaries to polylines. The new polylines were then used as inputs for the tool
“Multipart to Singlepart,” a data management tool that separated the multipart polylines to single
part lines. Figure 9 illustrates the change in geometry applied by use of the Multipart to
Singlepart tool. With a single line geometry, the feature classes were in the correct format and
29
were exported as individual geopackages. Lastly, the geopackages were sent to Bertrandt, Inc., to
test if the data was NDS transferable.
Figure 9. Illustration of the Multipart to Singlepart tool. Source: Esri 2019
3.4. Tool-based Extraction
The second processing method of the orthomosaic was done in ArcGIS Pro. The goal of
this method was to create a tool-based model workflow that would automatically extract the
desired features from the orthomosaic with little manual input by the user. To achieve this goal,
object-based image classification and segmentation methods were applied based on Esri’s object-
oriented feature extraction workflow, Esri’s post-classification processing workflow, and the
Vector machine classification approach described in Tzotsos and Argialas (2008). The Vector
machine classification approach used was a supervised classification method that has gained
much attention due to its high classification accuracy and small needed training sets, according
to Tzotsos and Argialas.
As shown in Figures 1 and 2, the raster inputs (orthomosaics) from the two selected study
sites had very different spectral and spatial characteristics. The Bertrandt study site was made of
cobble stone and parking spots were mostly occupied by vehicles of all colors and sizes. The
German school study site had no vehicles within the orthomosaic and the dominant surface type
was asphalt.
30
To accommodate the differences between the selected study sites, three ArcGIS models
were developed. The main ArcGIS model included the processes applied to both study sites to
extract the desired polygon feature classes from the raster input. Two extension models, labeled
as the Bertrandt extension model and German school extension model, were created as the
following processing tools varied due to the spectral and spatial differences between the study
sites. The main ArcGIS model, combined with the extension model for the specified study site,
rendered the final output of line feature class roads, parking spots, parking areas, and green
areas. Table 4 shows a list of all the tools utilized in the three models and gives a brief
description of each. Each of these models is explained in detail in the sections following.
Table 4. Name and description of all geoprocessing tools applied.
Tool Name Tool Description Tool Type
Extract by mask Extracts raster cells which lay within the area defined
by a mask.
Raster
Convolution smooth
(5x5)
Smooths the raster with a 5-cell x 5-cell moving
window by calculating the pixel value based on the
weighs of its neighbors.
Raster
Segment mean shift Groups together adjacent pixels that have similar
spectral and spatial characteristics.
Raster
Train support vector
machine classifier
A supervised classification method well suited for
segmented images. The tool is a classification
training tool and generates an Esri classifier
definition file (.ecd).
Raster
Classify Classifies a raster dataset based on a Esri classifier
definition file and the raster inputs.
Raster
Reclassify Reclassifies the values in a raster. Can be used to
separate or join ranges of values.
Raster
Boundary clean Smooths the boundaries between zones by changes
regions of less than 3 cells.
Raster
Region group Groups cells in a raster into regions where a unique
number is assigned to each region. Individual regions
are created for small pixel groups of the same value.
Raster
Select by attribute Selects features by their specified attributes. A
Clause is used to select certain attributes.
Raster/
Vector
31
Tool Name Tool Description Tool Type
Set null Sets identified cell values to NoData. Setting a false
null value is often used to change all values that meet
specified conditions to NoData, or to create a mask.
Raster
Nibble Replaces the cells in a raster, according to a mask,
with the values of the nearest neighbors.
Raster
Raster to polygon Converts a raster dataset to a polygon feature. Raster
Eliminate Removes small sliver polygons by merging them
with the largest neighboring polygon.
Vector
Copy features Copies specified features to a new feature class. Vector
Buffer Creates a buffer around a point, line, or polygon. Vector
Erase Creates a feature class that only has the portions of
the input feature class that lie outside of the erase
feature. In other words, it allows the area of one
feature to be cut out of another feature.
Vector
Delete rows Deletes all selected rows from the input. Vector
Polygon to line Convert a polygon feature to a line feature. Vector
Multipart to singlepart Separates multipart features into single part features. Vector
Source: pro.arcgis.com
For some tools, optional parameters were entered to refine the output or to search for
certain attributes. The “select by attribute” tool was used four separate times and was thus given
a number for each tool use. Although mostly the same, a few parameters varied between study
sites. Table 5 describes in more detail the tools’ parameters selected for the two study sites.
Table 5. Tool input parameters.
Tool Input parameter study site 1 Input parameter study site 2
Segment mean shift spectral detail: 20
spatial detail: 5
segment size: 30
spectral detail: 20
spatial detail: 5
segment size: 30
Reclassify 0 1
1 2
2 3
3 4
4 1
5 1
6 3
0 1
1 2
2 3
3 4
32
Tool Input parameter study site 1 Input parameter study site 2
Select by attribute 1 count < 10 000 count < 10 000
AND link ≠ 2
OR link = 2
AND count < 500
Select by attribute 2 shape area < 3 shape area < 3
AND gridcode ≠ 2
Select by attribute 3 (run 4x)
gridcode = 1
gridcode = 2
gridcode = 3
gridcode = 4
(run 3x)
gridcode = 1
gridcode = 2
gridcode = 3
Select by attribute 4 Not applicable shape area < 5
OR shape length > 75
Buffer 0.3 m Not applicable
3.4.1. Main Model
The main ArcGIS model, shown in Figure 10, depicts all geoprocessing tools used to
extract polygon feature classes, along with their inputs and outputs. Dependencies within the
model are shown by arrows, where each arrow starts from the dependent step and ends at the
succeeding step. The model tools were color-coded where tools that require manual input are
shown in blue.
33
Figure 10. Main ArcGIS model: Orthomosaic to polygon feature classes
The initial model inputs were the orthomosaics exported from Pix4D mapper and a mask
extent to reduce the size of the raster to the exact region of interest. The mask extent was created
in an editor session. Once the extent was reduced and smoothed, the orthomosaic was segmented
and then classified using training sample polygons created in the train sample manager, found in
34
the image classification pane. The training samples allowed for a supervised classification where
samples are picked from the segmented image to represent a class value. For the Bertrandt
parking lot, training samples were taken for the road, parking area, vehicles, and green areas. For
the German school parking lot, training samples were taken of the white-painted parking lines,
road, sidewalks, and green areas. Although the sidewalk in the German school parking lot was
not used in the final classification, training samples were created to ensure that sidewalks were
not categorized with roads or other class values of significance.
After the raster was classified, new values were selected by the reclassify tool to group
class values together as needed. The reclassify tool was important as the model may be run with
different inputs and different training sample classes. After the reclassify tool, however, all class
values are known, and similar classes are grouped together as one. In a high-resolution raster,
small pixel regions can be classified incorrectly. To further process and regroup small pixel
regions to the majority pixel class in its neighborhood, the remaining tools in the main model
were based on the post-classification processing workflow from Esri. Boundaries between
regions were cleaned and small pixel groups were removed from the classified raster by use of
the tools Region Group and Nibble. Next, the raster was converted to a polygon feature class and
small insignificant polygons were removed with the Eliminate tool. Lastly, the polygon feature
class was exported to multiple polygon feature classes based on the class value attribute (Copy
Features tool). With individual polygon classes produced for each desired feature, the work of
the main ArcGIS model was complete and the separate extension models were applied to each
study site for final processing.
35
3.4.2. Extension Models
The extension models include additional steps taken to extract line feature classes from
each polygon feature class created by application of the main ArcGIS model. The class values,
gridcode X, were subjected to various geoprocessing tools to render the final outputs.
Due to the vehicles present in the Bertrandt parking lot, a separate feature class was
created for all vehicles within the parking lot and given the class value four. Parking spots were
not marked with white lines, instead, individual cobble stones formed a striped pattern with
slightly different spectral signatures to the surrounding cobble stones. To extract parking spaces,
the vehicles within the parking spots were used to represent areas used for parking, instead of
using the parking stripes. The vehicle feature class received a small buffer of 30 cm as vehicles
are smaller than the allotted parking area.
Shown in Figure 11, the Bertrandt extension model received four inputs from the main
ArcGIS model. After the vehicle feature class received a buffer, all polygon inputs were
converted to polylines, followed by the conversion to single part lines. The final outputs of the
extension model were again exported as geopackages and sent to Bertrandt, Inc., to test if the
data was NDS transferable.
36
Figure 11. Bertrandt extension model: Polygon to line feature class
The German school extension model varied significantly from the Bertrandt extension
model. Only three input features were rendered from the main ArcGIS model and additional
geoprocessing tools were used on the three inputs, as shown in Figure 12. No vehicles were
present in the German school parking lot during data collection, thus no feature class was created
for the vehicles. As the parking lot was empty and parking lines were clearly marked, the parking
line grid painted on top of the asphalt road was the focus point to create parking spaces.
37
Figure 12. German school extension model: Polygon to line feature class
The German school parking lot required some manual input to create the rectangular
parking spots. The parking line polygons, gridcode two, were used to create the parking spaces
by use of the “Auto-Complete polygon” tool, available in an edit session when creating
additional polygon features in a polygon layer. The autocomplete polygon created a new polygon
by using the existing polygon’s geometry and the edit sketch to define the edges of the new
polygon, as described on the Esri tool support website. In other words, the autocomplete polygon
tool was used to quickly snap individual parking spaces on to the existing parking line grid.
Figure 13 demonstrates the use of the autocomplete tool to create individual parking spaces.
38
Figure 13. “Auto-Complete Polygon” tool demonstration. In step a) the user draws a line across
the desired area where new polygons should be created. In step b) the polygons are created by
double-clicking the mouse. Step c) shows one of the newly created polygon features outlined in
blue.
Once all parking spaces were complete, the parking space polygons were used in
conjunction with the road feature class to erase the road within the extent of the parking spaces.
Due to an empty asphalt lot, the road polygon included all parking spaces, except for the white
painted parking lines, as road. If exported as is, the parking spaces would have been categorized
as drivable road. Thus, the erase tool was used to eliminate road within the extent of the parking
area. As described in Section 3.3, all parking spaces required a unique boundary, so no two
39
parking spaces could share a line. The “delete rows” tool was therefore used on the parking area
polygons to eliminate the white parking lines between the individual parking spaces. Lastly, all
polygon feature classes were converted to single part polylines and exported as individual
geopackages.
3.5. Validation Methods
The results of this research study include the drone-generated orthomosaics, the two HD
datasets generated by manual delineation, and the two HD datasets generated by the application
of various geoprocessing tools. The orthomosaics were evaluated based on accuracy, overall
quality, and time. The quality of the orthomosaics and the collection process of drone imagery
were the key components to answer the question if drones are a viable alternative mapping
method. The accuracy was distinguished into absolute and relative accuracy. The absolute
accuracy was defined by the x (east-west), y (north-south), and z (elevation) difference between
the location of features on the orthomosaic and their true positions on the planet. The absolute
accuracy of the drone-generated orthomosaic depended on the GCPs’ accuracy, distribution, and
number. The relative accuracy is the positional accuracy of individual features on the map
compared to the location of other features on the same map.
The four HD datasets generated by manual delineation and tool-based extraction methods
were evaluated based on time of the processing method and data structure. The drone-generated
HD datasets and the prototype HD data standards were compared to analyze if the drone-
generated HD datasets matched the prototype HD standards. Lastly, HD datasets were sent to
contacts at Bertrandt, Inc. to see if the data structure and quality was good enough to convert the
HD datasets in to the NDS file format and could be displayed in the NDS database inspector.
40
Chapter 4 Results
This chapter presents the key findings of this research study. The UAV-based remote sensing
technique applied in this study showed that high quality orthomosaics can be generated from low
cost recreational drones with utilization of ground control points (GCPs). The orthomosaic
processing method of manual delineation tested in this study shows successful conversion and
display of the digitized objects in the NDS format. The tool-based extraction method, which
included object-based image classification and post-processing methods, was successful in
segmenting and classifying the orthomosaic and extracting the desired features. The tool-based
extraction method was successfully converted to the NDS format but was not able to be
displayed in the NDS database. The manual delineation processing method was therefore
deemed the best practice for HD dataset development in this research study.
4.1. Assessment of Drone-generated Images and Orthomosaics
The imagery of the Bertrandt study site and German school study site were collected by a
Phantom 4 drone with a 12.4-megapixel camera. The time invested in data collection consisted
of flight planning, data collection, and processing in Pix4D mapper. Table 6 shows in detail the
time taken for each step to generate the final orthomosaics. Once permission for drone flight was
approved, the total time for data collection was approximately 11 hours for the Bertrandt study
site and approximately 7 hours for the German school study site. The Bertrandt study site
required multiple flights as the battery life of a Phnatom 4 drone is 30 minutes. The Bertrandt
study site included two separate flights where each flight heavily overlapped imagery from the
previous flight. The German school study site was flown twice where each flight was 11 minutes
and 10 seconds.
41
Table 6. Time required for orthomosaic generation by Phantom 4 drone.
Bertrandt Study Site 1 German School Study Site 2
Allotted time for data
collection
6 days 14 days
Time for flight planning 8 hrs 5 hrs
Total flight time 58 min 46s (2 flights) 22 min 20s (2 flights)
Time for initial image
processing
1 hr 06 min 11s 45 min 02s
Time for Orthomosaic
generation
32 min 31s 29 min 56s
All drone-collected images were processed in Pix4D mapper which produces extensive
quality reports after initial processing. All quality results presented in this chapter were drawn
from these reports. The complete quality reports summarized in this chapter can be found in
Appendix C for study site 1 results and Appendix D for study site 2 results.
A summary of the processing results is shown in Table 7. Here, the ground sampling
distance (GSD) is measured as the distance between two adjacent pixel-centers. The smaller the
GSD, the greater the spatial resolution of the image. For both study sites, a sub-centimeter
ground sampling distance was achieved.
Table 7. Summary of Pix4D processing results.
Bertrandt Study Site 1 German School Study Site 2
Ground sampling distance
(GSD)
0.56 cm (0.22 inches) 0.62 cm (0.24 inches)
Orthomosaic Resolution 0.57 cm/pixel 0.626 cm/pixel
Median matches per
calibrated image
12795.6 3520.87
Area Covered 0.0184 km
2
(0.0071 sq. miles) 0.0091 km
2
(0.0035 sq. miles)
Ground control points (GCPs) 3 4
Manual tie points (MTPs) 9 0
Image Overlap High (5+) High (5+)
42
The number of image overlap ranges from 1 (low) to 5 (high), where low image overlap
may give poor results. The number of overlapping images is computed for each pixel in the
orthomosaic. Good results were generated for both study sites as most of the orthomosaics had
high overlap with sufficient number of keypoint matches. Figure 14 shows the image overlap
throughout the orthomosaics in both study sites. Low overlap can be seen along the edges of the
orthomosaic, in particular for study site 1 where the flight extent was very restricted.
Figure 14. Image overlap results. Study site 1 (left), study site 2 (right). Red through yellow
indicate poor overlap, green indicates high overlap from which good results were generated.
Source: Pix4D quality reports.
To evaluate the accuracy and overall quality of the drone-generated orthomosaics, the
differences between the initial and computed image positions were calculated, as shown in
43
Table 8. The difference between the initial and computed positions is the error. Large error
values indicate that much stretching and skewing had to occur for the data to match. With high
overlap and sub-centimeter ground sampling distances of 0.56 cm for study site 1 and 0.62 cm
for study site 2, small error ranges were achieved between the initial and computed image
positions for the study sites.
Table 8. Error assessment calculated as the difference between initial and computed positions.
Bertrandt Study Site 1 German School Study Site 2
Mean absolute geolocation
error (m)
X Y Z
-0.22 0.51 -0.07
X Y Z
-0.05 0.01 -0.10
Mean absolute camera
position and orientation
uncertainty (m)
X Y Z
0.046 0.042 0.113
X Y Z
0.547 0.322 0.640
Mean relative camera
position and orientation
uncertainty (m)
X Y Z
0.020 0.025 0.071
X Y Z
0.010 0.012 0.017
Relative geolocation error
(cm)
X Y Z
±1.12 ±0.56 ±0.56
X Y Z
±0.62 ±0.62 ±0.62
Mean ground control point
error (m)
X Y Z
.000002 .000003 .000001
X Y Z
0.0037 0.0013 -0.004
Mean root mean square
(RMS) error
0.002 m 0.028 m
Mean projection error
(pixels)
0.165 pixels 0.210 pixels
The mean absolute geolocation error for the Bertrandt study site was found to be -22 cm
in the X direction, 51 cm in the Y direction, and -7 cm in the Z direction. The mean ground
control point error was virtually zero. With such low error values and a GCP accuracy of 4 cm,
the Bertrandt study site has achieved high quality results with centimeter (below one meter) level
relative and absolute accuracy. The mean RMS error of 0.2 cm for study site 1 is very small,
44
indicating a good and consistent transformation accuracy. The orthomosaic generated for study
site 1 is shown in Figure 15.
Figure 15. Bertrandt parking lot orthomosaic; study site 1.
45
The mean absolute geolocation error for the German school study site was found to be -5
cm in X, 1 cm in Y, and -10 cm in Z. The mean ground control point error was 0.37 cm in X,
0.13 cm in Y, and -0.4 cm in Z. As described in Section 3.1.2, the GCPs in the German school
study site were collected with a handheld GPS unit and achieved an average accuracy of 71 cm
in XY direction and 91.5 cm in the Z direction. With low error values and sub-meter accurate
GCPs, the German school study site 2 has achieved good quality results with centimeter level
relative accuracy and absolute accuracy at or just below one meter. The mean RMS error of 2.8
cm for study site 2 is large, indicating issues and/or inconsistent transformation accuracy. Figure
16 shows the orthomosaic generated for study site 2. Inconsistencies and transformation issues
can be seen along the edges of the orthomosaic.
46
Figure 16. German school parking lot orthomosaic; study site 2
47
4.2. Manual Delineation Results
The Bertrandt study site and German School study site were successfully digitized by
manual efforts. The required data structure for conversion to the NDS data format was followed.
As the total area covered for both study sites was relatively small, the required time for manual
digitizing was a few hours. As shown in Table 9, the total time for digitizing includes all feature
classes such as roads, parking spots, parking areas, and green surfaces. The time to set up the
data structure includes the editing of the attributes, running the tools to convert polygons to lines,
and multipart to single part lines, running the tool to create the SQLite database, and exporting
the feature classes as geopackages. Due to the large (1.07 GB) size of the Bertrandt study site
orthomosaic, some of the digitizing time of the Bertrandt study site went to patiently waiting as
the orthomosaic loaded.
Table 9. Time required for manual delineation.
Bertrandt Study Site 1 German School Study Site 2
Total time for digitizing 4 hrs 43 min 1 hr 46 min
Time to set up data structure 25 min 20 min
Both study sites were successfully exported to geopackages as single line feature classes
and sent to Andreas Pehlke at Bertrandt, Inc. to test NDS conversion. Figure 17 and 18 show the
final data representations of study site 1 and 2 that were exported as line feature classes from
ArcGIS Desktop. The BMD_LINES feature class marks the entry and exit border of the parking
lot.
48
Figure 17. Manual delineation; final representation of study site 1.
49
Figure 18. Manual delineation; final representation of study site 2.
50
4.3. Tool-based Extraction Results
The vector machine classification approach, in conjunction with Esri’s object-oriented
feature extraction workflow and post-classification processing workflow, proved successful in
extracting the specified features from the orthomosaic with little noise in the polygon feature
classes created. For both study sites, the training samples entered in the Train Sample Manager
wizard and used for supervised classification correctly grouped the specified objects into classes.
From the orthomosaic to NDS-ready line feature classes, the total time for the Bertrandt study
site was 9 hours and 33 minutes and for the German School study site 3 hours and 42 minutes.
Compared to the manual delineation method, the manual input in the tool-based extraction was
significantly less, with 10 minutes of manual input for the Bertrandt study site and 34 minutes
for the German School study site. Table 10 shows in more detail the runtime for the tools and
model components. The tools in Table 10 are grouped into components based on their processing
purpose.
Table 10. Tool and model component runtime for tool-based extraction method.
Tool and Model component
Runtime
Bertrandt Study Site 1 German School Study Site 2
Segment mean shift runtime 4 hrs 19 min 47 min
Vector support classification
tools runtime
(train support vector machine
classifier, classify raster,
reclassify)
3 hrs 11 min 1 hr 41 min
Post-processing tools runtime
(boundary clean, region group,
nibble, eliminate)
1 hr 49 min 35 min
Extension model runtime
(tool run only time)
4 min 6 min
Autocomplete polygons
(manual input)
N/A 24 min
Train sample manager
(manual input)
10 min 10 min
Total time 9 hrs 33 min 3 hrs 42 min
51
The extraction process of features from the Bertrandt study site was successful and
included vehicle, vegetation, parking area, and road extraction. The cobble stone pattern of the
road was initially an issue as the road was divided into hundreds of small rectangle polygons. By
use of the post-processing tools that removed insignificant pixel groups and polygons, the cobble
stone pattern was, however, removed and the road was rendered correctly. The results of the
main model run, orthomosaic to polygon feature class extraction, for the Bertrandt study site are
shown in Figure 19. The results of the Bertrandt parking lot extension model run are shown in
Figure 20. The result shown in Figure 20 is the final representation of the Bertrandt study site by
tool-based extraction methods. Compared to the final representation by manual delineation
(Figure 17), the results of tool-based extraction are more chaotic. The main reason for large
differences between the processing methods for the Bertrandt study site is the fact that the
vehicles were included in the tool-based extraction method but were ignored during manual
delineation.
52
Figure 19. Main model result for the Bertrandt study site 1.
53
Figure 20. Bertrandt extension model result; final representation of study site 1.
54
The extraction processes of features from the German School study site were also
successful and included parking spots, road, and vegetation. Figure 21 depicts the results of the
main model run, orthomosaic to polygon feature classes, for the German school study site. The
results of the German School extension model are shown in Figure 22. The extension model
included the manual input to the “Autocomplete Polygons” tool to complete parking polygons
out of the parking lines. With a manual input of 24 minutes, significantly less time was spent
digitizing than by methods of manual delineation. Due to the lack of vehicles in the German
school parking lot and the visibility of parking lines, the final representation shown in Figure 22
is significantly less chaotic than tool-based extraction results seen in study site 1.
55
Figure 21. Main model result for the German School study site 2.
56
Figure 22. German School extension model result; final representation of study site 2.
57
4.4. NDS Conversion
The manual delineation results of the Bertrandt study site and German School study site
were successfully converted to the NDS data format and the resulting datasets were successfully
displayed in the NDS database Inspector. The tool-based extraction results of the study sites
were successfully converted to the NDS data format but did not display in the NDS database
Inspector.
In an e-mail thread with the author from June 23
rd
to July 16
th
, 2019, Bertrandt employee
Andreas Pehlke explained that no objects were rendered for the tool-based extraction method in
the NDS-database Inspector which is most likely due to the lack of order and direction in the
vertices of the objects. Objects in the NDS format are only displayed if the vertices of the lines
are in the correct order. The geometry type and attribute information were according to NDS
format specifications and therefore ran through the conversion successfully. For the manual
delineation method, the results were converted successfully but a few vertices had to be changed
by relocating the vertex for correct display in the NDS-database Inspector.
The final NDS-formatted datasets from the output produced by the manual delineation
method are shown in Figures 23 and 24 for study site 1 and study site 2, respectfully. The NDS
formatted datasets are displayed in the NDS-database Inspector in Figures 23 and 24. In Figure
23, some discontinuity can be seen in the road feature class where vertices did not have the
correct order or direction. The breaks in continuity can also be due to uncertainty in the specified
coordinates of the vertices. With some manual adjustment in the NDS database, the coordinates
of these vertices can be shifted.
58
Figure 23. Bertrandt parking lot in NDS format. Parking spots and road are white, vegetation is
green, exit and entry borders are purple, and rock area is black. Source: Andreas Pehlke,
Bertrandt, Inc.
59
Figure 24. German School parking lot in NDS format. Parking spots and road are white,
vegetation is green, exit and entry borders are purple, and the parking area boundary is black.
Source: Andreas Pehlke, Bertrandt, Inc.
60
Chapter 5 Discussion and Conclusion
Although aerial drones have been used as a remote sensing method in fields such as agriculture
and environmental studies, the drone-based remote sensing method and processing methods
evaluated in this research study offer an initial investigation of an alternative approach to HD
dataset development for use in autonomous vehicle navigation.
5.1. Current HD Data Standards and the Drone-generated HD Datasets
Described in Section 3.3, the HD data standards uncovered from communications with
Here Inc., related literature research, and communication with Bertrandt Inc., specify that an HD
dataset has an absolute accuracy at or below one meter and a relative accuracy below the
absolute accuracy. The drone-generated orthomosaics showed that the accuracy for aerial
imagery relies heavily on the accuracy of the ground control points (GCPs) and the ground
sampling distance (GSD). According to the Pix4D website, relative accuracy is expected within
one to three times the GSD, absolute accuracy is one to two times the GSD horizontally, and one
to three times the GSD vertically, assuming the projects were reconstructed correctly. The
Bertrandt study site had centimeter level absolute and relative accuracy errors with a GSD of
0.56 cm and a GCP accuracy of 4 cm. Assuming the Bertrandt study site was reconstructed
correctly, the absolute and relative accuracy requirements were achieved at values far below one
meter. The German School study site had absolute and relative accuracy errors in the 0-10 cm
range with a GSD of 0.62 cm. The GCPs of the German School study site had an overall
accuracy just below one meter. Assuming the German School study site was reconstructed
correctly, the absolute and relative accuracy requirements were achieved with absolute accuracy
values at approximately one meter.
61
The logistics of data collection by mapping-vehicles were compared to the data collection
by the Phantom 4 drone, as shown in Table 11. Overall, this study showed that data collection by
aerial drones can be a faster and more cost-efficient alternative method for data collection
compared to mapping-vehicles.
Table 11. Data collection by mapping-vehicles vs. by aerial drone.
Data collection by mapping-vehicle Data collection by aerial drone
A stretch is driven 1-5 times for data
collection.
The study sites were flown 2-3 times
for data collection.
Data volume is high from 3-5 GB per
minute to multiple TB per hour.
Data volume was low at 1.5 GB to 2.5
GB per study site.
Processing time, from data collection to
HD dataset, is several days to weeks.
Total
processing
time
Study
site 1
Study
site 2
With manual
delineation
15.5 hrs 8.2 hrs
With tool-
based
extraction
20 hrs 10 hrs
Mapping companies for hire can charge
sums of $5,000 per kilometer for their
services.
The only additional cost in this study
was purchase of a Pix4D license.
The data volume obtained, time for data collection, and processing time by mapping-
vehicles is significantly higher than the aerial imagery obtained and processed in this study.
Important to remember, however, is that mapping-vehicles collect data with more than just one
camera. A mapping-vehicle used in a study by Dolgov and Thrun 2009, depicts the use of a
LiDAR sensor in addition to four different laser range finders (Dolgov and Thrun 2009).
Combining the data output from all sensors and cameras during mapping results in high data
volumes. The high data volumes, in turn, increase the processing time.
62
In this research study, a single-camera aerial drone was used to create high quality data.
If drones can be used, it begs the question if high volume LiDAR sensing, in addition to the other
sensors, is truly necessary to create a highly accurate map for use in autonomous vehicles. The
company Waymo, which was formerly Google’s self-driving car project, currently develops fully
autonomous vehicles for shuttle and commercial services. These autonomous vehicles are
equipped with three sensors; a 360-degree LiDAR sensor, a long-range sensor in the front, and a
short-range sensor to monitor the car’s perimeter (Randall 2019). If autonomous vehicles will be
equipped with centimeter level sensors to navigate merely on their own, a high-quality map
developed by means of aerial imagery may suffice.
5.2. Limitations and Challenges in Drone-based Data Collection
The allotted time for drone-based data collection at the Bertrandt study site in Germany
was less than one week in January 2019. Within the week, weather challenges of rain and stormy
winds limited data collection to three days. With flexible work hours at Bertrandt, Inc., the time
of data collection was limited as well due to heavy people-traffic in the study site in the morning,
at noon, and after 3:30 pm. With the limited possible flight time and weather uncertainty in
January, the Bertrandt study site was flown in two parts during work hours. Unfortunately,
parking at the Bertrandt company was limited and vehicles were present in the study site during
data collection.
The orthomosaic generated for the Bertrandt study site experienced some challenges as
well, as certain areas of the orthomosaic were distorted. Figure 25 illustrates the distortion of the
northwest corner in the Bertrandt study site. The quality report generated for the Bertrandt
parking lot shows that two blocks were generated for the calibrated images. This northwest
region of the parking lot was separated into its own dataset block and thus extreme discontinuity
63
can be seen between the two blocks. The small northwest corner section was not tied well
enough to the rest of the model. As a result, the northwest corner of the Bertrandt study site was
not included in the processing.
Figure 25. Distortion in the northwest corner of the Bertrandt parking lot orthomosaic.
For the German School study site, no previously-surveyed benchmark points were
available for use as GCPs, thus GCPs were measured with a GPS unit. Although a survey-grade
64
GPS unit would have been ideal for precise GCP marking, the accuracy of the GCPs in the
German school study site were limited to the GPS unit available.
5.3. Proprietary Information
In addition to limitations and challenges faced during data collection, the lack of publicly
available information on the subject surrounding autonomous driving posed another challenge.
Literature and related work on subjects regarding the development of autonomous vehicles, high
definition maps, HD map data structure, and current standards in the industry were very difficult
to obtain or, for certain details, unobtainable. In particular, information regarding the data
structure of HD maps and requirements for such maps was not publicly available. Fortunately,
personal communication with members of participating companies provided some insight on the
subject of HD maps and their structure. Yet even here, some information could not be shared due
to the proprietary nature of the information.
Many automotive companies, their providers, and mapping companies are working
towards self-driving cars and maps to support them. If HD maps are a key ingredient to make
autonomous vehicles a reality, proprietary information on how to structure and create an HD
map would not be publicly available as competing companies could use that information to their
advantage. Even with a navigation data standard such as that being developed by the NDS
consortium, the participating companies do not share their technological advancements with one
another. Rather, the NDS format allows sharing of the end-products over various platforms.
5.4. The Feature Extraction Processes
The processing methods used in this research study included hands-on digitizing and
object extraction by the use of various tools. The requirements of the final data structure were
clear, and the processing methods were constructed accordingly. Manual delineation efforts
65
included the tracing of desired features in the orthomosaic and were a time-consuming process,
however, the outputs were therefore very controlled. Feature extraction by use of various tools
allows for significantly less manual input by the user. Successful combinations of tools can also
result in an automated model run or script development to extract the desired features. The model
workflows used in this research project were successful in extracting the desired features,
however, tool parameters were tested by trial and error as no guidelines or related research could
be found on tool-based extraction methods of parking lot features. In addition, the tools used did
not allow directional input of the lines or polygons created. Line and vertex direction could not
be predetermined for the tools, resulting in random line and vertex direction. As all data structure
requirements were fulfilled in the tool-based extraction method, the line feature classes were
converted to the NDS format. With random and non-readable vertex order, the resulting line
feature classes could not be displayed in the NDS format.
The extraction of vehicles in the Bertrandt parking lot was difficult. Some vehicles were
not parked correctly within the parking lines, other vehicles parked entirely on the road, and
again other vehicles were parked so far in the parking space that the front of the vehicles
overlapped into the vegetation extent (see Figure 26). In theory, extracting the vehicle as a
polygon with a small buffer could render the approximate area of a parking spot, however,
vehicles would have to be parked perfectly within the parking spot.
66
Figure 26. Illustration of parking spot challenge from vehicle extraction. Orange lines are the
buffered parking spot lines.
The extraction of parking lines in the German School study site worked very well. The
extracted parking lines rendered small polygons that matched the pattern of the parking spots.
With some manual input, the parking spaces were created from the parking lines. Overall, the
tool-based extraction results show that clearly marked parking spaces and empty parking lots are
beneficial for tool-based feature extraction.
67
5.5. Future Research
The advantages of drone-based data collection are several, including the possibility of
operating a number of drones at the same time. A large area, such as a company campus, could
be flown by a number of drones within just minutes. Other advantages of drone-based data
collection compared to traditional HD data collection methods include low-cost financial
investment, small data volumes, and no fossil fuel combustion.
To further investigate the use of aerial drones as a means of data collection to develop
HD maps, next steps may include the use of additional sensors or an upgraded aerial drone. As
drone-based surveying methods are growing, miniature LiDAR sensors are available for
installation on recreational drones. To gather point-cloud data similar to mapping-vehicles,
LiDAR sensors could be added to the aerial drone. Upgrades could also include the use of a more
accurate drone, such as a real-time kinematic (RTK) drone system. As implied by the name,
RTK is a GPS correction technology technique in which location data is corrected in real-time
while the survey drone captures the images (Rabkin 2018). Other advantages of new correction
technologies, such as RTK or post-processing kinematic (PPK) techniques, includes the removal
for need of ground control points (Rabkin 2018). The recreational Phantom 4 drone utilized in
this research study could be replaced by its technologically advanced successor, the DJI Phantom
4 RTK. According to an accuracy study carried out by DroneDeploy, the Phantom 4 RTK
delivers a relative horizontal accuracy of 1.2 cm and a linear measurement accuracy of 3.65 cm,
without the use of GCPs (Willoughby 2019). Eliminating the need of GCPs would significantly
decrease data collection time while maintaining high accuracy standards. At a price of $6,500,
the Phantom 4 RTK drone would continue to be a cost-efficient alternative to sensor-equipped
mapping-vehicles.
68
Future development may also include 3D aerial drone flights instead of, or in addition to,
2D flights. As described in Section 3.1, tree canopy clearance can be difficult when capturing 2D
imagery. Flight routes must be planned to be flown at an altitude that allows for tree canopy
clearance to avoid collisions. Unfortunately, the tree canopy often covers roads and other objects
that are important for extraction. Mapping on a 3D instead of 2D plane would allow the drone to
fly below the tree canopy line. In cases where roads are bordered by rows of trees that cover the
road from above, flying the drone on a 3D plane would allow the road to be followed and
mapped below the trees. Adding 3D flights would also allow the mapping of important features
such as bridges or powerlines that do not appear well in a 2D space. Knowing the existence of
bridges and powerlines and what heights they are would be extremely important for vehicles
above average heights, such as semi-trailer trucks. Another advantage of 3D flight includes the
generation of dense point clouds, similar to points clouds created by LiDAR sensors. From the
point clouds, additional information could be calculated such as traffic-light heights or bridge
heights.
Other future research could include the development of a tool or application that could
specify the direction of vertices in automatically extracted vector feature classes. Once the order
and direction of vertices in the lines can be determined, tool-based extracted objects from aerial
images could be displayed in the NDS format. As the method of manual delineation has clearly
defined and known values, this research study could be expanded to include a confusion matrix
that compares the results from the different methods. The confusion matrix would evaluate the
performance of the classification models. For example, the manual delineation method and tool-
based extraction method could be compared by dropping 1000 points on top of each dataset and
then comparing the extracted land uses found.
69
5.6. Final Conclusions
The use of drone-based remote sensing techniques to develop high quality datasets has
proven effective and accurate. With use of ground control points or other GPS correction
techniques, a low-cost drone could serve as an alternative mapping-method to current leading
mapping-techniques for the development of HD maps for autonomous vehicles.
Manual delineation was successful in extracting the desired features from the drone-
generated orthomosaics. Although manual delineation is a time-consuming process, the direction
of lines and vertices drawn can be controlled and is thus still the best-suited option for dataset
conversion to the NDS format. The tool-based extraction method applied a supervised
classification process by use of training samples, in addition to post-processing methods, and
proved effective in extracting objects from high resolution orthomosaics. The required data
structure for conversion to the NDS format was followed successfully and all datasets were
converted. Unfortunately, vertex direction could not be controlled in the tool-based extraction
technique and thus the resulting datasets could not be displayed in the NDS database inspector.
As the use of aerial drones and the data volume of imagery increases, the need for an automated
tool method to derive certain land covers increases. With further modifications to the tool-based
extraction method, it would show superior results to manual delineation.
Overall, this research shows that aerial drones are capable of producing high quality
imagery that can be used for HD map development, all in a time-efficient, cost-efficient, and
environmentally friendly manner.
70
References
Automotive World. 2018. “HD maps--the key to autonomous driving success?” Automotive
World, October 3, 2018. https://www.automotiveworld.com/articles/hd-maps-the-key-to-
autonomous-driving-success/.
Bonetti, Pino. 2016. “HERE introduces HD Live Map to show the path to highly automated
driving HD map” Disqus, January 5, 2016. https://360.here.com/2016/01/05/here-
introduces-hd-live-map-to-show-the-path-to-highly-automated-driving/.
Centers for Disease Control and Prevention. 2017. “Global Road Safety.” Last modified June 7,
2017. https://www.cdc.gov/motorvehiclesafety/global/index.html.
Chellapilla, Kumar. 2018. “Rethinking Maps for Self-Driving.” The Medium, October 15, 2018.
https://medium.com/@LyftLevel5/https-medium-com-lyftlevel5-rethinking-maps-for-
self-driving-a147c24758d6.
Chen, Anning, Arvind Ramanandan, and Jay A. Farrell. 2010. "High-precision lane-level road
map building for vehicle navigation." Paper presented at the IEEE/ION Position Location
and Navigation Symposium (PLANS), Indian Wells, CA, May 4-6, 2010, 1035-1042.
http://socal.ion.org/wp-content/uploads/2013/06/Presentation.pdf.
Dokic, Jadranka, Beate Müller, and Gereon Meyer. 2015. "European roadmap smart systems for
automated driving." European Technology Platform on Smart Systems Integration, April
1, 2015. http://www.a3ps.at/site/sites/default/files/newsletter/2015/no08/EPOSS.pdf.
Esri. 2019. “Multipart To Singlepart.” http://desktop.arcgis.com/en/arcmap/latest/tools/data-
management-toolbox/multipart-to-singlepart.html
Hyatt, Kyle. 2018. “Here’s why people still crash with driver assistance systems.” Road Show,
June 14, 2018. https://www.cnet.com/roadshow/news/heres-why-people-still-crash-with-
driver-assistance-systems/.
Here Technologies. 2018. “HD Live Map Data Specification.”
https://developer.here.com/olp/documentation/hd-live-map/topics/hdlm2-logical-
road.html.
Klemas, Victor V. 2015. "Coastal and environmental remote sensing from unmanned aerial
vehicles: An overview." Journal of Coastal Research 31 (5): 1260-1267.
doi: 10.2112/JCOASTRES-D-15-00005.1.
Kent, Leo. 2015. “HERE introduces HD maps for highly automated vehicle testing.” Disqus,
July 20, 2015. https://360.here.com/2015/07/20/here-introduces-hd-maps-for-highly-
automated-vehicle-testing/.
Kummerle, Rainer, Dirk Hahnel, Dmitri Dolgov, Sebastian Thrun, and Wolfram Burgard. 2009.
"Autonomous driving in a multi-level parking structure." In 2009 IEEE International
71
Conference on Robotics and Automation, Kobe, Japan, May 12-17, 2009, 3395-3400.
doi: 10.1109/ROBOT.2009.5152365.
Litman, Todd. 2017. “Autonomous vehicle implementation predictions: Implications for
Trnasport Planning.” Victoria Transport Policy Institute, March 18, 2019.
https://www.vtpi.org/avip.pdf.
Laliberte, Andrea S., Jeffrey E. Herrick, Albert Rango, and Craig Winters. 2010. "Acquisition,
orthorectification, and object-based classification of unmanned aerial vehicle (UAV)
imagery for rangeland monitoring." Photogrammetric Engineering & Remote Sensing 76,
(6): 661-672. doi: 10.14358/PERS.76.6.661.
Massow, Kay, B. Kwella, N. Pfeifer, F. Häusler, Jens Pontow, Ilja Radusch, Jochen Hipp, Frank
Dölitzscher, and Martin Haueis. 2016. "Deriving HD maps for highly automated driving
from vehicular probe data." In 2016 IEEE 19th International Conference on Intelligent
Transportation Systems (ITSC), Rio de Janeiro, Brazil, November 1-4, 2016, 1745-1752.
doi: 10.1109/ITSC.2016.7795794.
National Highway Traffic Safety Administration (NHTSA). 2018. “Traffic Safety Facts.” Last
modified April, 2018.
https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812517
Navigation data standard. 2019. “What is NDS?” https://nds-association.org
O'Neill-Dunne, Jariath. 2015. "Image boost: drones produce sharper images for mapping." Roads
& Bridges 53 (9). https://trid.trb.org/view/1372835.
Pix4D. 2019. “What is accuracy in an aerial mapping project?” Last modified April 25, 2019.
https://www.pix4d.com/blog/accuracy-aerial-mapping.
Putch, Andy. 2018. "Linear Measurement Accuracy of DJI Drone Platforms and
Photogrammetry." DroneDeploy, February 27, 2018. http://imageryelevated.com/linear-
measurement-accuracy-of-dji-drone-platforms-and-photogrammetry/.
Rabkin, Barry. 2018. “GCPs v. PPK/RTK: Which is Best to Receive Fast and Accurate Data?”
Identified Technologies, July 12, 2018. https://www.identifiedtech.com/blog/drone-
technology/gcps-ppk-rtk-best-receive-fast-accurate-data/.
Randall, Tom. 2019. “Waymo starts selling sensors to lower cost of self-driving cars.”
Bloomberg, March 6, 2019. https://www.bloomberg.com/news/articles/2019-03-
06/waymo-starts-selling-sensors-to-lower-cost-of-self-driving-cars.
Seif, Heiko G., and Xiaolong Hu. 2016. "Autonomous driving in the iCity—HD maps as a key
challenge of the automotive industry." Engineering 2 (2): 159-162.
doi: 10.1016/J.ENG.2016.02.010.
72
Synced. 2018. “The Golden Age of HD Mapping for Autonomous Driving” The Medium, August
11, 2018. https://medium.com/syncedreview/the-golden-age-of-hd-mapping-for-
autonomous-driving-b2a2ec4c11d.
TomTom. 2015 “TomTom delivers HAD Map for all highways in Germany.” Last modified
September 14, 2015. https://corporate.tomtom.com/news-releases/news-release-
details/tomtom-delivers-had-map-all-highways-germany?ReleaseID=931328
TomTom. 2017 “HD Map; Highly Accurate Border-To-Border Model of The Road.”
http://download.tomtom.com/open/banners/HD-Map-Product-Info-Sheet-improved-1.pdf.
TomTom. 2018. “TomTom HD Map with RoadDNA; High definition Map with Sensor-
Agnostic Localization.” https://www.tomtom.com/automotive/automotive-
solutions/automated-driving/hd-map-roaddna/.
Tzotsos, Angelos, and Demetre Argialas. 2008. "Support vector machine classification for
object-based image analysis." In Object-Based Image Analysis, 663-677. Springer,
Berlin, Heidelberg.
Van Brummelen, Jessica, Marie O’Brien, Dominique Gruyer, and Homayoun Najjaran. 2018.
"Autonomous vehicle perception: The technology of today and
tomorrow." Transportation research part C: emerging technologies 89 (April): 384-406.
doi: 10.1016/j.trc.2018.02.012.
Vardhan, Harsha. 2017. “HD Maps: New age maps powering autonomous vehicles.” Geospatial
World, September 22, 2017. https://www.geospatialworld.net/article/hd-maps-
autonomous-vehicles/.
Willoughby, James. 2019. “DJI Phantom 4 RTK: Survey-grade accuracy confirmed!” Heliguy,
January 31, 2019. https://www.heliguy.com/blog/2019/01/31/dji-phantom-4-rtk-
accuracy-confirmed/.
Winner, Hermann, Stephan Hakuli, and Gabriele Wolf. 2009. “Digitale Karten im Navigation
Data Standard Format.” In Handbuch Fahrerassistenzsysteme: Grundlagen,
Komponenten und Systeme für aktive Sicherheit und Komfort, 514-518. Springer Vieweg.
Xiang, Haitao, and Lei Tian. 2011. "Development of a low-cost agricultural remote sensing
system based on an autonomous unmanned aerial vehicle (UAV)." Biosystems
engineering 108 (2): 174-190. doi: 10.1016/j.biosystemseng.2010.11.010.
73
Appendix A: Enlarged Version of the NDS Building Blocks
74
Appendix B: Questions and Answers from HERE Technologies
75
76
Appendix C: Bertrandt Parking Lot; Pix4D Quality Report
77
78
79
80
81
82
83
84
85
Appendix D: German School Parking Lot; Pix4D Quality Report
86
87
88
89
90
91
92
93
94
Abstract (if available)
Abstract
The future of the automotive industry continues to head towards the development of autonomous vehicles. Without a human driver behind the wheel, the self-driving vehicle must be able to navigate itself within the road network. This research project investigates the application of aerial drones, also known as unmanned aerial vehicles (UAVs), as an alternative data collection method to create HD datasets for use in autonomous vehicles. Drones may be a low-cost alternative method to the current leading data collection method of sensor-equipped mapping-vehicles. A Phantom 4 drone was used in two case studies to create orthomosaics of parking lots. The drone-generated orthomosaics were processed by methods of manual delineation and tool-based extraction to evaluate different methods of processing high-resolution data. In addition, current HD data standards were acquired from various sources to evaluate the results of the research project and to compare data collection methods. The results show that drone-based data collection with GPS correction techniques can be an accurate and low-cost alternative method. Both manual delineation and tool-based extraction techniques proved successful in extracting desired feature classes from the high-resolution imagery.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
Utilizing advanced spatial collection and monitoring technologies: surveying topographical datasets with unmanned aerial systems
PDF
Multi-robot strategies for adaptive sampling with autonomous underwater vehicles
PDF
Exploitation of wide area motion imagery
Asset Metadata
Creator
Scherelis, Victoria Elvira
(author)
Core Title
An application of aerial drones in high definition mapping for autonomous vehicles
School
College of Letters, Arts and Sciences
Degree
Master of Science
Degree Program
Geographic Information Science and Technology
Publication Date
09/23/2019
Defense Date
08/16/2019
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
aerial drone,aerial imagery,ArcGIS,Autonomous,autonomous car,autonomous driving,autonomous vehicle,drone,drones,feature extraction,HD map,high definition mapping,image analysis,manual delineation,Model Builder,navigation data standard,NDS,OAI-PMH Harvest,object-based extraction,Pix4D,remote sensing,self-driving car,UAV,unmanned aerial vehicle
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Kemp, Karen (
committee chair
), Fleming, Steven (
committee member
), Marx, Andrew (
committee member
)
Creator Email
schereli@usc.edu,victoria.sch95@yahoo.de
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-c89-223324
Unique identifier
UC11673107
Identifier
etd-ScherelisV-7837.pdf (filename),usctheses-c89-223324 (legacy record id)
Legacy Identifier
etd-ScherelisV-7837.pdf
Dmrecord
223324
Document Type
Thesis
Rights
Scherelis, Victoria Elvira
Type
texts
Source
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the a...
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Tags
aerial drone
aerial imagery
ArcGIS
autonomous car
autonomous driving
autonomous vehicle
drone
drones
feature extraction
HD map
high definition mapping
image analysis
manual delineation
Model Builder
navigation data standard
NDS
object-based extraction
Pix4D
remote sensing
self-driving car
UAV
unmanned aerial vehicle