Unreal engine-based photorealistic aerial data generation and unit testing of artificial intelligence algorithms

Andrew Buck; Raub Camaioni; Brendan Alvey; Derek T. Anderson; James M. Keller; Robert H. Luke III; Grant Scott

doi:10.1117/12.2618812

27 May 2022 Unreal engine-based photorealistic aerial data generation and unit testing of artificial intelligence algorithms

Andrew Buck, Raub Camaioni, Brendan Alvey, Derek T. Anderson, James M. Keller, Robert H. Luke III, Grant Scott

Author Affiliations +

Proceedings Volume 12099, Geospatial Informatics XII; 1209908 (2022) https://doi.org/10.1117/12.2618812
Event: SPIE Defense + Commercial Sensing, 2022, Orlando, Florida, United States

Abstract

A number of real-time object detection, tracking, and autonomy artificial intelligence (AI) and machine learning (ML) algorithms are being proposed for unmanned aerial vehicles (UAVs). A big challenge is can we stress test these algorithms, identify their strengths and weaknesses, and assess if the UAV is safe and trustworthy? The process of collecting real-world UAV data is costly, time consuming, and riddled by lack of quality geospatial ground truth and metadata. Herein, we outline a fully automated framework and work ow to address the above challenges using free or low-cost assets, the photorealistic Unreal Engine (UE), and AirSim aerial platform simulator. Specifically, we discuss the rapid prototyping of an outdoor environment combined with the robotic operating system (ROS) for abstracting UAV data collection, control, and processing. Real and accurate ground truth is collected and metrics are presented for individual frame and entire flight collection evaluation. Metrics recorded and analyzed include percentage of scene mapped, 3D mapping accuracy, time to complete task, object detection and tracking statistics, battery usage, altitude (from ground), collisions, and other statistics. These metrics are computed in general and with respect to context, e.g., clutter, view angle, etc. Overall, the proposed work is an automated way to explore UAV operation before real-world testing or deployment. Promising preliminary results are discussed for an outdoor environment with vegetation, short and long range objects, buildings, people, vehicles, and other features for a UAV performing loitering and interrogation.

Conference Presentation

Citation Download Citation

Andrew Buck, Raub Camaioni, Brendan Alvey, Derek T. Anderson, James M. Keller, Robert H. Luke III, and Grant Scott "Unreal engine-based photorealistic aerial data generation and unit testing of artificial intelligence algorithms", Proc. SPIE 12099, Geospatial Informatics XII, 1209908 (27 May 2022); https://doi.org/10.1117/12.2618812

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available