The problem of assisting a low-vision person with environment awareness using a drone is addressed. Specifically, the first stage task of detecting the User and their heading which does not require any user adaptive training is tackled. The modalities of 3D and 2D vision on a drone are compared for this task. 3D data is provided using a stereo sensor mounted on the drone that communicates using RF to a mobile device based android application. For the task of localization, a Single Shot MulitBox Detector is utilized. Different networks in terms of input modalities and related structure are developed including a 2D only network and a 3D+2D fused network. Performance of these networks are compared and results discused. In addition, a comparison of retrained networks versus training from scratch is made. In all cases, approximately 34,000 user heading samples were collected for training. Real data from outdoor drone flights that communicate with the Android based application are shown. Detecting both the user in the scene and their heading is an important first step necessary in a drone-based system that helps low-vision persons with environment awareness. The success and challenges faced are presented along with future avenues of work.
KEYWORDS: Artificial intelligence, Information fusion, Data modeling, Systems modeling, Sensors, Data fusion, Algorithm development, Evolutionary algorithms, Detection and tracking algorithms, Neural networks
During the 2018 SPIE DSS conference, panelists were invited to highlight the trends and use of artificial intelligence and deep learning (AI/DL) for information fusion. This paper highlights the common issues presented from the panel discussion. The key issues include: leveraging AI/DL coordinated with information fusion for: (1) knowledge reasoning and reasoning, (2) information fusion enhancement, (3) object recognition and tracking, (4) data with models fusion, and (5) deep multimodal fusion cognition strategies to support the user.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.