Real-time object detection and geolocation using 3D calibrated camera/LiDAR pair

Brian Robinson; Darrell Langford; John Jetton; Logan Cannan; Katie Patterson; Robert Diltz; Woody English

doi:10.1117/12.2586043

12 April 2021 Real-time object detection and geolocation using 3D calibrated camera/LiDAR pair

Brian Robinson, Darrell Langford, John Jetton, Logan Cannan, Katie Patterson, Robert Diltz, Woody English

Proceedings Volume 11748, Autonomous Systems: Sensors, Processing, and Security for Vehicles and Infrastructure 2021; 117480A (2021) https://doi.org/10.1117/12.2586043
Event: SPIE Defense + Commercial Sensing, 2021, Online Only

Abstract

We present results from testing a multi-modal sensor system (consisting of a camera, LiDAR, and positioning system) for real-time object detection and geolocation. The system’s eventual purpose is to assess damage and detect foreign objects on a disrupted airfield surface to reestablish a minimum airfield operating surface. It uses an AI to detect objects and generate bounding boxes or segmentation masks in data acquired with a high-resolution area scan camera. It locates the detections in the local, sensor-centric coordinate system in real time using returns from a low-cost commercial LiDAR system. This is accomplished via an intrinsic camera calibration together with a 3D extrinsic calibration of the camera- LiDAR pair. A coordinate transform service uses data from a navigation system (comprising an inertial measurement unit and global positioning system) to transform local coordinates of the detections obtained with the AI and calibrated sensor pair into earth-centered coordinates. The entire sensor system is mounted on a pan-tilt unit to achieve 360-degree perception. All data acquisition and computation are performed on a low SWAP-C system-on-module that includes an integrated GPU. Computer vision code runs real-time on the GPU and has been accelerated using CUDA. We have chosen Robot Operating System (ROS1 at present but porting to ROS2 in the near term) as the control framework for the system. All computer vision, motion, and transform services are configured as ROS nodes.

Conference Presentation

Citation Download Citation

Brian Robinson, Darrell Langford, John Jetton, Logan Cannan, Katie Patterson, Robert Diltz, and Woody English "Real-time object detection and geolocation using 3D calibrated camera/LiDAR pair", Proc. SPIE 11748, Autonomous Systems: Sensors, Processing, and Security for Vehicles and Infrastructure 2021, 117480A (12 April 2021); https://doi.org/10.1117/12.2586043

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available