3D multi-perspective depth detection using point clouds and machine learning

Andrew Esteves; Harry Bickford; Jaesung Yang; Xin Shen; Kiwon Sohn

doi:10.1117/12.3014029

7 June 2024 3D multi-perspective depth detection using point clouds and machine learning

Andrew Esteves, Harry Bickford, Jaesung Yang, Xin Shen, Kiwon Sohn

Proceedings Volume 13041, Three-Dimensional Imaging, Visualization, and Display 2024; 130410N (2024) https://doi.org/10.1117/12.3014029
Event: SPIE Defense + Commercial Sensing, 2024, National Harbor, Maryland, United States

Conference Poster

Abstract

Accurate object detection and depth estimation is critical for a variety of applications such as autonomous driving and robotics. In the context of object avoidance, one may use a LiDAR sensor to determine the position of nearby objects but, due to a lack of resolution, these sensors cannot be used to accurately categorize and label the object being detected. To contrast this, RGB cameras can provide rich semantic information, which can be used to categorize and segment an object but cannot provide accurate depth data. To overcome this, an abundance of algorithms has been created which are capable of fusing the two sensors, among others, allowing for accurate depth detection and segmentation of a given object. The problem with many of these systems is that they are complex in their approach and create 3D bounding boxes, which can result in an agent taking a less optimal path due to the size of the perceived object. The proposed approach in this paper simply determines the position of an object in an RGB image, using a CNN, and then translates two dimensions, found through the center pixel of the bounding box, to a point cloud to identify and segment point clusters.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Andrew Esteves, Harry Bickford, Jaesung Yang, Xin Shen, and Kiwon Sohn "3D multi-perspective depth detection using point clouds and machine learning", Proc. SPIE 13041, Three-Dimensional Imaging, Visualization, and Display 2024, 130410N (7 June 2024); https://doi.org/10.1117/12.3014029

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available