Poster + Paper
22 November 2024 Point cloud processing for mixed reality: semantic understanding, mesh generation, and completion
Author Affiliations +
Conference Poster
Abstract
This research presents a comprehensive exploration of point cloud processing, with a particular emphasis on its application in mixed reality systems. It delves into the intricacies of semantic understanding, which is a sophisticated dimension of point cloud analysis that transcends basic segmentation by assigning meaningful labels to each segmented region. This process imparts semantic awareness to segments, providing a deeper understanding of the 3D scene captured.

The study further discusses the process of mesh generation and replacement, which is a pivotal element in computer graphics and 3D modeling. This process involves the creation of a mesh—a three-dimensional representation of a scene or object comprised of connected triangles. The significance of meshes extends beyond mere presentation, as they serve as the bedrock for dynamic, interactive 3D worlds.

The research introduces a novel framework for point cloud completion that leverages attention mechanisms to capture the structural information of 3D shapes. This framework eliminates the need for explicit local region operations, alleviating the influence of data density distribution and achieving high-quality complete shapes with precise geometrical details. The findings from this research provide a holistic understanding of point cloud processing, contributing significantly to tasks such as object recognition, tracking, real-time mesh building, and completing 3D shapes from partial 3D point clouds. These tasks are essential for enhancing the user experience in mixed reality systems, where understanding the semantic meaning of elements in a three-dimensional scene is paramount.
(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.
Maksim Sorokin, Dmitry Zhdanov, Andrei Zhdanov, and Julia Koroleva "Point cloud processing for mixed reality: semantic understanding, mesh generation, and completion", Proc. SPIE 13239, Optoelectronic Imaging and Multimedia Technology XI, 132390X (22 November 2024); https://doi.org/10.1117/12.3035967
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Point clouds

Mixed reality

Image segmentation

Semantics

3D modeling

Transformers

Neural networks

Back to Top