A vision system using depth inpainting for virtual content reconstruction in Augmented Reality

V. Voronin; N. Gapon; I. Khamidullin; O. Tokareva; A. Zelensky; E. Semenishchev

doi:10.1117/12.2642007

28 October 2022 A vision system using depth inpainting for virtual content reconstruction in Augmented Reality

V. Voronin, N. Gapon, I. Khamidullin, O. Tokareva, A. Zelensky, E. Semenishchev

Proceedings Volume 12275, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies VI; 122750M (2022) https://doi.org/10.1117/12.2642007
Event: SPIE Security + Defence, 2022, Berlin, Germany

Conference Poster

Abstract

Augmented Reality (AR) applications demand realistic rendering of virtual content in a variety of environments, so they require an accurate description of the 3-D scene. In most case AR system is equipped with Time-of-Flight (ToF) cameras to provide real-time scene depth maps, but they have problems that affect the quality of depth data, which ultimately makes them difficult to use for AR. Such defects appear because of poor lighting, specular or fine-grained surfaces of objects. As a result, the effect of increasing the boundaries of objects appears, and the overlapping of objects makes it impossible to distinguish one object from another. The article presents an approach based on a modified algorithm for searching for similar blocks using the concept of anisotropic gradient. A proposed modified exemplar block-based algorithm uses the autoencoder-learned local image descriptor for image inpainting, that extract the features of images, and the depth image by a decoding network. The encoder consists of a convolutional layer and a dense block, which also consists of convolutional layers. We also show the application for the proposed vision system using depth inpainting for virtual content reconstruction in augmented reality. Analysis of the results of the study shows that the proposed method allows you to correctly restore the boundaries of objects on the image of the depth map. Our system quantitatively outperforms state-of-the-art methods in terms of reconstruction accuracy in the real and simulated benchmark datasets.

Citation Download Citation

V. Voronin, N. Gapon, I. Khamidullin, O. Tokareva, A. Zelensky, and E. Semenishchev "A vision system using depth inpainting for virtual content reconstruction in Augmented Reality", Proc. SPIE 12275, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies VI, 122750M (28 October 2022); https://doi.org/10.1117/12.2642007

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available