13 March 2023 Info-Giraffe: virtual neural feature field generation for object in limited views condition
Fan Zhou, Jinyi Zhang, Yuxi Jiang, Tianxing Zhang
Author Affiliations +
Abstract

In the field of three-dimensional (3D) reconstruction, learning-based neural radiation field (NeRF) and 3D-aware generative model (3D-aware GAN) set the foundation for implicit reconstruction of a 3D object. However, this method has problems of slow reconstruction speed and uncontrollable reconstruction results. For this reason, we proposed Info-Giraffe, a virtual neural feature field generation method for object in limited views condition. Info-Giraffe is divided into two phases: pretraining and retraining. First, we generate the basic neural feature field of the object using the simplified Giraffe model in the pretraining phase. Then, during the retraining phase, we extract the 3D information of the object in limited views using a set of latent code encoders designed based on InfoGAN, which is then reparameterized into the latent space. Finally, we continue migration learning by introducing 3D information on the foundation of the object’s basic neural feature field and generate a controllable virtual neural feature field of the object in limited views, which enables fast 3D reconstruction of the object and control of the reconstruction results. Experimental results on Chairs 642 and CompCars 2562 natural image datasets reveal the effectiveness of the proposed method.

© 2023 SPIE and IS&T
Fan Zhou, Jinyi Zhang, Yuxi Jiang, and Tianxing Zhang "Info-Giraffe: virtual neural feature field generation for object in limited views condition," Journal of Electronic Imaging 32(2), 023009 (13 March 2023). https://doi.org/10.1117/1.JEI.32.2.023009
Received: 25 October 2022; Accepted: 20 February 2023; Published: 13 March 2023
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
3D modeling

Image restoration

3D image processing

3D image reconstruction

Cameras

Education and training

Visualization

Back to Top