A novel virtual viewpoint merging method based on machine learning

Di Zheng; Zongju Peng; Hui Wang; Gangyi Jiang; Fen Chen

doi:10.1117/12.2073526

4 November 2014 A novel virtual viewpoint merging method based on machine learning

Di Zheng, Zongju Peng, Hui Wang, Gangyi Jiang, Fen Chen

Proceedings Volume 9273, Optoelectronic Imaging and Multimedia Technology III; 92730P (2014) https://doi.org/10.1117/12.2073526
Event: SPIE/COS Photonics Asia, 2014, Beijing, China

Abstract

In multi-view video system, multiple video plus depth is main data format of 3D scene representation. Continuous virtual views can be generated by using depth image based rendering (DIBR) technique. DIBR process includes geometric mapping, hole filling and merging. Unique weights, inversely proportional to the distance between the virtual and real cameras, are used to merge the virtual views. However, the weights might not the optimal ones in terms of virtual view quality. In this paper, a novel virtual view merging algorithm is proposed. In the proposed algorithm, machine learning method is utilized to establish an optimal weight model. In the model, color, depth, color gradient and sequence parameters are taken into consideration. Firstly, we render the same virtual view from left and right views, and select the training samples by using a threshold. Then, the eigenvalues of the samples are extracted and the optimal merging weights are calculated as training labels. Finally, support vector classifier (SVC) is adopted to establish the model which is used for guiding virtual views rendering. Experimental results show that the proposed method can improve the quality of virtual views for most sequences. Especially, it is effective in the case of large distance between the virtual and real cameras. And compared to the original method of virtual view synthesis, the proposed method can obtain more than 0.1dB gain for some sequences.

Citation Download Citation

Di Zheng, Zongju Peng, Hui Wang, Gangyi Jiang, and Fen Chen "A novel virtual viewpoint merging method based on machine learning", Proc. SPIE 9273, Optoelectronic Imaging and Multimedia Technology III, 92730P (4 November 2014); https://doi.org/10.1117/12.2073526

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Video

Machine learning

Cameras

Image fusion

Scalable video coding

Distortion

Statistical modeling

Show All Keywords

Keywords/Phrases

Search In:

Publication Years