DNN-based video frame prediction can be a powerful tool to improve performance of motion-compensated prediction in video coding. In this paper, we propose a method applying multiple convolution kernels with different sizes to PredNet, which is one of the DNN-based video prediction schemes, to enhance the prediction accuracy by incorporating context adaptivity in its convolutional LSTM layers. We analyze the prediction performance of the proposal, and the results show that applying multiple-size kernels is effective than applying a single-size kernel in terms of prediction error reduction.
For generating motion-compensated predictive frames, which is one of the video coding processes, there has been a lot of studies on using DNN without using motion vectors. Conventional methods of generating motion-compensated predictive frames using DNN use only the source frames for prediction in the forward direction. However, in the ever-standardized video coding schemes, it has been confirmed that the bi-directional prediction, e.g., B-picture, improves coding efficiency. Thus, for generating motion-compensated predictive frames to be used in video coding, we propose to apply PredNet bidirectionally, that is a future frame generation model using DNN based on the prediction process of visual input stimuli in brain. In this paper, the performance of the predictive frames generated by the proposed method is evaluated by using MSE and SSIM compared with the prediction accuracy applying PredNet only to the forward direction. In addition, we also investigate whether the prediction accuracy of the predicted frames can be improved by increasing the amount of training frames in videos chosen from YouTube-8M. The results show the effectiveness of the proposed method in terms of less prediction error compared with the forward-only PredNet, as well as the performance increasing by more training data.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.