KEYWORDS: Motion estimation, Data processing, Video, Motion models, Image processing, 3D modeling, 3D image processing, Cameras, Chemical elements, Machine vision
Global motion estimation is a fundamental tool for image processing and computer vision. Since low-cost real-time devices have limited computational resources, the computational burden of global motion estimation remains considerable for low-cost devices. Hence, a fast global motion estimation algorithm for single instruction multiple data (SIMD) processors with which a CPU in modern mobile computing devices is commonly equipped is proposed. Conventional projection-based global motion estimation is modified to improve the prediction accuracy for videos containing large motions. To significantly increase the parallelism, the exact sizes of the variables in each processing step of the proposed algorithm are determined by considering the size of an SIMD register. Simulation results demonstrate that the proposed scheme is considerably fast and accurate, even for videos containing large motions.
Complementary metal-oxide semiconductor image sensors suffer from undesirable geometric image distortion due to the rolling shutter effect. However, conventional algorithms cannot be used for strict real-time use in mobile devices. An extremely low computational method to compensate rolling shutter effect for the mobile devices is proposed. The proposed method efficiently predicts the rotational camera motion in three-dimensional space from the global motion vectors predicted in two-dimensional image space. Hence, the computational complexity is extremely reduced. To further improve the video quality, the video stabilization to reduce the high-frequency jitter of the camera motion without increasing the complexity is introduced. The results of simulation demonstrate that while the computational complexity is extremely low, the proposed algorithm provides acceptable performance.
A video stabilization method based on a new concept inspired by the human visual system is presented. The human eye provides a stable scene by continuously changing the eye’s orientation in a way that always places the focused target in the center of one’s view. Similar to the human eye, the proposed algorithm focuses only on a single target object within a scene and stabilizes the target on the two-dimensional image plane by rotating a camera in three-dimensional space while most previous methods consider all objects in a video. The rotational angles of the camera along the x and y axes are simply predicted from a translational motion vector of the target object on the image plane. Hence, the proposed algorithm can provide a vivid video as if it was seen through the eye. Efficient video stabilization by approximating the human visual system is introduced, a practical method for real-time devices. Experimental results demonstrate that the visual feelings of the compensated videos are different depending on the selected target object and the approximating method provides a reasonable performance.
Rolling shutter effect commonly exists in a video camera or a mobile phone equipped with a complementary metal-oxide semiconductor sensor, caused by a row-by-row exposure mechanism. As video resolution in both spatial and temporal domains increases dramatically, removing rolling shutter effect fast and effectively becomes a challenging problem, especially for devices with limited hardware resources. We propose a fast method to compensate rolling shutter effect, which uses a piecewise quadratic function to approximate a camera trajectory. The duration of a quadratic function in each segment is equal to one frame (or half-frame), and each quadratic function is described by an initial velocity and a constant acceleration. The velocity and acceleration of each segment are estimated using only a few global (or semiglobal) motion vectors, which can be simply predicted from fast motion estimation algorithms. Then geometric image distortion at each scanline is inferred from the predicted camera trajectory for compensation. Experimental results on mobile phones with full-HD video demonstrate that our method can not only be implemented in real time, but also achieve satisfactory visual quality.
A context-adaptive-variable-length-coding (CAVLC) decoder cannot determine an exact start position of the k 'th syntax element until it decodes the (k−1 )'th syntax element, which makes parallel CAVLC decoding difficult. We propose a new bitstream structure to maximize parallelism of CAVLC decoding. Our algorithm enables us to simultaneously access multiple points from different MBs in a bitstream. Then, a CAVLC decoder can concurrently read multiple symbols from the multiple access points and decode them in parallel. Experimental results show that the proposed algorithm significantly increases decoding speed without sacrificing coding efficiency.
KEYWORDS: Video, Chemical elements, Computer programming, Video coding, Parallel processing, Data communications, Multimedia, Image processing algorithms and systems, Digital electronics, Electronics
A context adaptive variable length coding (CAVLC) decoder do not know the exact start position of the k-th syntax
element in a bitstream until it finishes parsing the (k-1)-th syntax element. It makes a parallel CAVLC decoding difficult.
It significantly increases implementation cost to predict the exact start position of a syntax element prior to parsing its
previous one. In this paper, we propose a new bitstream structure to concurrently access multiple syntax elements for
parallel CAVLC decoding. The method divides a bit-stream into N kinds of segments whose size is M bits and puts
syntax elements into the segments, based on a proposed rule. Then, a CAVLC decoder can simultaneously access N
segments to read N syntax elements from a single bitstream and decode them in parallel. This technique increases the
speed of CAVLC decoding by up to N times. Since the method just rearranges the generated bitstream, it does not affect
coding efficiency. Experimental results show that the proposed algorithm significantly increases decoding speed.
A lenticular system provides optimal display quality when a homogeneous lenticular sheet is precisely aligned on LCD subpixels and when a viewer is located within a predetermined viewing zone. In practice, however, many lenticular systems suffer from image distortion due to misalignment or inhomogeneity of the lenticular sheet. To alleviate such distortion, we propose a new multiplexing algorithm. The proposed algorithm first obtains the geometric relationship between LCD subpixels and lenticules by using various pattern images. Then, it generates a mapping matrix between LCD subpixels and multiple-view images based on the obtained relationship. While the proposed scheme is quite simple, it effectively compensates any misalignment or inhomogeneity of the lenticular sheet, and it significantly improves 3-D image quality. In addition, by simply adjusting the mapping parameters, the algorithm can change the predetermined viewing zone to include a viewer's actual position. Experimental results demonstrate that the proposed compensation scheme successfully works in a real 3-D lenticular display system.
Lenticular displays are a promising form of autostereoscopic technology and several products have recently been commercialized. For lenticular displays, typically several views of a scene are acquired from different viewpoints, or are generated by using depth information. If the depth information of a scene is known, then it is possible to easily generate several intermediate views simply by using the stereo image pair and depth information. The paper presents a simple method to correct the lenticular alignment error by compensating the correction coefficients to the viewpoint determination formula. For the realization of a fast lenticular display and removal of image distortion and artifacts, the proposed algorithm simultaneously performs intermediate floating-pointview interpolation and multiplexing on the scanline using the left-view and right-view images and depth information. Experimental results show that lenticular images having considerably reduced distortion and artifacts are generated by using the proposed algorithm.
Among various autostereoscopic display systems, the lenticular display is one of the most popular systems due to its easy manufacturability. For N-view lenticular display, N view images are to be regularly sub-sampled and interleaved to produce a 3D image. A lenticular system provides the best quality only when a viewer locates at a pre-determined optimal viewing distance and the lenticular sheet is precisely aligned on the LCD pixel array. In our previous work, we have proposed an algorithm to compensate the viewer's position change and the lenticular misalignment. However,
since the previous algorithm requires a considerable computational burden, we propose a new fast multiplexing algorithm. To improve the processing speed, we introduce a mapping table instead of directly using complex equations. In contrary to the previous algorithm, the proposed one can make real time compensation possible without degrading image quality.
Stereo video becomes an important issue with the developments of 3D display technologies. While a stereo system provides the perception of 3D depth, the amount of data for stereo video may be doubled compared to mono video. So an efficient stereo video coding technique is essential. Since the stereo video is taken from the same object in the two different views, most of objects in a stereo pair are translated only to the horizontal direction, while objects in
subsequent frames of a mono video can be translated to an arbitrary direction, rotated, and zoomed. Hence, unlike in the mono video, the disparity in a stereo pair can be well estimated by a translational motion or disparity if the object boundary is exactly described. In this paper, we propose an efficient disparity estimation scheme based on a edge model describing object boundary in a block, and apply the estimated disparity to stereo video compression. In addition, a
disparity regularization scheme, which is proper for the edge model, is proposed to reduce the bits required for coding the block motion vectors and disparity values. It has been found that the proposed algorithms significantly improve the coding efficiency of stereo video sequences.
In manufacturing a lenticular display system, precise alignment of the lenticular sheet on the LCD panel may not be practically achievable. Hence, observed view images inevitably produce unwanted distortion. We propose a novel method to alleviate the display distortion of each observed view image in a lenticular 3-D display. We first derive the relationship between subpixel values on the LCD pixel array and the image to be observed at each viewing zone in terms of system design parameters and the viewer's eye position. Based on this relationship, we analyze the distortion between the observed and original view images. We then derive a compensation algorithm to minimize the distortion and generate high-quality 3-D images. To verify the proposed scheme, we examine displayed results from several 3-D images of synthetic and real scenes. The results demonstrate that the proposed scheme significantly reduces distortions and improves the image quality in the lenticular display system.
A lenticular display system provides 3D images to a viewer without wearing glasses. For N-view lenticular display, N view images are N:1 sub-sampled and multiplexed to generate a multi-view image, Then, the generated image is allocated to the LCD pixel array. Since the lenticular sheet may not be exquisitely placed on an LCD panel without alignment error, and the rays from a viewer’s eye to lenticules on the LCD panel are not parallel, any view image observed from a multi-view image inevitably produces undesirable distortion. In this paper, we propose a novel method to alleviate the display distortion of each view image in the lenticular display. In this method, we first derive the relationship between pixel values on the LCD pixel array and the image to be observed at each viewing zone in terms of hardware parameters and viewer’s eye position. Based on this relationship, we analyze the distortion between the observed and original view images. Finally, we derive the compensation algorithm to minimize the distortion and generate high quality a 3D image. To verify the proposed scheme, we examine the displayed results from several 3D images of synthetic and real scenes. The experimental results show that the proposed scheme significantly reduces distortions and improves the image quality in the lenticular display.
As many fast integer-pixel motion estimation algorithms have become available, an integer-pixel motion vector can be found by examining less than 10 search points. Meanwhile, 8 half-pixel positions around the integer-pixel motion vector are to be examined for half-pixel motion estimation. Hence, it becomes more meaningful to reduce the computational complexity of half-pixel motion estimation. In this paper, we propose a fast half-pixel motion estimation algorithm, by combining a directional search and linear modeling of SAD curve. The proposed algorithm reduces the number of search points to 2.21 in average, while the image quality of reconstructed sequences in terms of PSNR is similar to existing fast half-pixel motion estimation algorithms. In addition, by adjusting a user-defined parameter, the proposed algorithm can significantly reduce the number of search points to 0.34 on average with a slight PSNR degradation.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.