|
Temporal snapshot compressive imaging (SCI) allows high-dimensional temporal images to be reconstructed from a two-dimensional (2D) set of measurements. This is valuable for capturing color-polarized video data that can be used for robust material classification, considering that each material has a unique behavior in the way it polarizes the reflected light. In contrast to conventional commercial video cameras, which often compromise spatial, color, or polarization resolution to accommodate more information in the sensor, the SCI paradigm exploits optics, electronics and algorithms to produce high-resolution high-dimensional imaging from far fewer measurements. Commercial cameras can be adapted to capture additional information beyond their conventional sensing range when integrated with the SCI framework. This is achieved by incorporating an intensity modulation element that encodes and compress the data, and prevents incoherent sampling for nonlinear reconstruction based on compressive sensing principles. In this paper, an off-the-sheld camera is modified to compressively acquire and reconstruct high resolution spatio temporal polarization and color data from 2D measurements, leading to color-polarized video.The sensor camera uses a Bayer and a polarization filter superimposed on each other, aka RGB-P sensor. Then, a time-based designed coded aperture (CA) is incorporated into the optical path to temporally modulate each Bayer-polarized frame, where the modulated frames are integrated into a single measurement in the sensor. The CA is built with spatiotemporal block-unblock elements that encodes the information, in which its design restricts the distribution of those elements across the time dimension for reducing the temporal measurements redundancy, and, in turn, leading to better reconstructions. The compressed color-polarized video measurements are then recovered by using the alternating direction method of multipliers (ADMM) reconstruction algorithm. Numerical experiments show that temporal designed CA patterns outperforms random CA structures in terms of PSNR (Peak signal-to-noise ratio) and SSIM (structural similarity index measure), providing better-quality reconstructions of a color-polarized video from a dynamic scene.
|
