This PDF file contains the front matter associated with SPIE Proceedings Volume 8299, including the Title Page, Copyright Information, Table of Contents, and the Conference Committee listing.

Digital sensors have obviously invaded the photography mass market. However, some photographers with very high expectancy still use silver halide film. Are they only nostalgic reluctant to technology or is there more than meets the eye? The answer is not so easy if we remark that, at the end of the golden age, films were actually scanned before development. Nowadays film users have adopted digital technology and scan their film to take advantage from digital processing afterwards. Therefore, it is legitimate to evaluate silver halide film "with a digital eye", with the assumption that processing can be applied as for a digital camera. The article will describe in details the operations we need to consider the film as a RAW digital sensor. In particular, we have to account for the film characteristic curve, the autocorrelation of the noise (related to film grain) and the sampling of the digital sensor (related to Bayer filter array). We also describe the protocol that was set, from shooting to scanning. We then present and interpret the results of sensor response, signal to noise ratio and dynamic range.

Sensor yield is directly related to the probability of defective pixel occurrence and the screening criteria. Assuming a spatially independent distribution of single pixel defects, effective on-the-fly correction of singlepixel defects in a color plane, and effective correction of two-pixel defects in a color plane (couplets) through a defect map, sensor yield can be computed based on the occurrence of three adjacent defective pixels in a color plane (triplets). Closed-form equations are derived for calculating the probability of occurrence of couplets and triplets as a function of the probability of a single pixel being defective. If a maximum of one triplet is allowed in a 5-megapixel sensor, to obtain a 98% yield, the probability of a pixel being defective (p) must not exceed 1.3E-3 (6500 defective pixels). For an 8-megapixel sensor, the corresponding requirement would be p < 1.1E-3 (8900 defective pixels). Numerical simulation experiments have confirmed the accuracy of the derived equations.

A cost-effective and convenient approach for color imaging is to use a single sensor and mount a color filter array (CFA) in front of it, such that at each spatial position the scene information in only one color channel is captured. To estimate the missing colors at each pixel, a demosaicing algorithm is applied to the CFA samples. Besides the filter arrangement and the demosaicing method, the spectral sensitivity functions of the CFA filters considerably affect the quality of the demosaiced image. In this paper, we propose an algorithm to compute the optimum spectral sensitivities of filters in the single sensor imager. The proposed algorithm solves a constrained optimization problem to find optimum spectral sensitivities and the corresponding linear demosaicing method. An important constraint for this problem is the smoothness of spectral sensitivities, which is imposed by modeling these functions as a linear combination of several smooth kernels. Simulation results verify the effectiveness of the proposed algorithm in finding optimal spectral sensitivity functions, which outperform measured camera sensitivity functions.

We propose a multi-component metric for the evaluation of digital or video cameras under wide dynamic range (WDR) scenes. The method is based on a single image capture using a specifically designed WDR test chart and light box. Test patterns on the WDR test chart include gray ramps, color patches, arrays of gray patches, white bars, and a relatively dark gray background. The WDR test chart is professionally made using 3 layers of transparencies to produce a contrast ratio of approximately 110 dB for WDR testing. A light box is designed to provide a uniform surface with light level at about 80K to 100K lux, which is typical of a sunny outdoor scene. From a captured image, 9 image quality component scores are calculated. The components include number of resolvable gray steps, dynamic range, linearity of tone response, grayness of gray ramp, number of distinguishable color patches, smearing resistance, edge contrast, grid clarity, and weighted signal-to-noise ratio. A composite score is calculated from the 9 component scores to reflect the comprehensive image quality in cameras under WDR scenes. Experimental results have demonstrated that the multi-component metric corresponds very well to subjective evaluation of wide dynamic range behavior of cameras.

The Transverse Field Detector (TFD), a filter-less and tunable color sensitive pixel, is based on the generation of specific electric field configurations within a depleted Silicon volume. Each field configuration determines a set of three or more spectral responses that can be used for direct color acquisition at each pixel position. In order to avoid unpredictable changes of the electric field configuration during the single image capture, a specific active pixel (AP) has been designed. In this AP the dark- and photo-generated charge is not integrated directly on the junction capacitance, but, for each color, it is integrated on the feedback capacitance of a single-transistor charge pre-amplifier. The AP further includes a bias transistor, a reset transistor and a follower. In this work the design of such a pixel is discussed and the experimental results obtained on a 2x2 matrix of these active pixels are analyzed in terms of spectral response, linearity, noise, dynamic range and repeatability.

For small camera modules in consumer applications, such as mobile phones or webcams, size and cost are important constraints. An autofocus system increases both size and cost and can degrade optical performance by misalignment. Therefore, a monolithic optical system with a fixed focus is preferable for these applications. On the other hand, the optical system of the camera has to exhibit a very large depth of field, as it is expected to deliver sharp images for all typical working distances. The depth of field of a camera system can be increased by using a larger F-number, but this is undesirable due to light sensitivity considerations. On the other hand, it can also be increased by reducing focal length. Multi-aperture systems use multiple optical channels, each of them with a smaller focal length than a comparable single-aperture system. Accordingly, each of the channels has a large depth of field. However, as the channels are displaced laterally, parallax becomes noticeable for close objects. Therefore, the channel images have to be shifted accordingly when recombining them into a complete image. We demonstrate an algorithm that compensates for parallax as well as chromatic aberration and geometric distortion. We present a very flat camera system without moving parts that is capable of taking photos and video at a wide range of distances. On the demonstration system, object distance can be adjusted in real time from 4 mm to infinity. The focus position can be selected during capture or after the images were taken.

The focused plenoptic camera is based on the Lippmann sensor: an array of microlenses focused on the pixels of a conventional image sensor. This device samples the radiance, or plenoptic function, as an array of cameras with large depth of field, focused at a certain plane in front of the microlenses. For the purpose of digital refocusing (which is one of the important applications) the depth of field needs to be large, but there are fundamental optical limitations to this. The solution of the above problem is to use and array of interleaved microlenses of different focal lengths, focused at two or more different planes. In this way a focused image can be constructed at any depth of focus, and a really wide range of digital refocusing can be achieved. This paper presents our theory and results of implementing such camera. Real world images are demonstrating the extended capabilities, and limitations are discussed.

Plenoptic cameras are intended to fully capture the light rays in a scene. Using this information, optical elements can be applied to a scene computationally rather than physically-allowing an infinite variety of pictures to be rendered after the fact from the same plenoptic data. Practical plenoptic cameras necessarily capture discrete samples of the plenoptic function, which together with the overall camera design, can constrain the variety and quality of rendered images. In this paper we specifically analyze the nature of the discrete data that plenoptic cameras capture, in a manner that unifies the traditional and focused plenoptic camera designs. We use the optical properties of plenoptic cameras to derive the geometry of discrete plenoptic function capture. Based on this geometry, we derive expressions for expected resolution from a captured plenoptic function. Our analysis allows us to define the "focused plenoptic condition," a necessary condition in the optical design that distinguishes the traditional plenoptic camera from the focused plenoptic camera.

Plenoptic cameras are designed to capture different combinations of light rays from a scene, sampling its lightfield. Such camera designs capturing directional ray information enable applications such as digital refocusing, rotation, or depth estimation. Only few address capturing spectral information of the scene. It has been demonstrated that by modifying a plenoptic camera with a filter array containing different spectral filters inserted in the pupil plane of the main lens, sampling of the spectral dimension of the plenoptic function is performed. As a result, the plenoptic camera is turned into a single-snapshot multispectral imaging system that trades-off spatial with spectral information captured with a single sensor. Little work has been performed so far on analyzing diffraction effects and aberrations of the optical system on the performance of the spectral imager. In this paper we demonstrate simulation of a spectrally-coded plenoptic camera optical system via wave propagation analysis, evaluate quality of the spectral measurements captured at the detector plane, and demonstrate opportunities for optimization of the spectral mask for a few sample applications.

An observer study was run to determine the detection thresholds of several representative examples of column fixed pattern noise, in the presence of varying levels of shot noise, which is known to mask structured noise. The data obtained were fit well at relevant shot noise levels by a simple model based on signal detection theory. Individual metrics of fixed pattern noise and shot noise, used in the masking equation, were computed from one dimensional integrations involving the capture noise power spectra (mapped to CIELAB space); the modulation transfer function of the display; the display pixel pitch; the viewing distance; and the S-CIELAB luminance contrast sensitivity function. The results of this work can be used to predict detection thresholds that can be added to photon transfer curves for the purpose of determining whether fixed pattern noise will be visible.

Recently, research of Objective Image Quality Assessment (IQA) has gained much attention due to its wide application prospect. Among them, the Reduced-Reference (RR) methods estimate perceptual quality of distorted images with partial information from the reference images. This paper proposes a novel universal RR-IQA metric based on the statistics of edge patterns. Firstly, the binary edge maps of the reference and distorted images are created by the LOG operator and zero-crossing detection. Based on them, 15 groups of typical edge patterns are extracted and then their statistical distributions are calculated respectively for the reference and distortion images. The proposed RR-IQA metric is achieved by computing the L-1 Minkowski distance between those two distributions. We have evaluated this metric on six publicly accessible subjective IQA databases. Experiments shows that the proposed metric featured with typical edge patterns outperform other methods in terms of data volume, accuracy and consistency with human perception. In a way, our work provides a new view to the IQA metric design.

Chromatic Aberration of lenses is becoming increasingly visible with the rise of sensor resolution, and methods to algorithmically correct it are becoming increasingly common in commercial systems. A popular class of algorithms undo the geometric distortions after demosaicking. Since most demosaickers require high frequency correlation of primary colors to work effectively, the result is artifact-ridden as Chromatic Aberration destroys this correlation. The other existing approach of undistorting primary color images before demosaicking requires resampling of sub-sampled primary color images and is prone to aliasing. Furthermore, this algorithm cannot be applied to panchromatic CFAs. We propose a joint demosaicking and Chromatic Aberration correction algorithm that is applicable to both panchromatic and primary color CFAs and suffers from none of the above problems. Our algorithm treats the mosaicing process as a linear transform that is invertible if luminance and chrominance are appropriately bandlimited. We develop and incorporate Chromatic Aberration corrections to this model of the mosaicing process without altering its linearity or invertibility. This correction works for both space variant linear filter demosaicking and the more aggressive compressive sensing reconstruction.

Recently, we developed a method for optimally estimating focus error given a set of natural scenes, a waveoptics model of the lens system, a sensor array, and a specification of measurement noise. The method is based on first principles and can be tailored to any vision system for which these properties can be characterized. Here, the method is used to estimate defocus in local areas of images (64x64 pixels) formed in a Nikon D700 digital camera fitted with a 50mm Sigma prime lens. Performance is excellent. Defocus magnitude and sign can be estimated with high precision and accuracy over a wide range. The method takes an integrative approach that accounts for natural scene statistics and capitalizes (but not does depend exclusively) on chromatic aberrations. Although chromatic aberrations are greatly reduced in achromatic lenses, we show that there are sufficient residual chromatic aberrations in a high-quality prime lens for our method to achieve good performance. Our method has the advantages of both phase-detection and contrastmeasurement autofocus techniques, without their disadvantages. Like phase detection, the method provides point estimates of defocus (magnitude and sign), but unlike phase detection, it does not require specialized hardware. Like contrast measurement, the method is image-based and can operate in "Live View" mode, but unlike contrast measurement, it does not require an iterative search for best focus. The proposed approach could be used to develop improved autofocus algorithms for digital imaging and video systems.

A softcopy quality ruler study involving 12 scenes and 34 observers was performed to quantify the dependence of quality on color saturation, in the absence of noise, with saturation measured using Imatest software. It was found that quality falls off symmetrically with deviation of color saturation from the preferred value of about 110%, with a 20% change in saturation reducing quality by about two just noticeable differences (JNDs). Optimization of noise versus color saturation was investigated using (1) the aforementioned transform of color saturation to JNDs of quality; (2) a previously published objective metric and JND transform for isotropic noise; and (3) the multivariate formalism, for combining JNDs from independent attributes into an overall quality JNDs. As noise increases and signal to noise ratio (SNR) decreases, the optimal color saturation decreases from the 110% position, so that there is less noise amplification by the color correction matrix. A quality contour plot is presented, showing a region of plausible color saturation values, as a function of SNR, for a representative use case. One example of a reasonable strategy is to provide 80% color saturation at SNR = 5, 90% at SNR = 10, 100% at SNR = 20, and 110% at SNR 50 and above.

We propose a bio-inspired framework for automatic image quality enhancement. Restoration algorithms usually have fixed parameters whose values are not easily settable and depend on image content. In this study, we show that it is possible to correlate no-reference visual quality values to specific parameter settings such that the quality of an image could be effectively enhanced through the restoration algorithm. In this paper, we chose JPEG blockiness distortion as a case study. As for the restoration algorithm, we used either a bilateral filter, or a total variation denoising detexturer. The experimental results on the LIVE database will be reported. These results will demonstrate that a better visual quality is achieved through the optimized parameters over the entire range of compression, with respect to the algorithm default parameters.

In this paper, we describe a method to fuse multiple images taken with varying exposure times in the JPEG domain. The proposed algorithm finds its application in HDR image acquisition and image stabilization for hand-held devices like mobile phones, music players with cameras, digital cameras etc. Image acquisition at low light typically results in blurry and noisy images for hand-held camera's. Altering camera settings like ISO sensitivity, exposure times and aperture for low light image capture results in noise amplification, motion blur and reduction of depth-of-field respectively. The purpose of fusing multiple exposures is to combine the sharp details of the shorter exposure images with high signal-to-noise-ratio (SNR) of the longer exposure images. The algorithm requires only a single pass over all images, making it efficient. It comprises of - sigmoidal boosting of shorter exposed images, image fusion, artifact removal and saturation detection. Algorithm does not need more memory than a single JPEG macro block to be kept in memory making it feasible to be implemented as the part of a digital cameras hardware image processing engine. The Artifact removal step reuses the JPEGs built-in frequency analysis and hence benefits from the considerable optimization and design experience that is available for JPEG.

In this paper, the theoretical foundation and practical implementation of a controllable anti-aliasing filter for digital film cameras is presented. A prototype of an optical anti-aliasing filter that is based on moving a parallel optical window was designed and built to demonstrate the ability to control the spatial frequency response of an acquisition system. During the image exposure, four spring-preloaded voice coils rapidly change the pitch and yaw of the parallel window, resulting in a displacement of the image content that is projected onto the sensor. The image content displacement during the exposure results in altering the frequency response of the scene that is captured by the sensor. Specifically, during the exposure time, a carefully controlled movement of the parallel optical window results in a circular trajectory of the image content that is projected onto the sensor. By increasing or decreasing the radius of the circular trajectory, the spatial cut-off frequency of the system is dynamically modified. In addition to the circular path, this paper shows theoretical justification and demonstrates the use of more complex trajectories, such as double circular, elliptical, and one-dimensional rectangular trajectories. These trajectories improve the suppression of aliased components in the acquired image.

Digital cameras and camera phones are now very widely used but there are some issues that affect their use and the quality of the images captured. Many of these issues are due to problem of interaction or feedback from the camera. Modern smartphones have a wide range of sensors, rich feedback mechanisms and lots of processing power. We have developed and evaluated a range of new interaction techniques for cameras and camera phones that improve the picture taking process and allow people to take better pictures first time.

Computer Vision enables mobile devices to extract the meaning of the observed scene from the information acquired with the onboard sensor cameras. Nowadays, there is a growing interest in Computer Vision algorithms able to work on mobile platform (e.g., phone camera, point-and-shot-camera, etc.). Indeed, bringing Computer Vision capabilities on mobile devices open new opportunities in different application contexts. The implementation of vision algorithms on mobile devices is still a challenging task since these devices have poor image sensors and optics as well as limited processing power. In this paper we have considered different algorithms covering classic Computer Vision tasks: keypoint extraction, face detection, image segmentation. Several tests have been done to compare the performances of the involved mobile platforms: Nokia N900, LG Optimus One, Samsung Galaxy SII.

Hyperspectral sensors generate useful information about climate and the earth surface in numerous contiguous narrow spectral bands, and are widely used in resource management, agriculture, environmental monitoring, etc. Compression of the hyperspectral data helps in long-term storage and transmission systems. Lossless compression is preferred for high-detail data, such as hyperspectral data. Due to high redundancy in neighboring spectral bands and the tendency to achieve a higher compression ratio, using adaptive coding methods for hyperspectral data seems suitable for this purpose. This paper introduces two new compression methods. One of these methods is adaptive and powerful for the compression of hyperspectral data, which is based on separating the bands with different specifications by the histogram and Binary Particle Swarm Optimization (BPSO) and compressing each one a different manner. The new proposed methods improve the compression ratio of the JPEG standards and save storage space the transmission. The proposed methods are applied on different test cases, and the results are evaluated and compared with some other compression methods, such as lossless JPEG and JPEG2000.

Digital cameras are used under a wide spectrum of illuminants and should adjust the white point according to the illuminant used (white balance correction). The white balance correction makes "white" object be reproduced as "white" but the reproduction of chromatic objects will not necessarily be appropriate (color constancy error). Three types of sensor models were used for the simulation. The error was reduced by a primary conversion that made overlaps and widths of the channels smaller. Thus two new metrics that evaluated the overlaps and the widths were defined and used to optimize the conversion to a color space suitable for white-balancing and it was shown that the color constancy errors were reduced. It was also shown that the color constancy error was small for the sensor model whose overlaps between channels were small and channel widths were narrow. The narrower widths and smaller overlaps of RGB channels gave less accurate colorimetric reproduction but less noisy images. In view of consumer digital cameras, the narrower widths and smaller overlaps of RGB channels is suitable because it can give less noisy images and consistent color reproduction with simple white balance processing.

Multispectral imaging is highly demanded for precise color reproduction and for various computer vision applications. Multispectral imaging with a multispectral color filter array (MCFA), which can be considered as a multispectral extension of commonly used consumer RGB cameras, could be a simple, low-cost, and practical system. A challenge of the multispectral imaging with the MCFA is multispectral demosaicking because each spectral component of the MCFA is severely undersampled. In this paper, we propose a novel multispectral demosaicking algorithm using a guided filter. The guided filter is recently proposed as an excellent structurepreserving filter. The guided filter requires so-called a guide image. A main issue of the guided filter is how to obtain an effective guide image. In our proposed algorithm, we generate the guide image from the most densely sampled spectral component in the MCFA. Then, ohter spectral components are interpolated by the guided filter. Experimental results demonstrate that our proposed algorithm outperforms other existing demosaicking algorithms both visually and quantitatively.

The availability of multispectral scene data makes it possible to simulate a complete imaging pipeline for digital cameras, beginning with a physically accurate radiometric description of the original scene followed by optical transformations to irradiance signals, models for sensor transduction, and image processing for display. Certain scenes with animate subjects, e.g., humans, pets, etc., are of particular interest to consumer camera manufacturers because of their ubiquity in common images, and the importance of maintaining colorimetric fidelity for skin. Typical multispectral acquisition methods rely on techniques that use multiple acquisitions of a scene with a number of different optical filters or illuminants. Such schemes require long acquisition times and are best suited for static scenes. In scenes where animate objects are present, movement leads to problems with registration and methods with shorter acquisition times are needed. To address the need for shorter image acquisition times, we developed a multispectral imaging system that captures multiple acquisitions during a rapid sequence of differently colored LED lights. In this paper, we describe the design of the LED-based lighting system and report results of our experiments capturing scenes with human subjects.

This document considers the planar motions of camera, that is, the rotation, and the horizontal and vertical translations in the image plane. The approach based on projection including both Cartesian coordinate system and polar coordinate system can estimate the three parameters comparably quickly with simple calcuation. The potential applications cover motion deblurring, noise reduction, super-resolution, image fusion, high dyanmic range image processing, EDOF, 3D imaging or those techniques which require global or local registration.

Aberrations occur in multispectral cameras featuring filter wheels because of color filters with different optical properties being present in the ray path. In order to ensure an exact compensation of these aberrations, a mathematical model of the distortions has to be developed and its parameters have to be calculated using the measured data. Such a model already exists for optical filters placed between the sensor and the lens, but not for bandpass filters placed in front of the lens. For this configuration, the rays are first distorted by the filters and then by the lens. In this paper, we derive a model for aberrations caused by filters placed in front of the lens in multispectral cameras. We compare this model with distortions obtained with simulations as well as with distortions measured during real multispectral acquisitions. In both cases, the difference between modeled and measured aberrations remains low, which corroborates the physical model. Multispectral acquisitions with filters placed between the sensor and the lens or in front of the lens are compared: the latter exhibit smaller distortions and the aberrations in both images can be compensated using the same algorithm.

This paper proposes a novel method to correct saturated pixels in images. This method is based on the YCbCr color space and separately corrects the chrominance and the luminance of saturated pixels. Dynamic thresholds are adopted to identify saturated pixels, i.e. the thresholds for different images and different color channels are different. So our method can correct not only RAW images but also processed images. Once the saturated pixels are identified, there are three kinds of saturated pixels: 1-channel saturated pixels, 2-channel saturated pixels and 3-channel saturated pixels. They are denoted as Ω₁, Ω₂ and Ω₃ respectively. Different strategies are implemented to these three kinds of regions. The color of saturated pixels in Ω₁ is corrected according to their original color and the color of their neighborhood. And the color of saturated pixels in Ω₂ and Ω₃ is corrected only according to the color of their neighborhood. The luminance of saturated pixels is corrected using the model proposed in this paper. Experiment results show our method is effective in correcting saturated pixels of RAW images and process images.

This paper presents a real-time multi-directional 2D fast wavelet transform. As a real-time implementation of wavelet, small pixel windows in raster scan order are sequentially used as inputs to the transform instead of full image frame. This approach is corresponding to a recent image sensor releasing image pixels in line-by-line scanning order. The proposed method has high locality property so as to achieve low-latency real-time implementation adequate for recent mobile highly cost-effective application and it uses multi-directional decomposition which enables the transform to reduce the directional artifacts often occurring in conventional two-directional separable decomposition wavelet. As a possible application of the proposed wavelet, denoised sharpening algorithm was devised and the result is presented, representing the enhancement of directional and blocking artifacts problem.

In this work we present an automatic local color transfer method based on semantic image annotation. With this annotation, images are segmented into homogeneous regions, assigned to seven different classes (vegetation, snow, water, ground, street, and sand). Our method permits to automatically transfer the color distribution from regions of the source and target images annotated with the same class (for example the class "sky"). The amount of color transfer can be controlled by tuning a single parameter. Experimental results will show that our local color transfer is usually more visually pleasant than a global approach.