With the emergence of high-dynamic range (HDR) imaging, the existing visual signal processing systems will need to deal with both HDR and standard dynamic range (SDR) signals. In such systems, computing the objective quality is an important aspect in various optimization processes (e.g., video encoding). To that end, we present a newly calibrated objective method that can tackle both HDR and SDR signals. As it is based on the previously proposed HDR-VDP-2 method, we refer to the newly calibrated metric as HDR-VDP-2.2. Our main contribution is toward improving the frequency-based pooling in HDR-VDP-2 to enhance its objective quality prediction accuracy. We achieve this by formulating and solving a constrained optimization problem and thereby finding the optimal pooling weights. We also carried out extensive cross-validation as well as verified the performance of the new method on independent databases. These indicate clear improvement in prediction accuracy as compared with the default pooling weights. The source codes for HDR-VDP-2.2 are publicly available online for free download and use.

A three-dimensional (3-D) measurement method for large-scale bending plates is presented. The proposed method, which combines the advantages of laser and a vision measurement method, makes use of a 3-D scanner, a texture projector, and a laser total station. The 3-D scanner is used to measure multiple partial sections of a large-scale bending plate, the texture projector is used to project a textured pattern onto the bending plate to perform alignment, and the total station is used to correct the aligned result to obtain more accurate 3-D data. The performance and effectiveness are evaluated by experiments.

Various techniques have previously been proposed for thresholding of images to separate objects from the background. Although these thresholding techniques have been proven effective on particular types of images, none of them is able to produce consistently good results on a wide range of existing images. The nonextensive cross-entropy (also known as Tsallis cross-entropy) is introduced to determine the optimal threshold value. The new thresholding scheme aims to minimize the Tsallis cross-entropy between the original image and the thresholded image. The effectiveness of the proposed scheme is demonstrated by using examples from the synthetic images, natural scene images, and an image dataset that includes nondestructive testing images and document images, on the basis of comparison with the traditional cross-entropy, Otsu’s, minimum error thresholding, and two state-of-the-art methods. Furthermore, a tunable parameter q of Tsallis cross-entropy in the presented scheme gives the proposed methods the potential to handle the different image segmentation tasks.

Digital images may be impaired by periodic or quasiperiodic noise, which manifests itself by spurious long-range repetitive patterns. Most of the time, quasiperiodic noise is well localized in the Fourier domain; thus it can be attenuated by smoothing out the image spectrum with a well-designed notch filter. While existing algorithms require hand-tuned filter design or parameter setting, this paper presents an automated approach based on the expected power spectrum of a natural image. The resulting algorithm enables not only the elimination of simple periodic noise whose influence on the image spectrum is limited to a few Fourier coefficients, but also of quasiperiodic structured noise with a much more complex contribution to the spectrum. Various examples illustrate the efficiency of the proposed algorithm. A comparison with morphological component analysis, a blind source separation algorithm, is also provided. A MATLAB® implementation is available.

Invariant harmonic transforms based on the fractional Fourier transform are proposed in this paper. The so-called fractional polar harmonic transforms (FrPHTs) with the order parameter α are first defined, which are generalizations of the PHTs. Second, a watermarking scheme is presented and discussed in detail associated with the newly defined FrPHTs. Finally, the simulations are clearly performed to verify the well capabilities of the transforms on image watermarking, which show that the proposed transforms with suitable parameters outperform the traditional PHTs. In addition, the experimental results also demonstrate that the order parameter α has an effect on the performance of FrPHTs in the image watermarking robustness and can improve the watermarking safety.

Recently, facial expression recognition has attracted much attention in machine vision research because of its various applications. Accordingly, many facial expression recognition systems have been proposed. However, the majority of existing systems suffer from a critical problem: geometric variability. It directly affects the performance of geometric feature-based facial expression recognition approaches. Furthermore, it is a crucial challenge in appearance feature-based techniques. This variability appears in both neutral faces and facial expressions. Appropriate face geometry normalization can improve the accuracy of each facial expression recognition system. Therefore, this paper proposes different geometric models or shapes for normalization. Face geometry normalization removes geometric variability of facial images and consequently, appearance feature extraction methods can be accurately utilized to represent facial images. Thus, some expression-based geometric models are proposed for facial image normalization. Next, local binary patterns and local phase quantization are used for appearance feature extraction. A combination of an effective geometric normalization with accurate appearance representations results in more than a 4% accuracy improvement compared to several state-of-the-arts in facial expression recognition. Moreover, utilizing the model of facial expressions which have larger mouth and eye region sizes gives higher accuracy due to the importance of these regions in facial expression.

We present a visual tracking method with feature fusion via joint sparse presentation. The proposed method describes each target candidate by combining different features and joint sparse representation for robustness in coefficient estimation. Then, we build a probabilistic observation model based on the approximation error between the recovered candidate image and the observed sample. Finally, this observation model is integrated with a stochastic affine motion model to form a particle filter framework for visual tracking. Furthermore, a dynamic and robust template update strategy is applied to adapt the appearance variations of the target and reduce the possibility of drifting. Quantitative evaluations on challenging benchmark video sequences demonstrate that the proposed method is effective and can perform favorably compared to several state-of-the-art methods.

The segmentation of images is a critical step in many computer vision applications. Additionally, some applications require the achievement of acceptable segmentation quality while the algorithm is executed in real time. In this study, we present a split-and-merge segmentation methodology that uses integral images to improve the execution time. We call our methodology integral split and merge (ISM) segmentation. The integral images are used here to calculate statistics of the image regions in constant time. Those statistics are used to guide the splitting process by identifying the homogeneous regions in the image. We also propose a merge criterion that performs connected component analysis of the homogeneous regions. Moreover, the merging procedure is able to group regions of the image showing gradients. Furthermore, the number of regions resulting from the segmentation process is determined automatically. In a series of tests, we compare ISM against other state-of-the-art algorithms. The results from the tests show that our ISM methodology obtains image segmentations with a comparable quality, using a simple texture descriptor instead of a combination of color-texture descriptors. The proposed ISM methodology also has a piecewise linear computational complexity, resulting in an algorithm fast enough to be executed in real time.

Stego images embedded by unknown steganographic algorithms currently may not be detected by using steganalysis detectors based on binary classifier. However, it is difficult to obtain high detection accuracy by using universal steganalysis based on one-class classifier. For solving this problem, a blind detection method for JPEG steganography was proposed from the perspective of information theory. The proposed method combined the semisupervised learning and soft margin support vector machine with steganalysis detector based on one-class classifier to utilize the information in test data for improving detection performance. Reliable blind detection for JPEG steganography was realized only using cover images for training. The experimental results show that the proposed method can contribute to improving the detection accuracy of steganalysis detector based on one-class classifier and has good robustness under different source mismatch conditions.

Persistently tracking multiple objects in cluttered environments is very challenging. We present a tracking association approach based on the shortest path faster algorithm. We first formulate the multiple object tracking as an integer programming problem of the flow network. Under this framework, the integer assumption is relaxed to a standard linear programming problem. Therefore, the global optimal solution can quickly be obtained using the fast dynamic shortest path algorithm, which highlights the dynamic programming characteristic of the shortest path, thus faster, algorithm. The proposed method avoids the difficulties of integer programming; more importantly, it has a lower worst-case complexity than competing methods but a better tracking accuracy and robustness in complex environments. Simulation results show that our proposed algorithm takes less time than other methods and can operate in real time.

Tone mapping methods aim to display a high-dynamic-range image on a common 8-bit liquid crystal display by compressing the dynamic range. We propose a simple but effective tone mapping operator (TMO) by using localized gamma correction. As the TMO involves the contrast and the color saturation components, our idea is to make use of gamma correction to adjust the values of these two components locally. The local weights can be determined according to the pixel values within local regions. Such an adaption can be handled more effectively for the contrast and color saturation in dark and bright regions of an image. Numerical examples are given to demonstrate the merit of the localized gamma correction scheme. The experiments reported in this paper illustrate that the proposed method can provide visually pleasing results, and it is also competitive with the other testing methods.

When estimating a sharp image from a blurred one, blur kernel noise often leads to inaccurate recovery. We develop an effective method to estimate a blur kernel which is able to remove kernel noise and prevent the production of an overly sparse kernel. Our method is based on an iterative framework which alternatingly recovers the sharp image and estimates the blur kernel. In the image recovery step, we utilize the total variation (TV) regularization to recover latent images. In solving TV regularization, we propose a new criterion which adaptively terminates the iterations before convergence. While improving the efficiency, the quality of the final results is not degraded. In the kernel estimation step, we develop a metric to measure the usefulness of image edges, by which we can reduce the ambiguity of kernel estimation caused by small-scale edges. We also propose a hybrid ℓ_p norm, which is composed of ℓ₂ norm and ℓ_p norm with 0.7≤p<1, to construct a sparsity constraint. Using the hybrid ℓ_p norm, we reduce a wider range of kernel noise and recover a more accurate blur kernel. The experiments show that the proposed method achieves promising results on both synthetic and real images.

This paper proposes a method for constructing permutations on m position arrangements. Our objective is to encrypt color images using advanced encryption standard (AES), using variable permutations means a different one for each 128-bit block in the first round after the x-or operation is applied. Furthermore, this research offers the possibility of knowing the original image when the encrypted figure suffered a failure from either an attack or not. This is achieved by permuting the original image pixel positions before being encrypted with AES variable permutations, which means building a pseudorandom permutation of 250,000 position arrays or more. To this end, an algorithm that defines a bijective function between the nonnegative integer and permutation sets is built. From this algorithm, the way to build permutations on the 0,1,…,m−1 array, knowing m−1 constants, is presented. The transcendental numbers are used to select these m−1 constants in a pseudorandom way. The quality of the proposed encryption according to the following criteria is evaluated: the correlation coefficient, the entropy, and the discrete Fourier transform. A goodness-of-fit test for each basic color image is proposed to measure the bits randomness degree of the encrypted figure. On the other hand, cipher images are obtained in a loss-less encryption way, i.e., no JPEG file formats are used.

Multimodal image registration has received significant research attention over the past decade, and the majority of the techniques are global in nature. Although local techniques are widely used for general image registration, there are only limited studies on them for multimodal image registration. Scale invariant feature transform (SIFT) is a well-known general image registration technique. However, SIFT descriptors are not invariant to multimodality. We propose a SIFT-based technique that is modality invariant and still retains the strengths of local techniques. Moreover, our proposed histogram weighting strategies also improve the accuracy of descriptor matching, which is an important image registration step. As a result, our proposed strategies can not only improve the multimodal registration accuracy but also have the potential to improve the performance of all SIFT-based applications, e.g., general image registration and object recognition.

Region of interest (ROI) based image compression can offer a high image-compression ratio along with high quality in the important regions of the image. For many applications, stable compression quality is required for both the ROIs and the images. However, image compression does not consider information specific to the application and cannot meet this requirement well. This paper proposes an application-oriented ROI-based image-compression method using bit-allocation optimization. Unlike typical methods that define bit-rate parameters empirically, the proposed method adjusts the bit-rate parameters adaptively to both images and ROIs. First, an application-dependent optimization model is constructed. The relationship between the compression parameters and the image content is learned from a training image set. Image redundancy is used to measure the compression capability of image content. Then, during compression, the global bit rate and the ROI bit rate are adjusted in the images and ROIs, respectively—supported by the application-dependent information in the optimization model. As a result, stable compression quality is assured in the applications. Experiments with two different applications showed that quality deviation in the reconstructed images decreased, verifying the effectiveness of the proposed method.

This work proposes a detection-based tracking algorithm able to locate and keep the identity of multiple people, who may be occluded, in uncontrolled stationary environments. Our algorithm builds a tracking graph that models spatio-temporal relationships among attributes of interacting people to predict and resolve partial and total occlusions. When a total occlusion occurs, the algorithm generates various hypotheses about the location of the occluded person considering three cases: (a) the person keeps the same direction and speed, (b) the person follows the direction and speed of the occluder, and (c) the person remains motionless during occlusion. By analyzing the graph, our algorithm can detect trajectories produced by false alarms and estimate the location of missing or occluded people. Our algorithm performs acceptably under complex conditions, such as partial visibility of individuals getting inside or outside the scene, continuous interactions and occlusions among people, wrong or missing information on the detection of persons, as well as variation of the person’s appearance due to illumination changes and background-clutter distracters. Our algorithm was evaluated on test sequences in the field of intelligent surveillance achieving an overall precision of 93%. Results show that our tracking algorithm outperforms even trajectory-based state-of-the-art algorithms.

We present an image forensics method to detect least significant bit replacement steganography attack. The proposed method provides fine-grained forensics features by using the hierarchical structure that combines pixels correlation and bit-planes correlation. This is achieved via bit-plane decomposition and difference matrices between the least significant bit-plane and each one of the others. Generated forensics features provide the susceptibility (changeability) that will be drastically altered when the cover image is embedded with data to form a stego image. We developed a statistical model based on the forensics features and used least square support vector machine as a classifier to distinguish stego images from cover images. Experimental results show that the proposed method provides the following advantages. (1) The detection rate is noticeably higher than that of some existing methods. (2) It has the expected stability. (3) It is robust for content-preserving manipulations, such as JPEG compression, adding noise, filtering, etc. (4) The proposed method provides satisfactory generalization capability.

Compressed sensing has the potential to address the challenge of simultaneously requiring high temporal and spatial resolution in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI), by randomly undersampling the k-space with a predesigned trajectory. However, the traditional variable density (VD) design scheme includes inherent randomness since many probability density functions (PDFs) correspond to a given acceleration factor and one fixed PDF can generate different trajectories. This randomness may translate to an uncertainty in kinetic parameter estimation. We first evaluate how the one-to-many mapping in trajectory design influences DCE parameter estimation when high reduction factors are used. Then we propose a robust design scheme by adaptively segmenting k-space into low- and high-frequency domains considering the specific characteristics for different subjects and only applying the VD scheme in the high-frequency domain. Simulation results demonstrate high accuracy and robustness compared to the VD design.

This work proposes a weighted joint sparse representation (WJSR)-based classification method for robust alignment-free face recognition, in which an image is represented by a set of scale-invariant feature transform descriptors. The proposed method considers the correlation and the reliability of the query descriptors. The reliability is measured by the similarity information between the query descriptors and the atoms in the dictionary, which is incorporated into the l₀∖l₂-norm minimization to seek the optimal WJSR. Compared with the related state-of-art methods, the performance is advanced, as verified by the experiments on the benchmark face databases.

The study of the light emitted by transistors in a highly scaled complementary metal oxide semiconductor (CMOS) integrated circuit (IC) has become a key method with which to analyze faulty devices, track the failure root cause, and have candidate locations for where to start the physical analysis. The localization of defective areas in IC corresponds to a reliability check and gives information to the designer to improve the IC design. The scaling of CMOS leads to an increase in the number of active nodes inside the acquisition area. There are also more differences between the spot’s intensities. In order to improve the identification of all of the photon emission spots, we introduce an unsupervised processing scheme. It is based on iterative thresholding decomposition (ITD) and mathematical morphology operations. It unveils all of the emission spots and removes most of the noise from the database thanks to a succession of image processing. The ITD approach based on five thresholding methods is tested on 15 photon emission databases (10 real cases and 5 simulated cases). The photon emission areas’ localization is compared to an expert identification and the estimation quality is quantified using the object consistency error.

Images captured in foggy weather conditions often fade the colors and reduce the contrast of the observed objects. An efficient image fusion method is proposed to remove haze from a single input image. First, the initial medium transmission is estimated based on the dark channel prior. Second, the method adopts an assumption that the degradation level affected by haze of each region is the same, which is similar to the Retinex theory, and uses a simple Gaussian filter to get the coarse medium transmission. Then, pixel-level fusion is achieved between the initial medium transmission and coarse medium transmission. The proposed method can recover a high-quality haze-free image based on the physical model, and the complexity of the proposed method is only a linear function of the number of input image pixels. Experimental results demonstrate that the proposed method can allow a very fast implementation and achieve better restoration for visibility and color fidelity compared to some state-of-the-art methods.

We investigate the ability of fractional-order differentiation (FD) for facial texture representation and present a local descriptor, called the principal patterns of fractional-order differential gradients (PPFDGs), for face recognition. In PPFDG, multiple FD gradient patterns of a face image are obtained utilizing multiorientation FD masks. As a result, each pixel of the face image can be represented as a high-dimensional gradient vector. Then, by employing principal component analysis to the gradient vectors over the centered neighborhood of each pixel, we capture the principal gradient patterns and meanwhile compute the corresponding orientation patterns from which oriented gradient magnitudes are computed. Histogram features are finally extracted from these oriented gradient magnitude patterns as the face representation using local binary patterns. Experimental results on face recognition technology, A.M. Martinez and R. Benavente, Extended Yale B, and labeled faces in the wild face datasets validate the effectiveness of the proposed method.

Multiexposure fusion images have a higher dynamic range and reveal more details than a single captured image of a real-world scene. A clear and intuitive feature-based fusion technique for multiexposure image sequences is conceptually proposed. The main idea of the proposed method is to combine three image features [phase congruency (PC), local contrast, and color saturation] to obtain weight maps of the images. Then, the weight maps are further refined using a guided filter which can improve their accuracy. The final fusion result is constructed using the weighted sum of the source image sequence. In addition, for multiexposure image-sequence fusion involving dynamic scenes containing moving objects, ghost artifacts can easily occur if fusion is directly performed. Therefore, an image-alignment method is first used to adjust the input images to correspond to a reference image, after which fusion is performed. Experimental results demonstrate that the proposed method has a superior performance compared to the existing methods.

This paper presents a physical perspective to understand visual saliency on the scene and develops an anisotropic heat diffusion based formulation, named visual attention diffusion (VAD), to detect salient regions in natural images. Our idea is based on the assumption that the visual attention can be diffused from a part of the most representative salient elements to other salient regions for a given image. In particular, we first design a corner-points-driven technique to simultaneously identify candidate foreground and background regions on the image. Then both the local object location and the nonlocal background structure can be estimated accordingly. Finally, we design a geometry-driven anisotropic Poisson system with Dirichlet boundary for saliency detection which uses the representative salient elements in the candidate foreground as heat sources and both local and nonlocal priors as guidance. After heat diffusion reaches a stable state, salient regions can be easily detected based on the temperatures (i.e., score value) of the image elements. Experiments show that the proposed system can produce more precise and reliable results compared to state-of-the-art saliency detectors.

Two-layer schemes provide an effective method of encoding high dynamic range images, while maintaining backward compatibility with the legacy JPEG encoding format. The first layer is the tone-mapped low dynamic range version of the original image, whereas the residual information that cannot be preserved in the first layer is stored in the second layer. Additionally, an inverse tone-mapping function can be used which can reduce the information content in the second layer if designed and used carefully. In this research, we investigate the effect of using the inverse functions on the coding efficiency. We show that just by using a smooth function and hence ensuring that extra energy is not induced in the residual data as the result of inverse tone mapping, significant improvement can be expected in the coding efficiency, i.e., more information can be stored at the same bitrates.