Image restoration is the process of restoring an image from a degraded version, which is usually blurred and noisy. We are motivated by the problem of restoring blurred and noisy images using multiscale weighted Schatten p-norm minimization, which not only gives a better approximation to the original low-rank hypothesis but also considers the importance of different rank components. Similar patches are vectorized and grouped to construct a noisy low-rank matrix. Weighted Schatten p-norm minimization values of all image patch groups are simultaneously penalized by a new regularization term, which can represent both the sparsity and self-similarity of the image structure accurately. In addition, by calculating the similarity of patches on different scales of the image, the restoration effect of the image is further improved. The experimental results show that our method is superior to some existing excellent algorithms in both numerical and visual effects.

Emotion is strongly subjective, and different parts of the image may have a different degree of impact on emotion. The key for solving image emotion recognition is to effectively mine different discriminative local regions. We present a deep architecture to guide the network to extract discriminative and diverse affective semantic information. First, training a full convolutional network with a cross-channel max pooling strategy (CCMP) to extract discriminative feature maps. Second, to ensure that most of the discriminative sentiment regions are located accurately, we add a module consisting of the convolution layer and the CCMP. After obtaining the discriminative regions of the first module, the feature elements corresponding to the discriminant regions are erased, and then the erased features are fed into the second module. Such adversarial erasure operation can force the network to discover different sentiment discriminative regions. Third, an adaptive feature fusion mechanism is proposed to better integrate discriminative and diverse sentiment representations. Sufficient experiments are conducted on the benchmark datasets FI, EmotionROI, Instagram, and Twitter1 to achieve 72.17%, 61.13%, 81.97%, and 85.44% recognition accuracies, respectively. The results of the experiment demonstrate that the proposed network outperforms the state-of-the-art results.

The recent advancement in deep learning-based approaches vastly outperforms the traditional image descriptors. Deep learning models, such as residual networks (ResNet), are well known for finding salient features. Although effective, high-level description often has a high dimensionality that increases computational overhead. The autoencoders find the useful approximation of the input data without losing critical information. Considering this, we propose a content-based image retrieval system for natural color images using a deep stacked sparse autoencoder (DSSA). The DSSA model learns latent features in an unsupervised way from the high-level description obtained using ResNet. The DSSA model achieves a nearly 50% reduction in size compared with the full-length features for the simple distance-based retrieval approach while increasing accuracy. The image retrieval efficacy of the learned latent features is also evaluated for two classifier-based methods using a Softmax classifier. Further, this study investigates the impact of unsupervised feature learning on retrieval using three benchmark natural color image databases of varying complexities, viz., Corel-1K, Corel-10K, and Canadian Institute for Advanced Research (CIFAR)-10. The latent features learned by the DSSA model with the fuzzy class membership-based retrieval method achieve promising improvements and yield a highly competitive retrieval performance with the large-size CIFAR-10 database.

Loop closure detection (LCD) is an indispensable part of simultaneous localization and mapping (SLAM) systems, making it significant for eliminating accumulative trajectory error and constructing a consistent map. Focusing on visual SLAM, we propose a LCD method using local-global similarity measurement strategies. As with most deep learning-based methods, we treat the LCD problem as a classification problem. We use a pretrained VGG-16 network to obtain advanced features and a multiscale feature extraction module to integrate the features of different receptive fields. Finally, we overcome the problems of dynamic object occlusion and large-area repetitive texture using a local-global similarity measurement module to measure the similarity between current and historical frames in all dimensions. This approach decouples feature extraction from classification determination, which enables us to use the advantages of both supervised and unsupervised learning to cope with the imbalance in the number of nonloop and loop pairs. Experimental results on three different datasets showed that the proposed method is significantly better than the state-of-the-art appearance-based LCD methods in dynamic urban, suburban, and highway scenarios.

Deep-learning-based approaches to depth estimation are rapidly advancing, offering superior performance over existing methods. To estimate the depth in real-world scenarios, depth estimation models require the robustness of various noise environments. We propose a pyramid frequency network (PFN) with spatial attention residual refinement module (SARRM) to deal with the weak robustness of existing deep-learning methods. To reconstruct depth maps with accurate details, the SARRM constructs a residual fusion method with an attention mechanism to refine the blur depth. The frequency division strategy is designed, and the frequency pyramid network is developed to extract features from multiple frequency bands. With the frequency strategy, PFN achieves better visual accuracy than state-of-the-art methods in both indoor and outdoor scenes on Make3D, KITTI depth, and NYUv2 datasets. Additional experiments on the noisy NYUv2 dataset demonstrate that PFN is more reliable than existing deep-learning methods in high-noise scenes.

Heart rate (HR) measurement from facial videos makes physiologic detection more convenient, which achieves much attention of many researchers. However, the existing methods are not very robust against motion disturbance caused by nonrigid movements of the face (e.g., smiling, yawning, and talking). This is due to the illumination variation on the local skin region, which results in the fluctuation of color intensity values in the region of interests. Different from previous approaches using fixed regions, we propose a self-adaptive region selection algorithm and then learn a 1D convolution neural network (CNN) by fusing these selected regions for HR estimation. Concretely, the observation blood volume pulse (BVP) matrix with sub-BVP is constructed from different separated regions and denoised by the rank minimization. To explore the sub-BVP of different regions, a clustering-based method is designed to generate the regional weights. In addition, a self-adaptive region selection is constructed to dynamically filter some useful face regions for robust HR estimation. Eventually, the regions and weights are separately embedded into a 1D CNN by a multiregion-fusion module. Experiments on the MAHNOB-HCI database show that the proposed approach is effective and reaches the best performance over five metrics.

We address the problem of video stabilization via image registration. We propose a single convolutional neural network, which is simultaneously a dense feature descriptor and a keypoint detector, to find reliable keypoints and their features under each frames. To obtain accurate keypoint localization, the authors leverage the inherent feature hierarchy to restore spatial resolution and low-level details. Compared with traditional detectors, the obtained keypoints are more stable for the following image registration. Based on the correspondences collected from registration results, we propose a technique for video stabilization which is a spatial smooth sparse motion field with motion vectors only at the mesh vertexes. In practice, the authors assign each vertex an unique motion vector via their neighboring correspondences and a median filter. The video stabilization is performed on the vertex profiles, which are motion vectors collected at the same vertex location over time. The quantitative and qualitative evaluations show that the proposed registration-based video stabilization method can produce comparable results with the state-of-the-art methods and achieve more stable performance on challenging situations.

Due to the rapid development of deep learning, salient target detection models have achieved significant progress. However, these models ignore the center prior in the collected images, that is, there is a greater probability of being in the center area for the salient object. The hidden information in the image is not fully utilized, which hinders further improvement of the detection result. We propose a center pooling algorithm that realizes the combination of image center prior and deep learning. The algorithm dynamically modifies the size of the receptive field according to the probability of the existence of salient objects and solves the problem of ignoring the importance of different regions when using the same pooling receptive field in the entire image by the standard pooling method. At the same time, the proportion of the foreground in the image is increased, and the problem of category imbalance is alleviated. In addition, a distance-weighted affinity loss function is also proposed to supplement the keyspace information by learning the grouping or separating force between pixel pairs at different distances and increase the confidence of edge regions. To further improve the detection results, a background detection subnetwork is introduced. Through the interactive learning of the foreground and background subnetworks, the opposite relationship between the predicted maps is explored. Finally, experiments on six different related datasets are conducted, and comparisons with other models prove that our model achieves the best performance.

We propose an action recognition method based on Riemannian manifold and adaptive weighted feature fusion. First, we divide the video into several sequence frames, and we propose to use the frame-action relevance to eliminate the slight disturbance of the background, so that the subsequent calculation can focus on the moving region of the target better. On this basis, our proposed framework extracts histogram of oriented gradients (HOG) and histogram of optical flow (HOF) features as original appearance and motion features. Aiming at the problem that the existing action recognition methods do not pay enough attention to the change rate of appearance and motion, an innovative feature names as Riemannian manifold distance feature (RMDF) is proposed, which can represent the topological relationship between features at different times in the same position and capture the change rate of current features. In the feature fusion stage, we use an unsupervised method of solving optimization problems to obtain the weights between different features. The strategy of adaptive weight allocation is adopted to fuse HOG, HOF, and their corresponding RMDFs to obtain the final representation of the video. Finally, it is sent to a back propagation neural network for classification and recognition. Experiments on three benchmark datasets show that our method still has great advantages compared with the most advanced methods.

We propose a saliency-enhanced two-stream convolutional network (SETNet) for no-reference image quality assessment. The proposed SETNet contains two subnetworks of image stream and saliency stream. The image stream focuses on the whole image content, while the saliency stream explicitly guides the network to learn spatial salient features that are more attractive to humans. In addition, the spatial attention module and dilated convolution-based channel attention module are employed to refine multiple levels features in spatial and channel dimensions. Finally, the image stream and saliency stream features fusion strategy is proposed to integrate features at the corresponding layer, and the final quality scores are predicted by using multiple levels of integrated features and weighting strategy. The experimental results of the proposed method and several representative methods on four synthetic distortion datasets and two real distortion datasets show that our SETNet has higher prediction accuracy and generalization ability.

An increasing amount of face images are being captured, shared, or applied in various applications. These images usually contain lots of sensitive information that may lead to privacy disclosure and misuse problems. Some pioneering works show that face image anonymization is one of the promising solutions. We present an innovative identity (ID) camouflage approach by synthesizing anonymous faces so that both artificial intelligence algorithms and humans are unable to recognize them and misuse them freely. Given a face image, our approach consists of two steps. First, we sample an anonymous ID feature point in the feature space. Then, we synthesize a camouflage face by training an anonymous deep generative adversarial network model. To reduce the risk of re-identification, we optimize our anonymous face generator based on the k-nearest neighbors to make a good balance between anonymity and utility of the original face image. The experimental results over the public dataset have verified the feasibility and state-of-the-art efficacy of our approach.

Converting three-dimensional (3D) point cloud data to two-dimensional (2D) image based on virtual structured-light system is a popular method in 3D data compression because the image format is easier to process by the exiting storing and transmitting methods. When this method is used in the 3D point cloud compression, the quantization error is introduced during the coordinate mapping. To solve this problem, a virtual structured-light 3D point cloud compression algorithm based on geometric reshaping is presented. And a least-square system parameter optimization method is proposed to further improve the data decoding accuracy. In the proposed method, the 3D spatial coordinates are reshaped to a 2D matrix first, and then the 2D matrix is stored as a “Holoimage” by using the parameter optimized virtual structured-light system. The quantization error introduced by coordinate mapping between X and Y coordinates of the 3D point cloud and the 2D image pixel coordinates is suppressed, so the decoding accuracy is improved. In addition, the geometric information of the 3D point cloud is hidden synchronously when the 3D point cloud is compressed, which is of great significance in the copyright protection of the point cloud data. Experiments verify the effectiveness of the proposed algorithm. The decoding root mean square error (RMSE) of the proposed method is decreased by 79.86% on average compared with the traditional one under the same compression ratio, and the compression ratio of the proposed method is 1.96 times bigger than the traditional one under the similar decoding accuracy.

With the development of productivity, people set higher demands on the quality of steel. In recent years, artificial intelligence, especially deep learning-based computer vision technology has attracted great attention, which can be used for detecting steel surface defects such as scratches, patches, and rust spots. However, due to the complexity of the strip steel surface, it is still a challenge to accurately and effectively detect defect regions by the existing defects detection methods. Therefore, we propose a unique saliency model, i.e., deeper feature integration network to highlight the defect regions on the strip steel surface. To be specific, after each encoder stage, we introduce the multiscale global feature extraction module to elevate the multiscale deep features from the encoder. Meanwhile, we deploy the deeper feature extraction module, which contains a bidirectional feature extraction unit, to dig the effective representation for defects. Particularly, the forward one is equipped with a channel-space weighted module and the backward one is equipped with a split attention module. After that, the features from the two branches are progressively integrated by the decoder, yielding the final high-quality saliency maps that give a good depiction of the strip steel surface defects. Extensive experiments are executed on the public dataset, and the comparison results show that our model performs better than the 15 state-of-the-art methods, which proves the effectiveness and superiority of the proposed model.

The existing image compression methods are mainly aimed at human perception tasks instead of machine vision tasks. Rich features learned by the shallow and deep layers of pre-trained visual geometry group (VGG)-net can serve human perception and machine vision tasks, respectively. To improve the machine analysis capabilities of the human-targeted compression method, we propose a scalable image compression framework at low bit-rates. Specifically, the scalable compression framework is composed of a base layer (BL) and an enhancement layer (EL), which utilize the correlation between the above features to perform machine analysis and human perception tasks, respectively. For effectively utilizing the above two types of features, a multi-branch shared module that utilizes the complementarity of multi-branch convolution kernels to retain compact information for the above features to support BL and EL tasks is proposed. In addition, for further improving the accuracy of machine analysis, the machine-vision importance map is introduced in BL; it adaptively utilizes the spatial and channel information from the deepest layer of VGG-net to guide local bit allocation. When the bit-rate is limited to 0.2 bpp, the average recognition accuracy (Top-1) of the BL of proposed method is 5.2%, 13.4%, 6.0%, and 7.4% higher than that of BPG, Webp, Mentzer, and NIC, respectively, on the ILSVRC2012 verification dataset. Meanwhile, the EL provides good visual experience.

Capsule network (CapsNet) is well known as an evolution of classical convolution neural network, which is good at recognizing postures, orientations, and textures, thus achieving promising results in areas such as image classification. However, the high computational complexity hinders them from building larger models. Fortunately, reducing the complexity of the routing of CapsNet through the channel attention mechanism has achieved significantly good results, whereas there is a possibility of further optimization of the storage and precision. Therefore, we propose an improved version of combination of multiscale routing and self-attention mechanism on the basis of the traditional channel attention routing. First, we absorb multiscale information into the routing process, thus enriching the representation capability of the capsule. Second, the global information of the capsule is captured by the self-attention routing, which leads to less misclassification errors. Finally, the proposed self-attention routing is further improved by a soft-threshold strategy, which resists the interference of background noises on complex datasets. Experimental results on MNIST, affNIST, and Cifar10 show that the proposed methods can obtain better performances with fewer training parameters compared with traditional methods on average.

Complicated underwater environments, such as occlusion by foreign objects and dim light, cause serious loss of underwater object features. This certainly increases the challenge of accurate detection of underwater objects. To tackle the above problems, we proposed an underwater occlusion object recognition algorithm combined with significant environmental features. First, the salient features of the underwater object were extracted by the salient feature extraction network. Second, salient environmental features associated with the object were extracted by the environmental feature attention mechanism. Finally, the contrast graph structure of the object’s salient features and related environmental features was constructed. The model proposed is compared with the existing object recognition model in the simulation of the three datasets: the underwater image enhancement benchmark dataset, video diver dataset, and underwater image dataset. The average recognition accuracy of the proposed model is 80.07%, which is improved by 2.38%. Experiments show that the object recognition algorithm proposed is effective and superior to existing algorithms.

We aim to address the near-duplicate image (NDI) detection problem with a deep learning network. With the advancement of digital acquisition devices and easy-to-use image editing software, NDI forgery is ubiquitous nowadays. This rising problem demands robust detection algorithms that can efficiently prevent the NDI forgery and its distribution. We present a modified deep learning model to detect NDI forensics attacks based on extracted wavelet Haar features. Unlike standard deep learning models, a wavelet decomposed preprocessing layer is used before the deep learning networks, and a support vector machine classifier is employed in the classification stage of a modified deep learning model. The model is tested on distinctly available image databases. The experimental results show that the proposed model more effectively detect the NDIs. In addition, the comparison result shows that the modified model outperforms the other approaches significantly.

For deep learning-based stereo matching, despite deep convolution networks possessing rich, high-level semantic features that are usually abstract and low-resolution, it is difficult to accurately calculate per-pixel correspondence without image detail information in challenging regions, such as thin objects, edge contours, reflective surfaces, etc. Motivated by the information propagation capability of the feature pyramid network (FPN), we focus on using FPN to improve the detailed comprehension of spatial pyramid pooling (SPP). We propose a feature SPP (FSPP), which consists of FPN and SPP. FSPP integrates high-frequency details reserved by FPN into multiscale features extracted by SPP to form a feature volume considering both detailed and contextual information. Using FSPP, features become distinct and alleviate the ambiguity of matching progress in challenging regions. Furthermore, we concatenate FSPP to Pyramid Stereo Matching Network with three-dimensional convolution and hourglass architecture. We also evaluate its effectiveness on the KITTI 2012/2015 and the Middlebury 2014/2021 datasets. The results show that the proposed algorithm performs better than current state-of-the-art algorithms in challenging regions while taking on better generalization.

The demand to design lightweight semantic segmentation models on mobile devices is growing. Current U-shape structures can improve the segmentation accuracy. However, they can hardly achieve lightweight requirements due to their inefficient encoders. Besides, partial details and edges are damaged during the process of repeated downsampling. To this end, we propose an efficient and lightweight U-shape network (ELUNet) for real-time semantic segmentation. In the encoder, a light split-shuffle convolution block is designed as the key component of feature extraction to achieve high-precision segmentation in the resource-limited scene. Furthermore, we propose a bridge channel attention module in the skip connection to selectively emphasize the valuable features. In the decoder, we propose an upsample feature fusion module to capture global contextual information, significantly improving the ability of the network to extract spatial information. Moreover, we design an edge refinement module to refine the segmentation predictions further. Extensive experiments prove the effectiveness of the ELUNet on Cityscapes and Camvid benchmarks. Specifically, the ELUNet contains only 2.0M parameters and achieves 73.3% mIoU on Cityscapes validation set with the speed of 52.6 FPS for a 512  ×  1024 input image on a single 1080Ti GPU.

Statistical information on underwater images related to subjective quality perception differs based on the environment. We initially propose whitish, yellowish, greenish, bluish topics for underwater images obtained by applying an integrated color distribution, that is, images under the same topic have the same hue. Inspired by the local multiscale, multidirectional statistical characteristics of the structure, contrast, and color of underwater images, we constructed visual attention factor sets for the above four typical underwater environment topics. Furthermore, a multitopic underwater image quality evaluation (MTUIQE) model is proposed to measure mix distortions in different underwater images. The method of measuring image quality based on image topic is proven to be more effective than directly fitting all the natural scene statistics features to the mean opinion scores. Experiments were conducted to show that the proposed MTUIQE approach is superior to existing objective underwater image quality evaluation methods. Compared with six commonly used blind image quality evaluation methods, the Pearson linear correlation coefficient and Spearman rank-order correlation coefficient are increased by more than 10%, which means that the quality assessment model adapted to specific degradation topics is more consistent with subjective underwater image quality perception, and it can effectively evaluate the performance of underwater image enhancement and restoration methods.

Millimeter-wave (MMW) imaging is a touch-free method for security inspection at railway stations and airports. However, automatic detection of dangerous objects in MMW images is challenging; the characteristics of images are remarkable with variable appearances of an object, different contrasts, and lower resolutions. We propose a fast and accurate concealed dangerous object detection method called MMW image Detection (MMWDet). MMWDet comprises two stages: single-image detection and multi-angle image detection. In the single-image detection stage, we design a more robust feature fusion relationship between adjacent feature maps in a feature pyramid network to achieve a more robust feature representation of an object. Meanwhile, we employ localization confidence rather than category confidence in the classification branch to address the negative sample interference in the training process. Then, multi-angle image detection is proposed to reduce the false and missed detection rate based on a single image by refining the detection results of multi-angle images, which maps and filters the prediction boxes frame by frame based on matching pairs of adjacent image feature points. We collected a MMW dataset that included 38k images. Extensive experiments on the MMW dataset demonstrate the effectiveness and improved performance of the proposed method compared with advanced general detection methods.

The crux of image deraining stems from the challenge of recognizing the diverse rain patterns within the rainy image. Most methods for image deraining remain visible rain residuals in the restored image, which suffers from insufficient modeling of rain streaks. In this work, we propose contrastive learning-based generative network (CLGNet), which follows a coarse-to-fine framework. In the coarse phase, our CLGNet employs the hierarchical encoder–decoder structure to remove obvious rain patterns, and first generates the coarse background image. Then, we introduce a well-designed multiscale feature aggregation module in the refining phase to extract and integrate global information dependencies from different scales. In additon, to facilitate the intra-stage and cross-stage information interaction, we propose the intra-stage feature fusion module and the cross-stage feature fusion module to encode broad contextual information. More importantly, we propose an innovative contrastive learning strategy and apply it to each stage of our proposed CLGNet to enhance the decoupling ability of the encoder and help the model recognize complex rain patterns. Extensive experiments on five benchmark datasets demonstrate the superiority of our proposed CLGNet than other state-of-the-art methods for single image deraining on both the visual quality and quantitative evaluation.

The object detectors based on deep learning, such as Yolov3, Yolov3-tiny, and Yolov4-tiny, have achieved real-time object detection in various vision tasks. However, long-tailed distribution of classes, small objects, and incomplete numbers on the dataset have brought challenges to the automatic recognition of wheel-type water meter digit reading using these object detectors. We propose a Mix suppression data augmentation method based on information deletion and resampling. The method improves the long-tailed distribution of the water meter dataset and the performance of small object detection. Moreover, to further improve the performance of detection model, we propose a block spatial attention module to connect with the output of the backbone of object detection model. This module can introduce priori knowledge into the convolutional neural network, reducing the phenomenon of missing objects. The experiments show that our methods successfully improve the performance of neural networks for the water meter digit reading recognition, and they could also be conveniently embedded into other mainstream general object detection models. In addition, we construct a dataset named waterDataset for noncommercial use, which is available at a Github repository: https://github.com/l1158984238/waterDataset.

Convolutional neural network has been successfully applied to image denoising. In particular, dilated convolution, which expands the network’s receptive field, has been widely used and has achieved good results in image denoising. Losing some image information, a standard network cannot effectively reconstruct tiny image details from noisy images. To solve this problem, we propose a pyramid dilated CNN, which mainly has three pyramid dilated convolutional blocks (PDCBs) and a gated fusion unit (GFU). PDCB uses dilated convolution to expand the network’s receptive field and the pyramid structure to obtain more image details. GFU fuses and enhances the feature maps from different blocks. Experiments demonstrate that the proposed method outperforms the comparative state-of-the-art denoising methods for gray and color images. In addition, the proposed method can effectively deal with real-world noisy images.

Convolutional neural network (CNN)-based facial emotion recognition (FER) lacks structural information, which affects the accuracy of FER. To automatically extract structural features and enrich the representation of expression features, we propose a dual-branch network based on CNN and graph CNN (GCNN) for FER. Specifically, one branch uses CNN to obtain the global features of expressions; in the parallel branch, we adopt a sparse coding strategy and propose a landmark-guided GCNN to extract the facial structure information under different expressions, enhancing the feature representation of key regions of facial expressions. The features of the two branches are fused through the channel attention mechanism to increase the semantic strength of the features to obtain more comprehensive and accurate expression features for expression classification. The entire deep neural network is trained end-to-end, and the extracted expression features are more discriminative, thereby improving the classification performance. Our experimental results prove the effectiveness of the proposed method on three publicly available datasets such as CK  +   (99.24%), Fer2013 (73.26%), and Raf-db (87.42%). The proposed algorithm for extracting image structure information can be flexibly implanted into any model of image classification, image segmentation, etc., where the input image has structural information.

An optical fringe analysis is very important in many scientific fields, such as optical metrology, rheology, and Fourier optics. It is proposed to analyze optical fringe patterns, obtained by a diffraction grating and Michelson’s interferometer, by treating them as training datasets for a multiclass classification problem, which is solved by using a support vector machine (SVM). The relation between a decision boundary function obtained by SVM and various optical parameters in the lab coordinate system is derived. The information that is extracted by the SVM algorithm is (1) the orientation angle of straight-line fringes; (2) the location of a smeared diffraction spot; (3) upper bounds on the spot size of each diffracted order; (4) the ratio between laser beam propagation distances; (5) the ratio between the radii of circular fringes of a Michelson’s interferogram; and (6) the orientation of elliptic fringes.

Existing supervised person re-identification (Re-ID) methods demonstrate excellent performance. However, their performances suffer degradation when tested on an unseen different distributed domain. Generative adversarial network-based (GAN-based) and clustering- or pseudolabel-based methods are proposed to alleviate this problem. Due to the transfer scheme and the ignorance of correlations between the style-transferred and the original target images, the performance of GAN-based methods is unsatisfactory. We resolve these problems by jointly employing a generative strategy and performing camera-aware clustering for the target domain. Style-transferred images are generated from source cameras to target cameras, and then they are merged into the target domain selectively after exploiting their domain-specific discriminative information. To reduce the noise in generated images, we propose a domain-level boundary separation loss to group the transferred images and push them away from the original target images. The camera-level neighborhood-based clustering is proposed to learn well-clustered features in a camera-aware manner. Extensive experiments on two commonly used person Re-ID datasets demonstrate that our proposed method can achieve state-of-the-art performance.

We propose a method to change the shadows cast by vehicles based on unpaired data, which is used to solve the problem of insufficient datasets for target detection in autonomous driving scenarios. This method can increase the training samples of road vehicles in different time periods and reduce the time needed to make complex annotations of datasets. Direct use of generative adversarial networks with cycle-consistency constraints can achieve mapping transformation of unpaired domains, but it is difficult to completely learn the mapping between different shadow domains. To solve this problem, V-ShadowGAN is proposed. First, to train the model, we construct an unpaired dataset of vehicle shadows in different time periods. Second, the shadow mask extractor and the shadow mask discriminator are constructed to extract shadow masks and guide the generation of shadows in the network. Finally, the spatial attention mechanism is added to the generator, which improves the overall quality of the generated image by changing the channel weight of the network and deepening the network. Various experiments show that V-ShadowGAN can effectively change the vehicle shadow.

Most screen content (SC) video applications are real time, and a low-delay transmission is needed. An effective low-delay transmission requires a precise video rate controller to fit the video bit rate to the capacity of the transmission channel. The rate controller has to support the unique standard coding tools of SC video. We propose a unique intra rate controller based on a unique content-based rate-distortion model for low delay applications of high efficiency video coding SC coding. The proposed model supports the unique SC coding tools and different modes and sizes of coding tree units. The proportional–integral–derivative control concept is used for frame-level bit allocation and rate control. The rate controller utilizes a small size virtual buffer to provide low-delay buffering constraints for the bit stream. Experimental results show that the proposed rate controller can control the bit rate of test sequences with a low buffering delay (about 20 ms) while the rate-distortion performance is maintained.

Maliciously forged images generated by image translation networks can cause significant security threats to personal privacy and national security. An emerging solution for forged images is preventing image forgery models from tampering with user images through adversarial attacks. Currently, conventional adversarial generation algorithms use random noise as the starting point, which makes the final adversarial output similar to the original output but does not prevent image tampering. The output-correlated initialization is applied to improve the adversarial attack algorithm for the image translation network and improve the visual effect of adversarial attacks. Moreover, a comparative experiment is performed on multiple loss functions, and the loss function with the best performance is selected as the adversarial loss function to complete the adversarial attack on the image translation network. The selected initialization method makes the search process of adversarial examples more comprehensive and makes the generation results of adversarial examples more diverse. The analysis of the visual effects of the attack reveals how the proposed adversarial attack methods affect the forgery results of different image translation frameworks and generate more chaotic images. Comparison of multiple indicators demonstrates that the proposed method has a high attack success rate and expands the image distance between the adversarial output and the original output, thereby improving the attack efficiency, preventing malicious tampering with the image, and protecting the user image.

Blind track scene has high complexity, and the existing target detection algorithm model is large, which cannot be directly applied to lightweight embedded platform, and it is easy to miss detection for small obstacles in long distance. To solve these problems, a real-time detection and ranging method of blind obstacles based on binocular vision is proposed. The method includes two stages: blind-track obstacle detection and blind-track obstacle ranging. In the obstacle detection stage, considering the requirements of embedded applications for model size and computing speed, a lightweight target detection network is designed. The network uses the deep separable convolution unit to extract the features of the blind road obstacles, and adds the channel attention module to filter and filter out high-quality feature information, so as to realize the fast detection of the blind road obstacles. To solve the problem of missed detection of long-distance small targets and obstacles, a multi-scale features fusion module is designed at the end of the network, which strengthens the detection ability of small targets of complex blind track scenes. In the obstacle ranging stage, an improved algorithm enhanced-speeded up robust features (ENH-SURF) based on binocular video is proposed to realize the real-time ranging of blind obstacles. Experimental results show that the detection accuracy of the detection algorithm is 83.2% in the blind road obstacle data set, and the evaluation indicators, such as model size and FPS, are better than the comparison algorithm. The distance measurement method is within range of 5 m, and the average relative error is 2.77%, which can meet the application needs of the blind.

It is one of the most critical technologies for unmanned electric locomotives to detect the obstacles in front of their operation quickly and accurately, which is of great significance for the safe operation of electric locomotives. Aiming at the problems of current computer vision detection methods, such as error warning, low detection accuracy, and slow detection speed, an obstacle intelligent detection method for unmanned electric locomotives based on an improved YOLOv3 (YOLOv3-4L) algorithm is proposed. The obstacle image data set of the electric locomotive running area is constructed to provide a testing environment for various obstacle detection algorithms. In the network structure, the darknet-53 feature extraction network is simplified, and the four-scale detection structure is formed by adding the shallow layer detection scale to the detection layer, which can improve the detection speed and accuracy of the algorithm for obstacles in front of the locomotive. Distance intersection over union loss function and Focal loss function are adopted to redesign the loss function of the target detector to further improve the detection accuracy of the algorithm. Traditional computer vision techniques such as perspective transformation, sliding window, and least square cubic polynomial are used to detect the track lines. By finding the area where the track was located and extending a certain distance to the outside of the track, the dangerous area of electric locomotive running is obtained. The improved YOLOv3 algorithm is utilized to detect obstacles, and only the types and positions of obstacles coincident with dangerous areas are output. The experimental results show that the traditional computer vision techniques such as perspective transformation, sliding window, and least square cubic polynomial can detect not only straight track but also curved track, which makes up for the shortcomings of the Hough transforms in detecting curved tracks. Compared with the original YOLOv3 algorithm, the YOLOv3-4L algorithm improves the mean average precision by 5.1%, and the detection speed increases by 7 fps. YOLOv3-4L detection model has high detection accuracy and speed, which can meet the actual working conditions and provide technical reference for unmanned driving of electric locomotives in underground coal mines.

In view of the difficulty in accurately recognizing vehicle models due to the wide variety of vehicle models and little differentiation between some models, a structure-guided attention network for weakly supervised learning fine-grained vehicle model recognition is proposed. First, we utilize the structure-guided attention location module to locate key features of images, extracting the strong and weak attention areas of the image by generating attention maps. Second, we investigate multichannel convolutional neural network for the learning of the strong-weak feature pairs and solve the problem of different network feature maps with different sizes by using feature map alignment. Third, an adaptive weighting method is proposed to balance the loss function. The experimental results show that our algorithm effectively improves the accuracy of the fine-grained vehicle recognition dataset and has universal applicability in other fine-grained recognition datasets.

The process of single-image super-resolution (SR) has certain limitations, such as an insufficient utilization of high-frequency information in images and a network structure that is insufficiently flexible to reconstruct the feature information of different complexities. Therefore, deep iterative residual back-projection networks are proposed. Residual learning was used to ease the difficulty in training and fully discover the feature information of the image, and a back-projection method was applied to study the interdependence between high- and low-resolution images. In addition, the network structure reconstructs smooth-feature and high-frequency information of the image separately and transmits only the residual features among all residual blocks of the network structure. The experiment results show that compared with most single-frame image SR methods, the proposed approach not only achieves a significant improvement in objective indicators, but it also provides richer texture information in the reconstructed predicted image.

The efficient identification of microcracks is of great significance to the early fault diagnosis of structures. The edge pixels of the microcracks have obvious changes that can be regarded as a salient object in the image for detection. Aiming at the differences in microcracks features at different scales of the encoder network, we proposed an embedded U-Net model that combines a cross-stage feature fusion module (CSF) and a layerwise feature fusion module (LWF). The embedded U-Net model can be used to enhance the spatial and edge information of high-resolution images and refine the extraction of low-resolution and high-dimensional channel information. In the low-scale saliency map produced by the decoder network, the bilinear interpolation method easily blurs and distorts the edges of the microcracks. To solve this problem, we send the low-quality saliency map into the subpixel super-resolution reconstruction module (SPR) to reconstruct high-quality pixel information. The SPR module is used to learn the low-resolution pixel information through the convolution operation, and the high-resolution pixels information can be obtained by the reorganization of learned pixels. A large number of experiments indicate that the 2_CSF+4_LWF+SPR has the best model performance. Its F1 score is 0.709, MAE is 0.0074, and the model parameter is only 55.6 MB. It can be seen from the qualitative analysis that LWF can improve the detection performance of microcrack ends and fuzzy cracks, and the SPR can refine the microcrack edges in the saliency map and reduce some interference from background impurities.

Multi-exposure image fusion algorithms in dynamic scenes usually have ghosts. Most of the existing ghost-free methods have complex calculations and the details will be lost. To solve these problems, we present a multi-exposure fusion method, which can remove ghosts and preserve more details of the image. The method detects ghost images by the improved frame difference method and calculates the early weight maps, combining the ghost detection binary image and initial weight maps to redefine weight maps. Then with the improved Laplacian pyramid algorithm, a high-quality image with more details is fused quickly. This is a significant improvement over existing techniques by objective and subjective comparisons.

Recent research on facial expression recognition (FER) in the wild shows challenges still remain. Different from laboratory-controlled expression in the past, images in the wild contain more uncertainties, such as different forms of face information occlusion, ambiguous facial images, noisy labels, and so on. Among them, real-world facial occlusion is the most general and crucial challenge for FER. In addition, because of the COVID-19 disease epidemic, people have to wear masks in public, which brings new challenges to FER tasks. Due to the recent success of the Transformer on numerous computer vision tasks, we propose a Collaborative Attention Transformer (CAT) network that first uses Cross-Shaped Window Transformer as the backbone for the FER task. Meanwhile, two attention modules are collaborated. Channel-Spatial Attention Module is designed to increase the attention of the network to global features. Moreover, Window Attention Gate is used to enhance the ability of the model to focus on local details. The proposed method is evaluated on two public in-the-wild facial expression datasets, RAF-DB and FERPlus, and the results demonstrate that our CAT performs superior to the state-of-the-art methods.

Constructing descriptors for key points in image matching is a crucial task in computer vision, visual tracking, and pattern recognition. Highly rated descriptors such as scale-invariant feature transform (SIFT) and speeded up robust features (SURF) utilize the grayscale gradient information of the square regions around the key points. Nevertheless, these famous descriptors fail to take advantage of the image color information. Furthermore, their performances will be severely degraded for distorted images and the error due to rotation of their square windows. To address the problems, we proposed Circular Coordinate Combining Shape–Color Descriptor Under Distortion Based SURF (CSCD-SURF) that achieves competitive performance for colorful images with distortion contamination and numerical similar structures. The proposed approach can generate scale-space by adaptive filter to rectify distortion and integrate the SURF descriptor and shape–color information into our own custom concentric circular coordinate with flying colors. Experiments with Institut National de Recherche en Infomatique et Automatique dataset and spherical radial distortion-SIFT distorted image datasets prove that the ability of distortion rectification as well as matching performance against rotation, scaling, viewpoints, and blurring is more competitive than the state-of-the-art methods and its computation cost is acceptable.

Establishing reliable correspondences in an image pair is a prerequisite and crucial in computer vision. It remains a difficult topic to separate true and false matches in the given putative dataset with a low inlier ratio. To address this problem, an image matching method is proposed, via the local neighborhood of feature points. Grid-based motion statistics are initially engaged to preprocess the putative dataset, especially in which the inlier ratio is low. The local neighborhood distributions of feature points are then collected as the quality function of correspondences. Progressive sample consensus is next employed to estimate a global deformation for removing false matches. Robust experiments on nine typical image pairs with low inlier ratios demonstrate the superiority of our proposed method over five state-of-the-art methods. The comparison experiments on the Oxford dataset illustrate that our method outperforms the other five image matching methods.

Automatic crack segmentation plays an important and challenging role in pavement maintenance. In recent years, researchers have been trying to figure out the task that long dependencies and global context information could not get well established using convolutional neural networks. A method of convolutional neural network (CNN) and transformer called CrackT-net is proposed to address this issue. In the aspect of the backbone network, we propose a new backbone network named richer features (RF) UNet++, in which skip connections, gated channel transformation, and polarized self-attention are added to the UNet++ to enhance feature representation capabilities. Then, to capture more long dependencies and global context information, the last feature extraction layer is replaced by the transformer in our network. In the deep supervision module, the proposed module can progressively polish the multilevel features to be more accurate. To prove the effectiveness of our proposed method, we evaluate it on the three public crack datasets, DeepCrack, CFD, and Crack500, which achieves F-score (F₁) values of 0.856, 0.700, and 0.637, respectively. After guided filtering, our method achieves F₁ values of 0.859, 0.710, and 0.637 on these three datasets.

The demand for nucleic acid detection is increasing worldwide as people are impacted by infectious diseases. Digital polymerase chain reaction (dPCR), which is a revolutionized nucleic acid absolute quantitative analysis technology, plays a significant role in the detection of genetic material. However, the traditional threshold-based methods fail to obtain high-accuracy segmentation results and real-time performance from dPCR fluorescence images with uneven illumination. Therefore, we proposed a robust two-stage automatic thresholding method to deal with dPCR fluorescence images. This method adopts the background subtraction method and the improved Otsu method to extract reaction chambers of dPCR images in two stages. Especially for the second stage, the peak-weaken method based on the improved Otsu method greatly improves the real-time segmentation of uneven illumination regions. Our method significantly improves the accuracy and processing speed of the dPCR image segmentation compared to several mainstream methods. This method could effectively segment the reaction chambers of dPCR images in complex environments. The average precision rate is 99.71%, the execution time is 0.139 s. Experimental results demonstrate that our proposed strategy has superior segmentation performance for high-throughput dPCR fluorescence images with uneven illumination.