We propose a cross-domain convolutional neural network (CNN) predictor-based reversible data hiding (RDH) using a masking strategy. The proposed cross-domain predictor fuses the dual features of the image’s spatial and frequency domains, which enhances the ability to simultaneously control both local and global features of the image. To the best of our knowledge, we are the first to report in detail on how to use cross-domain-based CNN to predict images for RDH. Furthermore, we develop the masking strategy to create a new training loss that ensures complete control over the predicted part. The experimental results show that the prediction error of the cross-domain predictor conforms very well to the Gaussian distribution, and the average number of zeros on the measured dataset is as high as 42,579. The peak signal-to-noise ratio (PSNR) value of the hidden boat image is able to reach 64.71 dB, and the average PSNR is also able to be above 51.63 dB in the case of a very large capacity (80,000 bits). Under different capacities, the average structural similarity index measure, universal quality index, and visual information fidelity values are as high as 0.9978, 0.9999, and 0.9736, respectively. This proves that the proposed cross-domain predictor outperforms the state-of-the-art predictors and generates hidden images with superior visual quality.

Light field salient object detection (SOD) is an essential research topic in computer vision, but robust saliency detection in complex scenes is still very challenging. We propose a new method for accurate and robust light field SOD via convolutional neural networks containing feature enhancement modules. First, the light field dataset is extended by geometric transformations such as stretching, cropping, flipping, and rotating. Next, two feature enhancement modules are designed to extract features from RGB images and depth maps, respectively. The obtained feature maps are fed into a two-stream network to train the light field SOD. We propose a mutual attention approach in this process, extracting and fusing features from RGB images and depth maps. Therefore, our network can generate an accurate saliency map from the input light field images after training. The obtained saliency map can provide reliable a priori information for tasks such as semantic segmentation, target recognition, and visual tracking. Experimental results show that the proposed method achieves excellent detection performance in public benchmark datasets and outperforms the state-of-the-art methods. We also verify the generalization and stability of the method in real-world experiments.

An event camera (EC) is a bioinspired vision sensor with the advantages of a high temporal resolution, high dynamic range, and low latency. Due to the inherent sparsity of space target imaging data, EC becomes an ideal imaging sensor for space target detection. In this work, we conduct detection of small space targets using a CeleX-V camera with a megapixel resolution. We propose a target detection method based on field segmentation, utilizing the event output characteristics of an EC. This method enables real-time monitoring of the spatial positions of space targets within the camera’s field of view. The effectiveness of this approach is validated through experiments involving real-world observations of space targets. Using the proposed method, real-time observation of space targets with a megapixel resolution EC becomes feasible, demonstrating substantial practical potential in the field of space target detection.

Yolo series models are extensive within the domain of object detection. Aiming at the challenge of small object detection, we analyze the limitations of existing detection models and propose a Vis-YOLO object detection algorithm based on YOLOv8s. First, the down-sampling times are reduced to retain more features, and the detection head is replaced to adapt to the small object. Then, deformable convolutional networks are used to improve the C2f module, improving its feature extraction ability. Finally, the separation and enhancement attention module is introduced to the model to give more weight to the useful information. Experiments show that the improved Vis-YOLO model outperforms the YOLOv8s model on the visdrone-2019 dataset. The precision improved by 5.4%, the recall by 6.3%, and the mAP50 by 6.8%. Moreover, Vis-YOLO models are smaller and suitable for mobile deployment. This research provides a new method and idea for small object detection, which has excellent potential application value.

X-ray imaging is essential for security inspection; nevertheless, the penetrability of X-rays can cause objects within a package to overlap in X-ray images, leading to reduced accuracy in manual inspection and increased difficulty in auxiliary inspection techniques. Existing methods mainly focus on object detection to enhance the detection ability of models for overlapping regions by augmenting image features, including color, texture, and semantic information. However, these approaches do not address the underlying issue of overlap. We propose a novel method for separating overlapping objects in X-ray images from the perspective of image inpainting. Specifically, the separation method involves using a vision transformer (ViT) to construct a generative adversarial network (GAN) model that requires a hand-created trimap as input. In addition, we present an end-to-end approach that integrates Mask Region-based Convolutional Neural Network with the separation network to achieve fully automated separation of overlapping objects. Given the lack of datasets appropriate for training separation networks, we created MaskXray, a collection of X-ray images that includes overlapping images, trimap, and individual object images. Our proposed generative separation network was tested in experiments and demonstrated its ability to accurately separate overlapping objects in X-ray images. These results demonstrate the efficacy of our approach and make significant contributions to the field of X-ray image analysis.

Local backlight dimming (LBD) is a promising technique for improving the contrast ratio and saving power consumption for liquid crystal displays. LBD consists of two crucial parts, i.e., backlight luminance determination, which locally controls the luminance of each sub-block of the backlight unit (BLU), and pixel compensation, which compensates for the reduction of pixel intensity, to achieve pleasing visual quality. However, the limitations of the current deep learning–based pixel compensation methods come from two aspects. First, it is difficult for a vanilla image-to-image translation strategy to learn the mapping relations between the input image and the compensated image, especially without considering the dimming levels. Second, the extensive model parameters make these methods hard to be deployed in industrial applications. To address these issues, we reformulate pixel compensation as an input-specific curve estimation task. Specifically, a deep lightweight network, namely, the curve estimation network (CENet), takes both the original input image and the dimmed BLUs as input, to estimate a set of high-order curves. Then, these curves are applied iteratively to adjust the intensity of each pixel to obtain a compensated image. Given the determined BLUs, the proposed CENet can be trained in an end-to-end manner. This implies that our proposed method is compatible with any backlight dimming strategies. Extensive evaluation results on the DIVerse 2K (DIV2K) dataset highlight the superiority of the proposed CENet-integrated local dimming framework, in terms of model size and visual quality of displayed content.

Video anomaly detection (VAD) is essential for monitoring systems. The prediction-based methods identify anomalies by comparing differences between the predicted and real frames. We propose an unsupervised VAD method based on frame memory bank (FMB) and decoupled asymmetric convolution (DAConv), which addresses three problems encountered with auto-encoders (AE) in VAD: (1) how to mitigate the noise resulting from jittering between frames, which is ignored; (2) how to alleviate the insufficient utilization of temporal information by traditional two-dimensional (2D) convolution and the burden for more computing resources in three-dimensional (3D) convolution; and (3) how to make full use of normal data to improve the reliability of anomaly discrimination. Specifically, we initially design a separate network to calibrate video frames within the dataset. Second, we design DAConv to extract features from the video, addressing the absence of temporal dimension information in 2D convolutions and the high computational complexity of 3D convolutions. Concurrently, the interval-frame mechanism mitigates the problem of information redundancy caused by data reuse. Finally, we embed an FMB to store features of normal events, amplifying the contrast between normal and abnormal frames. We conduct extensive experiments on the UCSD Ped2, CUHK Avenue, and ShanghaiTech datasets, achieving AUC values of 98.7%, 90.4%, and 74.8%, respectively, which fully demonstrates the rationality and effectiveness of the proposed method.

Recent advancements in transformer architectures have significantly enhanced image-denoising algorithms, surpassing the limitations of traditional convolutional neural networks by more effectively modeling global interactions through advanced attention mechanisms. In the domain of single-image denoising, noise manifests across various scales. This is especially evident in intricate scenarios, necessitating the comprehensive capture of multi-scale information inherent in the image. To solve transformer’s lack of multi-scale image analysis capability, a discrete wavelet and transformer based network (DTSIDNet) is proposed. The network adeptly resolves the inherent limitations of the transformer architecture by integrating the discrete wavelet transform. DTSIDNet independently manages image data at various scales, which greatly improves both adaptability and efficiency in environments with complex noise. The network’s self-attention mechanism dynamically shifts focus among different scales, efficiently capturing an extensive array of image features, thereby significantly enhancing the denoising outcome. This approach not only boosts the precision of denoising but also enhances the utilization of computational resources, striking an optimal balance between efficiency and high performance. Experiments on real-world and synthetic noise scenarios show that DTSIDNet delivers high image quality with low computational demands, indicating its superior performance in denoising tasks with efficient resource use.

Crowd counting has gained widespread attention in the fields of public safety management, video surveillance, and emergency response. Currently, background interference and scale variation of the head are still intractable problems. We propose an attention-injective scale aggregation network (ASANet) to cope with the above problems. ASANet consists of three parts: shallow feature attention network (SFAN), multi-level feature aggregation (MLFA) module, and density map generation (DMG) network. SFAN effectively overcomes the noise impact of a cluttered background by cross-injecting the attention module in the truncated VGG16 structure. To fully utilize the multi-scale crowd information embedded in the feature layers at different positions, we densely connect the multi-layer feature maps in the MLFA module to solve the scale variation problem. In addition, to capture large-scale head information, the DMG network introduces successive dilated convolutional layers to further expand the receptive field of the model, thus improving the accuracy of crowd counting. We conduct extensive experiments on five public datasets (ShanghaiTech Part_A, ShanghaiTech Part_B, UCF_QNRF, UCF_CC_50, JHU-Crowd++), and the results show that ASANet outperforms most of the existing methods in terms of counting and at the same time demonstrates satisfactory superiority in dealing with background noise in different scenes.

The current smart cooler’s commodity identification system first locates the item being purchased, followed by feature extraction and matching. However, this method often suffers from inaccuracies due to the presence of background in the detection frame, leading to missed detections and misidentifications. To address these issues, we propose an end-to-end You Only Look Once (YOLO) for detection and segmentation algorithm. In the backbone network, we combine deformable convolution with a channel-to-pixel (C2f) module to enhance the model feature extraction capability. In the neck network, we introduce an optimized feature fusion structure, which is based on the weighted bi-directional feature pyramid. To further enhance the model’s understanding of both global and local context, a triple feature encoding module is employed, seamlessly fusing multi-scale features for improved performance. The convolutional block attention module is connected to the improved C2f module to enhance the network’s attention to the commodity image channel and spatial information. A supplementary segmentation branch is incorporated into the head of the network, allowing it to not only detect targets within the image but also generate precise segmentation masks for each detected object, thereby enhancing its multi-task capabilities. Compared with YOLOv8, for box and mask, the precision increases by 3% and 4.7%, recall increases by 2.8% and 4.7%, and mean average precision (mAP) increases by 4.9% and 14%. The frames per second is 119, which meets the demand for real-time detection. The results of comparative and ablation studies confirm the high accuracy and performance of the proposed algorithm, solidifying its foundation for fine-grained commodity identification.

With the substantial surge in available internet video data, the intricate task of video summarization has consistently attracted the computer vision research community to summarize the videos meaningfully. Many recent summarization techniques leverage bidirectional long short-term memory for its proficiency in modeling temporal dependencies. However, its effectiveness is limited to short-duration video clips, typically up to 90 to 100 frames. To address this constraint, the proposed approach incorporates global and local multi-head attention, effectively capturing temporal dependencies at both global and local levels. This enhancement enables parallel computation, thereby improving overall performance for longer videos. This work considers video summarization as a supervised learning task and introduces a deep summarization architecture called multi-head attention with reinforcement learning (MHA-RL). The architecture comprises a pretrained convolutional neural network for extracting features from video frames, along with global and local multi-head attention mechanisms for predicting frame importance scores. Additionally, the network integrates an RL-based regressor network to consider the diversity and representativeness of the generated video summary. Extensive experimentation is conducted on benchmark datasets, such as TVSum and SumMe. The proposed method exhibits improved performance compared to the majority of state-of-the-art summarization techniques, as indicated by both qualitative and quantitative results.

Image inpainting techniques based on deep learning have shown significant improvements by introducing structure priors, but still generate structure distortion or textures fuzzy for large missing areas. This is mainly because series networks have inherent disadvantages: employing unreasonable structural priors will inevitably lead to severe mistakes in the second stage of cascade inpainting framework. To address this issue, an appearance flow-based structure prior (AFSP) guided image inpainting is proposed. In the first stage, a structure generator regards edge-preserved smooth images as global structures of images and then appearance flow warps small-scale features in input and flows to corrupted regions. In the second stage, a texture generator using contextual attention is designed to yield image high-frequency details after obtaining reasonable structure priors. Compared with state-of-the-art approaches, the proposed AFSP achieved visually more realistic results. Compared on the Places2 dataset, the most challenging with 1.8 million high-resolution images of 365 complex scenes, shows that AFSP was 1.1731 dB higher than the average peak signal-to-noise ratio for EdgeConnect.

Multi-view clustering has garnered significant attention due to its ability to explore shared information from multiple views. Applications of multi-view clustering include image and video analysis, bioinformatics, and social network analysis, in which integrating diverse data sources enhances data understanding and insights. However, existing multi-view models suffer from the following limitations: (1) directly extracting latent representations from raw data using encoders is susceptible to interference from noise and other factors and (2) complementary information among different views is often overlooked, resulting in the loss of crucial unique information from each view. Therefore, we propose a distinctive double-level deep multi-view collaborative learning approach. Our method further processes the latent representations learned by the encoder through multiple layers of perceptrons to obtain richer semantic information. In addition, we introduce dual-path guidance at both the feature and label levels to facilitate the learning of complementary information across different views. Furthermore, we introduce pre-clustering methods to guide mutual learning among different views through pseudo-labels. Experimental results on four image datasets (Caltech-5V, STL10, Cifar10, Cifar100) demonstrate that our method achieves state-of-the-art clustering performance, evaluated using standard metrics, including accuracy, normalized mutual information, and purity. We compare our proposed method with existing clustering algorithms to validate its effectiveness.

Camera localization is a technique for obtaining the camera’s six degrees of freedom using the camera as a sensor input. It is widely used in augmented reality, autonomous driving, virtual reality, etc. In recent years, with the development of deep-learning technology, absolute pose regression has gained wide attention as an end-to-end learning-based localization method. The typical architecture is constructed by a convolutional backbone and a multilayer perception (MLP) regression header composed of multiple fully connected layers. Typically, the two-dimensional feature maps extracted by the convolutional backbone have to be flattened and passed into the fully connected layer for pose regression. However, this operation will result in the loss of crucial pixel position information carried by the two-dimensional feature map and adversely affect the accuracy of the pose estimation. We propose a parallel structure, termed SMLoc, using a spatial MLP to aggregate position and orientation information from feature maps, respectively, reducing the loss of pixel position information. Our approach achieves superior performance on common indoor and outdoor datasets.

To address limitations in current flame and smoke detection models, including difficulties in handling irregularities, occlusions, large model sizes, and real-time performance issues, this work introduces FS-YOLO, a lightweight attention-based model. FS-YOLO adopts an efficient architecture for feature extraction capable of capturing long-range information, overcoming issues of redundant data and inadequate global feature extraction. The model incorporates squeeze-enhanced-axial-C2f to enhance global information capture without significantly increasing computational demands. Additionally, the improved VoVNet-GSConv-cross stage partial network refines semantic information from higher-level features, reducing missed detections and maintaining a lightweight model. Compared to YOLOv8n, FS-YOLO achieves a 1.4% increase and a 1.0% increase in mAP0.5 and mAP0.5:0.95, respectively, along with a 1.3% improvement in precision and a 1.0% boost in recall. These enhancements make FS-YOLO a promising solution for flame and smoke detection, balancing accuracy and efficiency effectively.

Universal source-free domain adaptation (USFDA) aims to explore transferring domain-consistent knowledge in the presence of domain shift and category shift, without access to a source domain. Existing works mainly rely on prior domain-invariant knowledge provided by the source model, ignoring the significant discrepancy between the source and target domains. However, directly utilizing the source model will generate noisy pseudo-labels on the target domain, resulting in erroneous decision boundaries. To alleviate the aforementioned issue, we propose a two-stage USFDA approach based on prompt learning, named USDAP. Primarily, to reduce domain differences, during the prompt learning stage, we introduce a learnable prompt designed to align the target domain distribution with the source. Furthermore, for more discriminative decision boundaries, in the feature alignment stage, we propose an adaptive global-local clustering strategy. This strategy utilizes one-versus-all clustering globally to separate different categories and neighbor-to-neighbor clustering locally to prevent incorrect pseudo-label assignments at cluster boundaries. Based on the above two-stage method, target data are adapted to the classification network under the prompt’s guidance, forming more compact category clusters, thus achieving excellent migration performance for the model. We conduct experiments on various datasets with diverse category shift scenarios to illustrate the superiority of our USDAP.

Thermal radiation and texture data from two different sensor types are usually combined in the fusion of infrared and visible images for generating a single image. In recent years, convolutional neural network (CNN) based on deep learning has become the mainstream technology for many infrared and visible image fusion methods, which often extracts shallow features and ignores the role of long-range dependencies in the fusion task. However, due to its local perception characteristics, CNN can only obtain global contextual information by continuously stacking convolutional layers, which leads to low network efficiency and difficulty in optimization. To address this issue, we proposed a global context fusion network (GCFN) to model context using a global attention pool, which adopts a two-stage strategy. First, a GCFN-based autoencoder network is trained for extracting multi-scale local and global contextual features. To effectively incorporate the complementary information of the input image, a dual branch fusion network combining CNN and transformer is designed in the second step. Experimental results on a publicly available dataset demonstrate that the proposed method outperforms nine advanced methods in fusion performance on both subjective and objective metrics.

Epidermal growth factor receptor (EGFR) is a key gene for diagnosing non-small cell lung cancer (NSCLC), which affects subsequent treatment arrangements. In clinical practice, compared with invasive testing, non-invasive detection of such genes can improve diagnostic efficiency and alleviate patient suffering. We propose a non-invasive decision-support method aimed at identifying EGFR mutations or wild type in NSCLC to avoid traumatic examination. To alleviate the inherent translational invariance of convolutional neural networks (CNNs) that results in the insufficiency to capture long-range dependencies, we propose a EGFR mutations or wild-type recognition method that combines CNNs with Transformers. In addition, we develop a lightweight dot-product strategy to address computational complexity, which not only reduces computational demands but also improves the interaction of image sequence information. The proposed method demonstrates superior performance, with an area under the curve of 88.56%, accuracy (ACC) of 83.17%, specificity (Spe) of 85.34%, and sensitivity (Sen) of 80.25%. In particular, the gradient-weighted class activation mapping analysis further shows that our algorithm more effectively concentrates on activation regions relevant to the predictions. The results of the experiments demonstrated that our proposed method is effective in distinguishing between EGFR mutations and wild type, showing superior classification performance compared with other methods. Notably, it introduces a non-invasive alternative for clinical examination, which is of considerable importance for the clinical diagnosis of NSCLC.

Recent years have witnessed tremendous success of convolutional neural networks in image classification. In the training of image classification, softmax cross-entropy (SCE) loss is widely used because of its clear physical definition and concise gradient calculation. We reformulate SCE loss into a more basic form, in light of which we can understand SCE loss better and rethink the loss functions in image classification. We propose a novel loss function called another metric (AM) loss, which is different from SCE loss in form but is the same in essence. AM loss not only retains the advantages of SCE loss but also provides more gradient information at the later phase of training and has larger inter-classes distances. Extensive experiments on CIFAR-10/100, Tiny ImageNet, and three fine-grained datasets show that AM loss can outperform SCE loss, which also shows the great potential of improving deep neural network training by exploring new loss functions.

Synthetic aperture radar (SAR) images provide high-resolution observations for military targets that can be used for target recognition. A SAR target recognition method is proposed based on correlation analysis and joint decision of multi-level bidimensional intrinsic mode functions (BIMFs) extracted by bidimensional variational mode decomposition (BVMD). As an image decomposition algorithm, BVMD represents SAR images by several BIMFs to describe the global and local properties of targets. Furthermore, non-linear correlation information entropy is adopted as a quantitative measure to choose the optimal subset from the multi-level BIMFs, which are assumed to share high correlations. In the classification phase, joint sparse representation is first employed to represent the selected BIMFs and output their corresponding reconstruction error vectors as well as decisions based on the minimum error principle. Afterward, a voting strategy is conducted on their individual decisions to reach a candidate target label. Once the voting decision is judged reliable, the classification process ends and confirms the candidate target label. Otherwise, a linear weighting algorithm is performed on the reconstruction error vectors from different BIMFs, in which the weights are decided by the voting result. Finally, the target label is determined based on the fused reconstruction errors. Experiments are conducted under different situations including the standard operating conditions and extended operating conditions from the moving and stationary target acquisition and recognition dataset. The results of the proposed method are analyzed and compared with other methods to validate its performance.

Semantic segmentation for road scenes aims to predict specific categories for each pixel in road images, playing a crucial role in various applications such as autonomous driving, scene recognition, and robotics. However, existing semantic segmentation methods face the challenge of maintaining competitive segmentation accuracy while achieving real-time inference speed. To address this, we introduce the multi-stage feature fusion network (MSF2Net) for real-time semantic segmentation of road scenes, achieving a balance between speed and accuracy. First, we develop a lightweight dilate symmetric interaction module (DSIM) to extract rich local and contextual information from images. Next, we enhance the spatial and semantic information of shallow and deep features by stacking DSIMs with different numbers and dilation rates, respectively. Finally, the position attention module supplements the global information of the image. Subsequently, a feature fusion module is utilized to integrate shallow, deep, and global features, thereby achieving real-time semantic segmentation through multi-stage feature fusion. Experimental results on multiple benchmark datasets demonstrate that MSF2Net achieves a good balance between segmentation performance and inference speed.

Dense fog often appears on highways at night and early in the morning, seriously affecting visibility and posing important traffic safety risks. The surveillance images taken during this period have poor imaging quality due to low light, which brings great challenges to visibility estimation from surveillance images. To cope with this problem, we propose a deep network with low-light enhancement capability to achieve automatic visibility estimation. The network model consists of two parts: the light enhancement sub-network and the visibility classification sub-network. The light enhancement sub-network adopts the generative adversarial network architecture to generate the light-enhanced image under the constraint of discriminator loss and self-feature retention loss. The visibility classification sub-network extracts the visual features of the enhanced image and combines them with bright channels to classify through a multi-head attention mechanism, achieving effective detection of visibility. At the same time, we construct a nighttime highway fog image dataset by collecting surveillance videos of several highways in China. Experimental results on this real-scene dataset show that the proposed algorithm achieves the highest detection accuracy, which can provide technical support for the intelligent management of the highway management department.

In harsh environments such as rain, snow, fog, and night, the road conditions are complex, and the existing road target detection algorithms are less studied, the amount of data is scarce, and the detection accuracy is not high. Aiming at the problems of less road target detection data and more interference factors in harsh environments, a road target detection method based on improved you only look once version 8n (YOLOv8n) in harsh environments is proposed. Based on the Berkeley DeepDrive-IW (BDD-IW) dataset, the improved cutout algorithm is used to simulate the occlusion of the targets to be detected and the uneven visibility of the detected images caused by the harsh environments in real scenarios to enhance the robustness. In YOLOv8n, a space-to-depth convolution (SPD-Conv) module consisting of a space-to-depth layer and a non-strided convolutional layer is added to the previous layer of each cross-stage partial Darknet53 to two-stage feature pyramid network (C2f) module of the backbone section to compensate for the loss of fine-grained image information caused by a strided convolution. On this basis, the C2f module and the parallel polarized self-attention (PPSA) mechanism are used to construct the feature extraction module cross-stage partial DarkNet53 and polarimetric self-attention to two-stage feature pyramid network and replace the C2f module in the backbone of YOLOv8n to reduce the information loss caused by more noise in harsh environments. To further improve the precision, the up-sampling part of YOLOv8n is modified to the content-aware reassembly of the feature operator to effectively aggregate the context information and expand the receptive field. The experimental results show that for BDD-IW data processed with improved cutout, compared with the original network, the improved YOLOv8n network has a small increase in the number of parameters and increases the precision by 6.0% and the mAP50 value by 5.0%, which effectively improves the performance of target detection in harsh environments.

In the natural driving state, the autonomous vehicle needs to make its driving decision, which requires precise traffic sign recognition with different distances and angles according to road traffic rules. However, the change in the observation points of autonomous cameras causes traffic signs to exhibit visual characteristics of variable size and complex backgrounds. The coupling of multiple characteristics poses severe challenges for traffic sign identification and positioning. In this paper, a multi-scale feature extraction and coordinate network based on the YOLO framework (MFEC-YOLOv7) is proposed to achieve accurate traffic sign recognition in complex situations. In more detail, the feature for traffic signs at different scales is extracted and enhanced by fusing the pooling layer and group convolution in the convolutional architecture of the backbone network, thus improving the multi-scale feature perception capability. Furthermore, the coordinate attention (CA) module is introduced to pay attention to traffic signs under multiple interference factors. And the collaborative working mechanism of the bidirectional feature pyramid is brought in the neck network, and combined with the CA module to accurately locate the position information of features, to enhance the feature extraction ability of the model. Besides, depth separable convolution is adopted to significantly reduce the number of convolution parameters and improve the efficiency without reducing the effectiveness of feature extraction. The experimental results on the public data set TT100K and the self-built data set show that compared with the YOLOv7 method, MFEC-YOLOv7 improves the detection accuracy by 19.9%, reduces the amount of parameter calculation by 9.5%, and improves the speed by about 16.9%.

We propose an approach of Ladder Bottom-up Convolutional Bidirectional Variational Autoencoder (LCBVAE) architecture for the encoder and decoder, which is trained on the image translation of the dotted Arabic expiration dates by reconstructing the Arabic dotted expiration dates into filled-in expiration dates. We employed a customized and adapted version of the convolutional recurrent neural network (CRNN) model to meet our specific requirements and enhance its performance in our context, and then we trained the custom CRNN model with the filled-in images from the year 2019 to 2027 to extract the expiration dates and assess the model performance of LCBVAE on the expiration date recognition. The pipeline of (LCBVAE + CRNN) can be then integrated into an automated sorting system for extracting the expiry dates and sorting the products accordingly during the manufacturing stage. In addition, it can overcome the manual entry of expiration dates that can be time-consuming and inefficient at the merchants. Due to the lack of available dotted Arabic expiration date images, we created an Arabic dot-matrix TrueType font for the generation of the synthetic images. We trained the model with unrealistic synthetic dates of 60,000 images and performed the testing on a realistic synthetic date of 3000 images from the year 2019 to 2027, represented as yyyy/mm/dd. We demonstrated the significance of the latent bottleneck layer by improving the generalization when the size is increased up to 1024 in downstream transfer learning tasks for image translation. The proposed approach achieved an accuracy of 97% on the image translation using the LCBVAE architecture that can be generalized for any downstream learning tasks for image translation and reconstruction.

We propose a reversible data-hiding approach in encrypted images (RDHEI) to cope with the new security threats emerging with artificial intelligence and the Industrial Internet of Things (IIoT). This dual image-based high-capacity RDHEI technique ensures security with better embedding capacity. On the sender side, the correlations of adjacent pixels are disrupted by encryption. Hence, except for the intended receiver, no one can intercept the image. This encrypted image is used as a cover image, and pixel value ordering is applied. After that, the center folding strategy is used to form dual stego images for hiding data inside both images. Pixels with maximum and minimum values are employed for data hiding. The experimental results indicate the greater resilience and imperceptibility of the discussed technique over several current strategies. The suggested plan incorporates secret bits based on the image properties. One benefit of using the suggested technique is that the process of decrypting the image and extracting the data are independent. The recipient with just the data-hiding key may recover the concealed data without any compromise, while the receiver with only the decryption key can precisely retrieve the original image without any knowledge of the data-hiding key. This makes this scheme suitable for IIoT devices with a limited processing power for image collection.

Existing watermarking schemes typically use binary or grayscale images as watermarks, which cannot meet the needs for color image copyright protection in the Internet of Things environment. To address these issues, we propose an adaptive robust watermarking scheme in YCbCr color space, based on integer wavelet transform, Hadamard transform, and singular value decomposition (SVD). Applying the Hadamard transform to SVD can effectively enhance the anti-interference capability of watermarking technology. The scheme utilizes the rime optimization algorithm (RIME) to identify the optimal embedding strength factor for each image, thereby achieving a balance between imperceptibility and robustness. To ensure the security of the watermark information, the algorithm employs dual encryption techniques, using the scrambled generalized Gray code transform and the four-dimensional hyperchaotic Lorenz system to encrypt the watermarked image. The experimental results show that the peak signal-to-noise ratio of watermarked images is above 54 dB, the structural similarity index metric is close to 1, and a hiding capacity of 0.125 bpp has been achieved. The normalized cross-correlation of the watermarked images extracted after various attacks is above 0.9. Compared with existing watermarking schemes, this scheme has significant advantages in terms of invisibility and robustness.

Convolutional neural networks (CNNs) can use larger convolutional kernels to provide a wide receptive field, achieving long-distance context processing similar to that of transformers with fewer model parameters. However, the complex and dynamic degradation of underwater environments makes it difficult for fixed large-kernel CNNs to adaptively capture complex multi-scale features underwater and dynamically integrate a broad range of contextual information. To address these issues, we propose the integrated dynamic context network, which adopts multi-scale receptive fields and adaptively processes global contextual information. In its core module, the integrated dynamic context module is incorporated; it uses multiple different-sized kernels to capture multi-scale features and designs a dynamic selection mechanism that adaptively emphasizes the most critical spatial features while fully utilizing extensive information. The proposed dynamic selective feature fusion module promotes valuable screening of redundant features through the fusion of multi-scale feature maps derived from the encoder and the previous layer decoder. Extensive experimental results verify the superior performance of the proposed method in addressing these challenges.

Underwater image enhancement is challenging due to the complex and variable degradations induced by environmental disturbances. While deep learning methods recently have achieved substantial progress in image enhancement, their performance is still hampered by the limited understanding of underwater image characteristics. To address this, we propose the hierarchical decomposition-based underwater image enhancement network (HDNet), which tackles key elements of underwater image degradation. Our model employs macroscopic channel decomposition and microscopic pixel decomposition to correct color distortion from light attenuation and restore detail loss from scattering. In the pixel decomposition process, HDNet interprets pixels as perturbed light signals and refines image details by representing and modulating these signals, emulating the principles of the Fourier transform. In addition, we introduce an auxiliary gradient guidance strategy to mitigate the effects of poor reference images during training. HDNet demonstrates good performance across multiple datasets. Notably, on the underwater image enhancement dataset, a real-world underwater dataset, HDNet achieves a peak signal-to-noise ratio of 24.158 dB and a structural similarity index of 0.923, outperforming many previous state-of-the-art models. Its low parameter count and computational efficiency make it suitable for practical, real-world applications.

Facial expression recognition (FER) is crucial for understanding and assessing human emotional states. However, in practical applications, due to the complexity and diversity of facial expressions, traditional self-supervised contrastive learning methods are often difficult to extract fine-grained expression features. To address this problem, we propose an attention-guided self-supervised distilled contrastive learning method for FER, which transfers the expression differential information learned by the teacher network to the student network by introducing attention-guided knowledge distillation in self-supervised contrastive learning. Specifically, we propose attention-guided joint feature distillation to strengthen the feature representation capability of the student network by guiding the student network through feature learning with joint attention-weighted features and comparison query vectors. In addition, to further utilize the key information in the teacher’s features, the facial key feature guidance is also proposed to make the student more focused on learning the key features extracted from the teacher’s network. These advances lead to significant performance improvements, showcasing the robustness of our method. Our method obtains excellent results of 76.83% on the Real-world Affective Face Database and 62.04% on the FER-2013 datasets, respectively, demonstrating its effectiveness in capturing subtle emotional expressions and advancing the field of self-supervised FER.

We propose a joint scrambling and diffusion image encryption algorithm based on RNA theory and row–column–level operation (JSDRR), encompassing the following procedures: initially, pseudo-random sequences are generated utilizing a six-dimensional hyper-chaotic system, which serves to augment the randomization in the image encryption process. Subsequently, the theory of RNA is integrated with the row–column–level cryptographic approach for combined scrambling and diffusion, thereby augmenting the efficacy of the proposed algorithm. Finally, we elucidate a cutting-edge joint scrambling and diffusion image encryption algorithm, designed to withstand a multiplicity of attacks. Employing simulation and comparative analytical methodologies, it is demonstrated that the JSDRR exhibits robustness against a varied spectrum of attacks with commendable efficiency.

In fields such as autonomous driving and 3D object reconstruction, complete 3D point cloud data is crucial. Existing methods often directly reconstruct complete point clouds from partial ones, overlooking the structural similarities within the point cloud data. To tackle this challenge, we introduce SAPCNet, an innovative network architecture that leverages the symmetry and structural similarities of point clouds to infer missing parts from known parts. We assume that incomplete point clouds share topological similarities with their symmetric counterparts. Through a feature-position pair extractor, we extract the center point and its features, which are then fused into an existing proxy. With our proposed symmetry-aware transformer, we analyze these features to accurately predict the positions of symmetric point proxies. In addition, we introduce a fine-seed generator to bridge the gap between the predicted missing point cloud and the original input point cloud, ensuring that the reconstructed point cloud maintains the geometric structure and visual characteristics consistent with the original data. Through a series of qualitative and quantitative evaluations, SAPCNet demonstrates outstanding performance across multiple datasets.

Relative pose estimation is crucial for various computer vision applications, including robotic and autonomous driving. Current methods primarily depend on selecting and matching feature points prone to incorrect matches, leading to poor performance. Consequently, relying solely on point-matching relationships for pose estimation is a huge challenge. To overcome these limitations, we propose a geometric correspondence graph neural network that integrates point features with extra structured line segments. This integration of matched points and line segments further exploits the geometry constraints and enhances model performance across different environments. We employ the dual-graph module and feature-weighted fusion module to aggregate geometric and visual features effectively, facilitating complex scene understanding. We demonstrate our approach through extensive experiments on the DeMoN and Karlsruhe Institute of Technology and Toyota Technological Institute Odometry datasets. The results show that our method is competitive with state-of-the-art techniques.

Localization of image forgery has gained significance now as image deception techniques and software are widely used. Currently, deep learning–oriented approaches and generative adversarial network (GAN)-based models are used for forgery detection and localization. But, the generation of images from noise is a challenge, which adds complexity to architecture resulting in bulk data requirements and long training times. We highlight a redesigned framework of GAN with comparatively less architectural complexity ensuring good performance with an acceptable amount of training data and training times. The proposed network consists of a generator based on an encoder-decoder architecture that generates masks corresponding to the images, followed by a mask refinement network to enhance the generated masks. The generator training is done with images and their corresponding masks and the refinement network using the truth masks. Eventually, after combined training using the images and matching masks, the combined network locates the forged areas. The reframed network is experimented with three publicly available datasets, and the model performance is qualitatively analyzed by the localization maps generated and is quantitatively analyzed by the performance metrics such as receiver operator characteristics, area under the curve (AUC), precision, and F1 score. Both analyses show that the model localizes forgeries well, and a comparison of the suggested model to the existing techniques reveals that it performs better while requiring less architectural complexity. The recommended strategy beats the most advanced method that was compared, by an average margin of 10.33% in AUC and 31.9% in F1 score. The suggested model has a 37.3% reduction in depth and a 10.3% gain in average AUC when compared with the best state-of-the-art technique. Moreover, the proposed model has 27.5% and 19.5% improvement in average F1 score and precision, respectively, compared with the GAN-based model.

The creation of three-dimensional (3D) models is a challenging problem, and the existing point cloud-based reconstruction methods have achieved some success by directly generating a point cloud in a single stage. However, these methods have certain limitations and cannot accurately reconstruct 3D point cloud models with complex surface structures. We propose a learning-based reconstruction method to generate dense point clouds by learning multiple features of sparse point clouds. First, the image encoder embedded in the attention mechanism is used to improve the attention of the network to the target local area, and the decoder is used to generate a sparse point cloud. Second, a point cloud feature extraction block was designed to extract the effective features describing the sparse point cloud. Finally, the decoder was used to generate dense point clouds to complete the point cloud refinement. By evaluating the targets with different surface structures, verifying the effectiveness of the network by comparing with other reconstruction methods with different principles, and carrying out measurement experiments on real objects, the 3D error of the point cloud obtained is <2 mm, which meets the practical requirements.

Deep learning-based algorithms for enhancing underwater images have demonstrated outstanding performance in recent years. However, numerous limitations exist in applying existing methods to various underwater environments, viewed as water types. The lack of generalization ability and visual consistency remain significant challenges, as they neglect the domain gaps of underwater scenes and the differences in the multi-frequency nature of information. A critical desideratum in underwater image enhancement is to establish a strong correspondence between the learned features and water types while also exploring multi-frequency analysis. In response to these issues, we propose a novel domain adaptive multi-frequency underwater image enhancement network. This network leverages the capability of adapting feature learning to varying water types while also supporting a simultaneous feature decode for high and low frequencies. Specifically, we first construct a module for domain adaptation with a water type classifier to distinguish the impacts of different water types, which naturally reveals domain adaptation between water types. This enhances the model’s sensitivity to water types by generating the domain-sensitive feature. In addition, we design a dual-branch decoder with self-attention to decompose the domain-sensitive features into sub-bands of high and low frequencies. Each branch contains distinct semantic content aimed at promoting multi-frequency feature complementarity. This helps the model to accurately capture local detail and global context, thereby further improving image quality. Finally, we design a multi-stage training strategy equipped with multiple loss functions to improve the stability of the model. The proposed method can adaptively deal with degraded images of different water types and enhance underwater images with high quality by processing high-frequency and low-frequency components synchronously. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art methods in terms of both visual quality and evaluation metrics.

Fires frequently result in significant casualties and property damage across various contexts. To explore a detection method that can accurately identify fire precursors, we propose a fire detection method of improved YOLOv8n to accurately identify flame and smoke. First, the network is enhanced through channel shuffling and reparameterized convolution, and the network computational efficiency is improved by aggregating multiple feature cascades at one time. The linear weighted self-attention mechanism is used to enhance the information extraction ability of the feature extraction network for 3D feature maps. In the neck network, the input feature map is dynamically upsampled based on the function-based point sampling method, which effectively improves the model’s feature extraction ability for high-resolution feature maps. Finally, a detection head structure based on an adaptive spatial feature fusion strategy is proposed to effectively filter spatial conflict information while suppressing the scale invariance of features of different scales, thereby improving the detection ability of objects of different scales. Experiments on the flame and smoke detection dataset proposed in this study show that the accuracy, recall, mAP50, and mAP95 of the improved network model reached 88.32%, 81.97%, 90.37%, and 68.20%, respectively, which are 7.01%, 3.84%, 4.16%, and 6.87% higher than the baseline model, respectively. All indicators are significantly better than the baseline and existing methods and still meet the real-time requirements.

Visual object tracking in rain is crucial for various civilian and military applications, such as rainy traffic, flood rescue, boundary surveillance, and all-weather combat. However, most existing trackers are designed for favorable weather conditions and experience a remarkable decline in performance when used in rainy environments. Resulting reasons include image degradation caused by rain streak pollution and contrast reduction, as well as frequent changes in target appearance in rainy video sequences. To solve this problem, we propose a visual object tracking algorithm called SiamDRS based on adaptive rain streak removal and fused dynamic template matching. The proposed algorithm first measures the rain streak interference level in the input image by using a specially designed calculation mechanism. An adaptive rain streak removal module then generates a high-contrast, rain-free image to ensure that the tracker can extract sufficient target features. Subsequently, the fused dynamic template matching mechanism continually updates the tracking template fused with historical target information for matching and locating the target during tracking, thereby reducing the tracking failure rate caused by target appearance changes. In addition, to break through the limit of scarce rainy scene datasets for object tracking testing, we present nine rainy testing datasets created through artificial synthesis and manual collection and annotation of real-world rainy data. Furthermore, we perform a comparative evaluation of the proposed algorithm against existing algorithms to demonstrate its effectiveness and robustness in rainy scenes.

In the architecture, engineering, and construction (AEC) industry, point cloud semantic segmentation provides comprehensive and accurate data support for building information modeling (BIM) and is one of the key technologies for building digital twins. However, the complexity and diversity of building semantic categories and the incompleteness of the current building point cloud datasets for training make deep learning–based semantic segmentation of building point clouds still a challenging task. We systematically summarize the existing classical point cloud semantic segmentation algorithms and further compare and analyze the state-of-the-art point cloud semantic segmentation algorithms of buildings according to two application scenarios: outdoor and indoor. Second, we summarize the point cloud datasets applicable to the AEC field and quantitatively analyze and compare the semantic segmentation performance of various algorithms according to different application scenarios. Finally, we explore the research directions and application prospects of point cloud semantic segmentation algorithms in the field of AEC, encompassing data acquisition and processing, scene detection and reconstruction, digital twin, etc.

Due to the rapid developments in network technology, the efficiency of image data acquisition, transmission, and storage has been significantly enhanced, leading to an explosive growth in the volume of image data on the internet. Deep hashing models have swiftly emerged as a prominent solution in the field of image retrieval due to their exceptional retrieval accuracy and efficiency. The influx of new data can result in changes in the data distribution, resulting in the problem of concept drift. Furthermore, the new data comprise a significant amount of unlabeled data. To address these issues, we propose a semi-supervised incremental deep hashing image retrieval model. (1) To address the issue of concept drift, we propose a method of using predefined cluster centers for clustering. This method employs the Hadamard matrix and Bernoulli distribution to generate cluster centers in advance, effectively ensuring a reasonable distribution of clustering centers of both original and incremental data. (2) To address the issue of unlabeled data, we propose a pseudo-label discriminator based on similarity probability to generate pseudo-labels for unlabeled data, which effectively improves the effect of semi-supervised training of the model. Extensive experimental results demonstrate the effectiveness of our proposed method, which can effectively address the concept drift issue under semi-supervised conditions.

The rapid development of deep neural networks has brought about significant risks of intellectual property infringement and misuse. We propose IrisMarkGN, an output image-based neural network watermarking model designed to protect the copyright of generative network (GN). However, high-capacity watermarking neural networks frequently face the challenge of balancing attack resistance with imperceptibility. We propose a network that utilizes dense connectivity to design a watermark embedder and extractor that are mutually matched. The network uses the iris information of the GN model creator as the key to generate the watermark. This approach enhances the watermark’s capacity while ensuring that it is embedded invisibly into the output image of the model. By inserting a noise simulator between the watermark embedder and extractor for joint training, our approach achieves strong resistance to attacks. Experimental results demonstrate that, compared with the currently proposed neural network watermarking methods, our proposed method, IrisMarkGN, is suitable for various GN models that output images. It invisibly embeds the watermark into the output image without significantly affecting the network’s main task. Moreover, it ensures high-capacity watermark embedding while resisting various image attacks. Our method achieves a peak signal-to-noise ratio value of over 45 dB with a structural similarity close to 1, and the watermark extraction accuracy is as high as 99.9%.

Zero-shot learning for image recognition recognizes the unseen classes by learning the transferable representations from seen to unseen classes in view of visuals or semantics. Recently, a few works have focused on learning the transferability of visual and semantic modals. They aim to conduct the adaptation between two modals to achieve more reliable transferability. However, their performances depend on the consistency of transferability in both modals. When the transferability of one modal is insufficient, the adaptation will lead to sparse and indistinguishable representations of unseen classes. To this end, a zero-shot method with visual-semantic mutual reinforcement is proposed, in which visual transferability and semantic transferability are reinforced mutually so that the transferable representations of two modals can complement each other, resulting in more discriminative transferable representations. Specifically, in visual transferability learning, local semantics are utilized to reinforce the key region representations, thus enriching the key regions and excluding the ambiguous regions in the image representations. In semantic transferability learning, visual class prototypes are used to reinforce the semantic representations with higher class discriminability. Finally, the image representations and attribute class prototypes for unseen classes are incorporated to recognize the unseen samples. Experimental results on multiple datasets show that the zero-shot and generalized zero-shot recognition of our method outperforms the state-of-the-art.

To enhance the speed and accuracy of insulator detection in unmanned aerial vehicle aerial photography, this study introduces a lightweight network, L-YOLOv8s, based on you only look once v8 (YOLOv8), designed for real-time detection of insulators and their defects. Initially, a lightweight backbone network, MobileNetv3-ECA-SPPF, is proposed. This network is capable of reducing the model parameter redundancy and improving the detection speed. The Slim-Neck module is employed to enhance the feature fusion component of YOLOv8s, thereby reducing the computational load and network complexity while maintaining model accuracy. For the precise recognition of small targets such as breakages and flash contamination, the wise intersection over the union v3 edge loss function is introduced to replace the original distance intersection over the union edge loss function, which is more beneficial for the model’s efficiency in recognizing small targets. Experimental results demonstrate that L-YOLOv8s can accurately and swiftly identify various types of insulators and their defects. The model’s accuracy reaches 94.8%, with a recognition speed of 454.55 frames per second. The number of model parameters is only 3.37M, and the floating-point operation is 7.2 Giga floating-point operations per second. Compared with the YOLOv8 model, the accuracy and floating-point operation of L-YOLOv8s are higher by 2.5%, the recognition speed is improved by 59.09%, the number of model parameters is reduced by 69.69%, and the floating-point operation is decreased by 74.65%. When compared with several traditional models, L-YOLOv8s proves to be practically significant in the identification of insulators and their defects.

Image classification is an essential component in the modern computer vision field, in which dictionary learning–based classification has garnered significant attention due to its robustness. Generally, most dictionary learning algorithms can be optimized through data augmentation and regularization techniques. In data augmentation research, focus is often placed on enhancing features of specific class samples, whereas the impact of shared information among images of different categories is overlooked. High inter-class shared information can make it challenging to differentiate among categories. To tackle this concern, the paper advocates an innovative data augmentation approach. The proposed method reduces excessive similarity within class samples by randomly replacing pixel values, thereby improving the classification performance. Building on this, we designed a joint dictionary learning algorithm that embeds label and local consistency. The basic steps of the proposed algorithm are as follows: (1) generate specific auxiliary samples as training samples, (2) initialize the dictionary and expression coefficients, (3) introduce label constraints and local constraints and update the dictionary, and (4) generate a classifier and classify the test samples. Extensive experiments have demonstrated the efficiency of the proposed approach.

Object detection in drone unmanned aerial vehicle (UAV) imagery is increasing. However, existing lightweight object detection models still face challenges in UAV aerial image object detection tasks due to the variable object scale and the existence of dense small objects. We propose a lightweight object detection model named YoloV8-RFCA. For small object features, the channel attention mechanism is integrated with receptive-field convolution to construct the Receptive-field conv with Channel Attention (RFCA) attention module, removing the parameter sharing issue and enhancing the feature extraction capability of the backbone network. Focusing on the feature information loss and degradation caused by multi-level transmission during the feature fusion operation, an asymptotic feature fusion strategy is proposed. Related experiment results indicate that the model achieved 82.5 mAP on the PASCAL VOC dataset and 41.9 mAP on the VisDrone2019 dataset. These experimental results confirm that our proposed model has a high practical application value in the field of UAV aerial image object detection.

Scene text image super-resolution aims to provide high-resolution and readable text images to support scene text recognition. Although existing methods based on deep learning have made significant progress, shallow information during image super-resolution is often ignored as the depth of the neural network increases. We propose a multi-level text feature enhancement super-resolution network (TESRN) to address this issue. TESRN adopts a coarse-to-fine feature extraction methodology, mainly including a shallow feature enhancement block (SFEB) and a multi-level feature fusion and extraction block (MFFEB). In SFEB, we design a framework based on wavelet transform for extracting coarse high-frequency signals. This framework works in parallel with convolution to accomplish shallow extraction. In MFFEB, we propose sequential group convolution blocks (SGCBs) based on group convolution and attention mechanism. Multi-level text features are generated step by step through stacking SGCBs. To comprehensively capture text features, we introduce a bottleneck attention mechanism (BAM) to execute feature selection in spatial and channel dimensions. BAM helps in selecting the most relevant features for text restoration. Finally, we conduct extensive experiments on the TextZoom dataset to evaluate the performance of TESRN. The results demonstrate that TESRN achieves high-quality image restoration and significantly improves the recognition accuracy of low-resolution text images in downstream text recognition tasks. Notably, our model outperforms existing methods in terms of recognition accuracy on the easy test subset. This further validates that TESRN effectively utilizes the shallow features of text images, emphasizing the crucial role of shallow features in text reconstruction.

Iris recognition is a reliable biometric identification method known for its low false-acceptance rates. However, capturing ideal iris images is often challenging and time-consuming, which can degrade the performance of iris recognition systems when using non-ideal images. Enhancing iris recognition performance for non-ideal images would expedite and make the process more flexible. Off-angle iris images are a common type of non-ideal iris images, and converting them to their frontal version is not as simple as making geometric transformations on the off-angle iris images. Due to challenging factors such as corneal refraction and limbus occlusion, frontal projection requires a more comprehensive approach. Pix2Pix generative adversarial networks (GANs), with their pairwise image conversion capability, provide the ideal foil for such a tailored approach. We demonstrate how Pix2Pix GANs can effectively be used for the problem of converting off-angle iris images to frontal iris images. We provide a comprehensive exploration of techniques using Pix2Pix GAN to enhance off-angle to frontal iris image transformation by introducing variations in the loss functions of Pix2Pix GAN for better capturing the iris textures and the low contrast, changing the medium of input from normalized iris to iris codes, and ultimately delving deeper into studying which regions of the Gabor filters contribute the most to iris recognition performance.

Thermal infrared imaging has great potential for applications in auto-vehicle, human-machine interactive, surveillance, and defense systems owing to its performance in low-light and low-visible conditions. Nevertheless, the shortage of published datasets and benchmarks in the infrared domain has hampered the development of contemporary research. To tackle this problem, we propose a method to produce synthetic thermal infrared (TIR) images using a diffusion-based image-to-image translation model to enrich the data for learning. The model translates the abundantly available Red, Green, Blue (RGB) images to synthetic TIR data closer to the domain of authentic TIR images. For this purpose, we explore the usage of an unpaired image translation neural model based on Schrödinger bridge algorithms. In addition, the visual characteristic of the object in the image is an important consideration in generating results. Thus, we take advantage of a segmentation module before the image-to-image translation model to discriminate the background and object regions. We practice the model’s performance with a self-proposed dataset comprising unpairs realistic RGB-TIR images. We evaluated the model’s performance in synthesizing thermal images by comparing them to our original thermal dataset, achieving a Fréchet inception distance score of ∼80, indicative of high-quality image generation. In particular, the synthesized images of our model increase the classification accuracy by 15% when only realistic TIR images are used.

Due to the influence of domain bias, domain generalization person re-identification models are not capable of generalizing well on unseen domains. The style factor is a critical factor that causes domain bias. To address the task, we propose a style-unaware meta-learning method that is less affected by domain migration. Specifically, we design a style fusion plugin that changes the specific source domain’s style and simulates more diverse domain differences. We add generated style factors to the input images to enhance the model’s generalization performance in unknown domains. This leads the model to pay more attention to the content of inputs and ignore the style changes as much as possible. To maximize the benefits of the model, we particularly combine our modules with a meta-learning algorithm. Moreover, we design a pretext task, the process of sifting samples, which is fundamental to different domains and can be utilized for other domains to learn domain-invariant features, improving the generalization ability of the model. In our model, in addition to those mentioned above, we set a trimming function that builds and fine-tunes the feature space we construct.

Depth information is crucial in unsupervised video object segmentation (UVOS) and has been shown to be beneficial in contemporary RGB-D saliency object detection approaches. Nonetheless, current UVOS techniques frequently overly depend on spatiotemporal cues while disregarding valuable depth information. Such overreliance on motion cues can result in performance degradation under conditions of inadequate optical flow. We introduce a depth-informed cross-modal three-stream network (DIMF-Net) for UVOS, representing pioneering efforts to incorporate depth information for UVOS assistance. To achieve this, we introduce a gate fusion module to integrate depth features, thereby extending the conventional two-stream network. In addition, we acknowledge that data from various sensing modalities are frequently complementary but can also contain noisy measurements. To mitigate this, we leverage features from another modality to filter and calibrate noisy information. These improvements substantially enhance the performance of UVOS, showcasing exceptional accuracy and speed in real-world scenarios. DIMF-Net achieves top-tier performance on the DAVIS, FBMS, and YouTube-Objects datasets.

As the size of vision models grows rapidly, the pre-training-then-fine-tuning paradigm becomes prohibitively expensive for model training and the dataset size required during the transfer learning stage. Inspired by the idea of a parameter-efficient transfer learning method in the natural language processing (NLP) domain, we proposed a lightweight, plug-and-play and trainable adapter (FCT-Adapter) to seamlessly fuse two heterogeneous networks, i.e., convolutional neural network (CNN) and Vision Transformer (ViT), and efficiently adapt during the transfer learning. The proposed FCT-Adapter serves to bridge CNN and ViT without adjusting the pre-trained backbone structure and parameters. Using only 0.75 million parameters for 19 tasks, the experimental results on the Visual Task Adaptation Benchmark show that the fused model Resnet-50 and ViT-B/16 via FCT-Adapters achieves an average accuracy of 77.4%, which is 12.3% higher than that by a single backbone through full fine-tuning. As the adapter contains a small number (less than 1% backbone parameters) of trainable parameters, the proposed method can achieve a plug-and-play function and be quickly deployed for various visual applications at low data volumes and storage costs.

Significant bodies of research have explored the topics of computer vision and image quality, but research on the intersection of these two disciplines remains limited. In addition, evidence suggests that image quality as determined by the human visual system may differ from image quality as determined by the performance of computer vision algorithms. Furthermore, most research on the relationships between image quality and computer vision performance has focused on single-label image classification and has not considered tasks such as semantic segmentation or object detection. We consider the relationship between three primary image quality factors—resolution, blur, and noise—and the performance of deep-learning-based object detection models. To do so, we examine the impacts of these image quality variables on the mean average precision (mAP) of object detection models, evaluating the performance of models trained on only high-quality images as well as models fine-tuned on lower-quality images. In addition, we map our primary image quality variables to the terms used in the general image quality equation—namely ground sample distance, relative edge response, and signal-to-noise ratio—and assess the suitability of the general image quality equation functional form for modeling object detector performance in the presence of significant image distortions.

Color quantization (CQ) is a fixed-rate vector quantization developed for color images to reduce their number of distinct colors while keeping the resulting distortion to a minimum. Various clustering algorithms have been adapted to the CQ problem over the past 40 years. Among these, hierarchical algorithms are generally more efficient (i.e., faster), whereas partitional ones are more effective (in minimizing distortion). Among the partitional algorithms, the effectiveness and efficiency of the Lloyd (or batch) k-means algorithm have been shown by multiple recent studies. We investigate an alternative, lesser-known k-means algorithm proposed by Jancey, which differs from Lloyd k-means (LKM) in the way it updates the cluster centers at the end of each iteration. To obtain a competitive color quantizer, we develop a weighted variant of Jancey k-means (JKM) and then accelerate the weighted algorithm using the triangle inequality. Through extensive experiments on 100 color images, we demonstrate that, with the proposed modifications, JKM outperforms LKM significantly in terms of efficiency without sacrificing effectiveness. In addition, the proposed JKM-based color quantizer is as straightforward to implement as the popular LKM color quantizer.

The study and analysis of underwater image enhancement are significant in marine engineering and aquatic robotics. The data-driven approach shows good performance in this field. However, the method faces challenges in addressing issues such as low contrast, blurring, and color deviation. Moreover, its performance is constrained by the availability of paired underwater images, making it challenging to capture the nuances among different underwater scenes. To address these challenges, we introduce a semantic attention-guided transfer learning method for stylization underwater image enhancement (SGTL-SUIE). This method enables the generation of multiple stylized and enhanced images from a single distorted underwater image. Within SGTL-SUIE, a style-filtering module is proposed to better bridge the domain gap between distorted images and style reference images. Subsequently, a semantic pairing module further mitigates the domain differences across varying semantics between reference and distorted images, guided by semantic information, producing multiple semantic pairing codes. The transfer enhancement module then takes these semantic pairing codes, co-encoding style features with distorted image features through an encoder. A decoder network subsequently decodes the encoded features into diverse stylized outputs. To validate the proposed method’s performance, qualitative and quantitative evaluations were conducted on multiple public datasets. The results demonstrate that the SGTL-SUIE method outperforms many state-of-the-art approaches and enhances the stylistic diversity of generated images.

The proliferation of the Internet of Things (IoT) has introduced a new paradigm of interconnected devices, enabling advanced functionalities across various domains such as healthcare, smart cities, and industrial automation. However, this rapid expansion has also led to significant security challenges, particularly concerning the authentication of devices within resource-constrained environments. Traditional security mechanisms are often too resource-intensive for IoT applications, necessitating the development of lightweight yet robust solutions. This paper presents a novel lightweight authentication protocol designed specifically for IoT environments. The proposed scheme integrates perceptual hashing with multimodal biometrics, enhancing security by leveraging the unique physiological characteristics of multiple biometric traits. In addition, the protocol incorporates elliptic curve cryptography to provide strong cryptographic guarantees with minimal computational overhead, making it suitable for devices with limited resources. We provide a formal security analysis of the proposed protocol using Burrows-Abadi-Needham logic, demonstrating that it achieves mutual authentication and resistance to common attacks such as replay and impersonation. The protocol’s security is further validated using the Scyther tool, which confirms its resilience against a wide range of potential threats. Our scheme not only ensures secure communication within IoT systems but also maintains the efficiency required for practical deployment in resource-constrained environments.

Deformable medical image registration is an important component of medical image analysis. Transformer networks have recently received more attention due to their ability to improve image registration by utilizing long-range spatial relationships. However, these approaches often struggle to accurately align images compromised by artifacts, typically caused by patients’ organ movement during scans across various diagnostic devices. To solve this challenge, this study introduces a triple-stream architecture that enhances organ alignment accuracy while maintaining the anatomical structure. Our proposed approach incorporates a cascade-based cross-attention technique within a Tri-UNet framework, allowing for the continuous integration of moving, fixed, and new moving images. The technique addresses the optimal alignment and extensive motion artifacts often encountered in clinical imaging. To ensure that the anatomical structure remains maintained during deformations, we optimized the framework by integrating motion correction techniques that generated the final deformation ϕ (DVF) by the maintaining anatomical structure from the multi-scale deformation of two primary sources: moving-fixed and new moving-fixed. This technique significantly aligns image pairs and reduces motion artifacts from deformation fields, resulting in stable and accurate deformation. The results show that the proposed methods achieve state-of-the-art performance compared with other learning-based methods. Our proposed method achieved an average Dice similarity coefficient of 0.787 and 0.726 on Open Access Series of Imaging Studies (OASIS) and LONI Probabilistic Brain Atlas (LPBA40), respectively.

Classical Chinese poetry and painting represent the epitome of artistic expression, but the abstract and symbolic nature of their relationship poses a significant challenge for computational translation. Most existing methods rely on large-scale paired datasets, which are scarce in this domain. We propose a semi-supervised approach using cycle-consistent adversarial networks to leverage the limited paired data and large unpaired corpus of poems and paintings. The key insight is to learn bidirectional mappings that enforce semantic alignment between the visual and textual modalities. We introduce novel evaluation metrics to assess the quality, diversity, and consistency of the generated poems and paintings. Extensive experiments are conducted on a new Chinese Painting Description Dataset. The proposed model outperforms previous methods, showing promise in capturing the symbolic essence of artistic expression.

Traditional facial recognition techniques often struggle to balance accuracy with model complexity. High accuracy typically demands intricate models, slowing recognition speeds on devices such as smartphones. Conversely, faster methods often sacrifice accuracy. We introduce a lightweight deep convolutional generative adversarial network (LW-DCGAN), designed specifically to address the challenges of occluded face recognition. By simplifying the network architecture and employing efficient feature extraction techniques such as transpose convolution, batch normalization, feature pyramid networks, and attention modules, we enhance both hierarchical sampling and contextual relevance. Furthermore, L1 regularization and channel sparsity techniques compress the model for resource-constrained environments. We thoroughly evaluate LW-DCGAN’s generalization and robustness, comparing its performance against other generative adversarial network variants and common face recognition models. The results demonstrate that LW-DCGAN achieves higher accuracy while significantly reducing model size and computational overhead, offering a promising advancement in lightweight face recognition technology.

Multi-view clustering is a prominent area of interest in machine learning and data mining. However, most existing methods are confined to static multi-view data, posing challenges for achieving multi-view information fusion and clustering in a dynamic streaming context. We propose a multi-view streaming clustering with incremental anchor learning, which effectively partitions continuous chunks of multi-view data into meaningful clusters. Initially, a shared subspace representation is derived to reveal the intrinsic structure hidden across views, which is adapted to the evolving data distribution through incremental learning of anchors. Furthermore, the shared subspace representation, anchors, and clustering assignments are learned simultaneously in a unified framework, where their interactive negotiation avoids the suboptimal solution problem and significantly enhances overall clustering performance. Finally, extensive experiments on several real-world datasets demonstrate that the proposed method achieves superior multi-view clustering performance and efficiency in a streaming context.

The rapid development of computer vision and deep learning has significantly advanced image aesthetic assessment, yet traditional methods, which primarily rely on low-level visual features such as color and texture, often struggle with the complexity of graphic design images. These images are characterized by diverse design elements, including color, typography, and layout, as well as various styles such as minimalism, retro, and modernism, presenting substantial challenges to conventional assessment techniques. To overcome these limitations, we propose an innovative multimodal learning approach that integrates image content with textual descriptions to comprehensively analyze the aesthetic qualities of graphic design images. The core innovation of our method lies in the utilization of two distinct textual description methodologies: holistic descriptions, which capture the main theme of the design, and detailed descriptions, which focus on specific aspects such as composition, color, detail, and atmosphere. This dual approach allows for a more nuanced and complete assessment of aesthetic value. To effectively merge these descriptions with visual content, we introduce a feature similarity blending mechanism that aligns and integrates features from both modalities, enhancing the representation of aesthetic attributes. In addition, we employ a score bagging technique to aggregate scores from multiple fused features, ensuring robustness and reliability in the assessments. Our method is implemented within a multi-task learning framework, enabling simultaneous prediction across multiple rating dimensions. Experimental results demonstrate that, compared with the state-of-the-art TAHF method, our approach achieves notable improvements in Spearman’s rank correlation coefficient—by 1.7%, 3.4%, and 2.6% on the HDDI, BAID, and TAD66K datasets, respectively—along with consistent gains in Pearson’s linear correlation coefficient and accuracy. Moreover, our method achieves these performance improvements with fewer parameters and lower computational complexity, highlighting its efficiency and effectiveness in graphic design image aesthetic assessment.

The pursuit of high-quality remote sensing images never ceases. In the field of remote sensing image processing, addressing image denoising and deblurring is of paramount importance. To reconstruct original images from degraded ones, we propose adaptive fourth-order partial differential equation (PDE) models for denoising and denoising–deblurring based on a variational framework. The proposed PDE models include second-order, third-order, and fourth-order terms. The models fully utilize the advantages of second-order and third-order terms in preserving boundaries and the fourth-order term in denoising and avoiding the stair-step effect. By adaptively adjusting the coefficients of the second-order, third-order, and fourth-order terms during the denoising and denoising–deblurring processes, the models achieve a good balance between denoising and boundary preservation. In the homogeneous regions of the images, the fourth-order term dominates and performs well at removing noise, whereas in the edge regions, the slow diffusion of the second-order term and the reverse diffusion of the third-order term dominate, resulting in good boundary preservation. We discretize the models in time and space using the Rothe method and finite difference method, respectively. Numerical experiments demonstrate significant improvements over traditional methods. When the noise standard deviation is 20, the denoising model achieves an ∼2 dB increase in peak signal-to-noise ratio compared with the total variation method.

YOLOv5 is a one-stage detector that achieves appealing performance in real-time object detection. When it comes to remote sensing object detection, there are many small objects in the scene, and the number of objects in different categories varies significantly. Directly applying YOLOv5 for remote sensing object detection usually ignores these small objects. Furthermore, imbalance among different categories also makes the model prone to some majority categories. In this way, we propose a smallness and imbalance-aware head and apply it to YOLOv5. The improved model is named SIA-YOLOv5. To be specific, a normalized Gaussian Wasserstein distance is designed to replace the commonly used intersection over union in the regression process, which substantially improves the localization accuracy for small objects. Meanwhile, an adaptive weighting strategy is designed to make a flexible emphasis on the classification accuracy among different categories, which relieves the unstable performance caused by imbalanced data. In addition, BiFPN and coordinate attention are utilized for better feature extraction. Experimental data and analysis have demonstrated the effectiveness of the proposed method.

Matrix completion (MC) has been successfully applied to many real-world applications, such as recommender systems and image inpainting. Traditional MC methods mainly aim at minimizing the rank function or its surrogate functions. However, the resulting algorithms inevitably calculate the singular value decomposition (SVD) at each iteration, which not only consumes a great deal of time but also greatly limits the potential applications in large-scale data. We propose an MC method that is based on smooth matrix factorization (MF) to deal with the MC problem in the absence of noise. Our method not only inherits the advantage of the MC methods, which are based on the MF and thus avoid calculating the SVD, but also can be easily embedded into many other MF-based MC methods, which makes our method more scalable and flexible. More importantly, different from the existing MF-based MC methods, our method imposes smoothness constraints on each of the factor matrices to help select the factor matrices of high quality, directly leading to the smooth MF, which largely improves the MC performance in both the recovery accuracy and the recovery speed. Two kinds of alternating minimization algorithms are also put forward to numerically solve the proposed model, and extensive experiments further confirm the superior performance of the proposed method over many state-of-the-art methods.

Grain is one of the basic human necessities, and its quality and safety directly impact human dietary health. Various issues occur during grain storage, primarily mold and pest infestation. With the development of artificial intelligence, increasingly, more technologies are applied to grain detection and classification. Transformer-based models are becoming popular in grain detection. Although transformer models exhibit excellent performance, they are often large and cumbersome, limiting practical applications. We propose a framework named KD-ASF based on intermediate layer knowledge distillation and one-shot neural architecture search, to optimize the hyperparameters of vision transformer (ViT) for detecting and classifying molded wheat kernels (MDK), Insect-Damaged wheat kernels (IDK), and undamaged wheat kernels (UDK). In KD-ASF, we use the ViT model as our teacher network. Next, we design a search space containing adjustable hyperparameters of transformer building blocks. The super-network stacks maximum transformer building blocks and is trained under the guidance of the teacher network. Subsequently, the trained super-network undergoes evolutionary search, and the resulting networks are used for classifying different wheat kernels. We conducted experiments using a five-fold cross-validation approach and obtained an F1 score of 97.13%, and the last model parameter size is only 5.94M. The results demonstrate that this method not only outperforms the majority of neural networks in terms of performance but also has a significantly smaller model size than most network models. Its lightweight nature facilitates easy deployment and application. These findings indicate that the structure of KD-ASF is feasible and effective.