The computational spectral imaging technology can increase imaging rates without sacrificing image quality. Because the imaging speed and the imaging spectrum of single-pixel spectral imaging technology is fast and wide, it is a promising imaging technology in the future. Here, we proposed a single-pixel spectral imaging system based on sorting coding and depth learning. We used a Hadamard matrix based on Cake-cutting sorting to reduce the sampling rate and consequently to get higher image reconstruction contributions. Meanwhile the Fourier Frequency Domain Regularization and Spectral self-attention Transformer was used to improve the image quality. A 16-channel spectral image is simulated, and the visual quality and spectral curve of the reconstructed image are analyzed. The simulation results show that compared to the traditional system, the new system has improved the speed and quality of imaging. The imaging speed is about 30 times faster than conventional systems.

Computational spectral imaging is an emerging interdisciplinary technique with extensive applications. Current systems based on meta-surfaces use broad-band nanophotonic encoders for spectral encoding, which requires a high sampling rate to achieve ideal spectral resolution. In this study, we propose a high-resolution computational spectral imaging system based on quasi-bound states in the continuum (quasi-BIC) achieved by breaking the C2 symmetry of a metarectangle structure. Compared to broad-band nanophotonic encoders, quasi-BIC features a high-quality factor (Q-factor), with resonance peaks that can be shifted across a wide wavelength range by adjusting a scale factor S, and the width of these peaks can be controlled by rotating orientation angles. This quasi-BIC spectral encoder enhances the accuracy of computational reconstruction and improves spectral resolution, achieving an average peak-signal-to-noise ratio (PSNR) of 35.12 dB and an average spectral angle mapper (SAM) of 0.0616 at a sampling rate of 29%. This advancement paves the way for the development of compact and high-resolution spectral imaging devices.

A single Light field (LF) imaging system tends to have a relatively small field of view (FoV). LF stitching can expand the imaging range and obtain more scene information. This is especially valuable for computer vision applications that require global environmental perception. The existing methods of using 2D feature points to perform sub-aperture image (SAI) matching to achieve LF stitching destroy the spatial and angular consistency of the stitched LF because it does not consider the connection between views. In this paper, a novel LF stitching method is proposed, the innovation of this method is to use LF feature points to replace the 2D feature points. The LF feature points we choose are the Harris feature points detected in the scale-disparity space established using fourier disparity layers (FDL) with circular gradient histogram descriptor. The selected LF feature points contain the coordinate information of feature points on the central SAI, scale information, and disparity information. Based on the disparity values and feature points on the central SAI, the feature points on each SAI can be obtained through disparity transformation and color difference threshold. For each pair of SAIs, the stitching method based on line-guided warping and line-point constraint is then applied to ensure correct alignment of the image content. Experimental results show that using LF feature points can effectively stitch 4D LFs, and preserve the spatial and angular consistency of the stitched LF compared with directly using the image stitching corresponding SAIs based on 2D feature points.

Hybrid cameras, which include a high-resolution camera surrounded by multiple low-resolution cameras, offer advantages over light-field cameras by avoiding trade-offs between spatial and angular resolution. However, reconstructing light field from hybrid cameras data requires overcoming the dual challenge of enhancing both spatial and angular resolution. Challenges include matching images from cameras with different resolutions and creating a trainable real capture dataset for hybrid cameras data. This study proposes a parametric representation of the neural light field with decoupled light field data, based on the spatial-angular consistency of the light field. The proposed method decomposes the light field into angle and color information, parametrically represents these two components using coordinate-based neural network, and builds neural disparity field and neural central view field. Using the disparity propagation equation, we connect two modules in series, forming a differentiable network architecture. To separate angle and color information in the light field, we match discrete camera images with continuous images expressed by neural central view field using a two-step training strategy. Initially, neural central view field is trained, followed by matching discrete camera images with continuous images expressed by neural central view field to train neural disparity field. Experimental results demonstrate that proposed method effectively utilizes the spatial and angular information of hybrid cameras data, resulting in high-quality light field reconstruction.

In recent years, domestic infrared focal plane array detectors have made comprehensive progress, but image-crosstalk remains a major unresolved issue. Research on infrared image-crosstalk correction has mostly focused on the detector fabrication, and methods for correcting image-crosstalk through image processing are relatively few. We address the crosstalk phenomenon commonly occurring in domestic infrared focal plane array detectors, analyze the crosstalk generation mechanism at the readout circuit level, and establish an RC system crosstalk model. Given this foundation, we further propose an image-crosstalk correction method for infrared focal plane array detector based on RC model, which can directly correct crosstalk images through image processing. Firstly, we simulate the degradation and restoration of a step function using the proposed crosstalk correction principles, thereby validating the effectiveness of the method conceptually. Subsequently, the proposed method is applied to correct images of rod targets of three different sizes that are actually collected, with both qualitative and quantitative analysis demonstrating the effectiveness of our imagecrosstalk correction method. The study shows that this method provides a simple and practical image processing solution for the crosstalk problem in domestic analog infrared focal plane array detectors.

Phacoemulsification with Toric intraocular lens (IOL) implantation is the main surgery for curing cataracts with astigmatism. During this surgery, accurate astigmatic axis alignment of the toric IOL is crucial for the postoperative astigmatism correction effect. Automatic cyclotorsion identification through computer-assisted navigation can help guide doctors to quickly and accurately align the axis without markers. However, navigation can be easily hampered by illumination variation, instrument intervention, and inconspicuous biomarkers during surgery. Moreover, existing advanced methods are difficult to balance alignment accuracy and real-time performance. To address these challenges, we propose a toric IOL navigation framework based on learnable keypoint matching. The framework detects dense and reliable keypoint features in frames through a self-supervised lightweight neural network, and then uses semantic filtering and keypoint matching to complete the inter-frame cyclotorsion alignment estimation. During navigation, the keyframe is updated and inserted in real-time according to the matching quality to overcome accumulated alignment errors and potential abnormal matching. The framework was tested using a self-developed surgical dataset. Test results show that compared with other advanced methods, our framework achieves the lowest alignment error (0.71°) on a lowconfiguration computer, with a processing speed of 26.3 FPS, and can adapt to non-ideal surgical scenarios such as strong intervention, low texture, and weak illumination.

Implicit Neural Representation (INR) achieves the mapping between coordinates and signal values through multilayer perceptrons (MLPs). However, coordinate-based MLPs struggle with high-frequency information due to spectral bias. A common solution to the spectral bias problem is to use Fourier features with fixed encoding frequencies instead of spatial coordinates as inputs to the MLPs. Natural scenes exhibit distinct frequency spectra across local areas, with the main regions characterized by low frequencies and the edges by high frequencies. The fixed encoding frequency Fourier feature will lead to redundant input information and reduce the running speed of MLPs. Therefore, this paper proposes an Implicit Neural Representation with Adaptive Learnable Filters framework (NRALF), which is controlled by a learnable parameter to achieve low-pass, band-pass, high-pass, all-pass and all-no-pass functions. This filter filters the Fourier features of different frequencies, making the filtered coding space more sparse, thereby enhancing the expression ability of MLPs for natural scenes and significantly improving the convergence speed of MLPs. Experimental results show that the proposed method has higher accuracy and faster convergence speed in scene representation.

Compared to traditional identification technology, biometric technology has attracted a lot of attention because it highlights the special physiological characteristics and behavioural patterns of the human body. Among many biometric technologies, finger vein recognition technology is highly resistant to forgery, reliable, and unaffected by changes in skin surface conditions. In this study, a specially designed 128-ring array fast photoacoustic imaging system is used to accurately capture finger vein, muscle and skin images for information acquisition. The image segmentation algorithm in deep learning was used to remove the noise from the images, and then using the uniqueness of finger veins and muscle structures between individuals, the features of each image were automatically extracted using a 2D convolutional neural network to produce the classification results for each image, which were then ensemble to produce the identification results for each subject.

Low-light remote sensing technology is crucial for surface observation during twilight and lunar phases; however, the acquired images often suffer from low contrast, low brightness, and low signal-to-noise ratios, which adversely affect observation quality. Traditional low-light image enhancement algorithms, such as Histogram Equalization, Gamma Correction, and Adaptive Histogram Equalization, can improve visual outcomes but also suffer from issues such as over-enhancement, loss of detail, noise amplification, and insufficient adaptability. To address these limitations, this paper proposes a low-light remote sensing image enhancement method based on Zero-Reference Deep Curve Estimation (Zero-DCE). This approach does not require paired samples and guides network learning through a non-reference loss function, making it particularly suitable for enhancing remote sensing images in low-light environments. Due to the lack of dedicated low-light remote sensing datasets, this study utilizes images from the UCMerced dataset to create simulated low-light remote sensing images for model fine-tuning. All color images are converted to grayscale to align with the characteristics of satellite-based low-light remote sensing images and to simplify the training process. Experimental results demonstrate that the proposed method significantly outperforms traditional techniques in terms of Structural Similarity Index (SSIM) and Peak Signal-to-Noise Ratio (PSNR), while also excelling in denoising and preserving texture authenticity. The optimized Zero-DCE++ not only maintains the original performance but also significantly reduces computational costs and enhances inference speed, which is of great importance for real-time low-light remote sensing image processing on satellite platforms.

Deep learning-driven structured light 3D measurement has garnered significant attention due to the fast speed, high precision and non-contact characteristic. However, the accurate prediction of edge discontinuity area is still one of the challenges. In single-frame end-to-end absolute phase prediction task, we initially proposed a mask semantic attention network (MSAN) to enhance the edge and whole accuracy. Firstly, mask serves to partition the scene into its background (shadow) and foreground (objects) elements, and it provides semantic attention for the network. Secondly, we designed a mask fusion (MF) module which can effectively integrates feature maps with mask semantics. Based on the MF module and mask semantic information, we developed a U-shaped network architecture, and each layer feature map of the decoder is fused with the input mask adopting the MF module. MSAN improves edge prediction accuracy by explicitly identifying edge regions and drawing the network's attention to the edges and objects rather than shadow areas, enhancing overall prediction accuracy. Validation on real datasets showed that the mean absolute error decreased by 33% and the root mean square error decreased by 76% with MSAN, demonstrating the network's capability to improve both overall and edge precision in structured light deep learning tasks. This advancement significantly benefits the development of high precise and rapid structured light 3D measurement technologies.

Two issues that hindering fast developments of diffractive waveguides based on surface relief grating (SRG) for augmented reality (AR) applications: light leakage at out-coupler and low light efficiency. A layer of subwavelength metal wire grid on commonly used protective glass sheet above diffractive waveguide was designed to prohibit the light leakage and reflect them back to AR waveguide for recycling light usage. One-dimensional (1D) SRG based AR waveguide was analyzed under TE-polarized illumination at 532nm wavelength with 30° field-of-view (FOV). It was demonstrated that 80% efficiency improvement and 9% intensity uniformity increasement could be obtained with completely prohibited light leakage by nanostructures glass cover, compared with that on planar protective glass.

A spherical coded imaging system combined with a controlled spherical aberration lens system and a digital sharpening filter can realize a fast and low-cost extended depth of field (EDoF) imaging system. At the same time, the wavefront coding technology is introduced, which can not only extend the depth of focus of the system, but also suppress the aberration including spherical aberration in the system design. However, for the wavefront coding system, due to the modulation of the incident light wave, the light distribution is more diffuse, so the blurred image generated by the wavefront coding system is a blurred image. It is necessary to decode and restore the intermediate blurred image to obtain a clear target image. In view of the lack of convergence and reliability of IBD algorithm, the Richardson-Lucy(RL) algorithm is introduced into RL-IBD algorithm, which can effectively reduce the sensitivity of the algorithm to noise. On the basis of vector extrapolation and exponential correction, this paper proposes improvements to the RL-IBD algorithm, which enhances the stability of the algorithm, and improves the convergence speed, noise suppression ability and adaptability of the algorithm.

Point cloud denoising is a crucial step in the processing of 3D data. Although model-based methods for point cloud denoising have seen some success, their performance often remains inconsistent due to the complex parameter selection required for objects with varying shapes. To address this challenge, we propose a new method called TPDn, which takes full advantage of the object geometry to select parameters automatically. After converting point clouds into a triangular mesh, TPDn defines two key textures: the normalized mesh size and the mesh normal. By fusing these two texture features, a global distribution function is established. TPDn adaptively determines the appropriate threshold by deriving it from a simple approximation function.

The task of reconstructing real-world objects arises in many areas of human life. Inverse rendering methods show high efficiency in solving this problem. For accurate reconstruction of object geometry from a set of images, we propose a geometry reconstruction method based on differentiable rendering. This method consists of two steps: forming a bounding volume and optimizing the geometry. To achieve high-speed geometry reconstruction, we suggest reducing the time spent on optimizing the silhouette gradients by conducting an initial stage of constructing the bounding volume of the object. The article considers the possibility of using three-dimensional visual hulls as a primary representation for optimizing geometry using differentiable rendering methods. Introducing this stage of geometry reconstruction allows the inverse rendering to start with the condition of matching the reconstructed object's contour and topology. Thus, geometry optimization occurs considering shading gradients, eliminating the need to account for object silhouette gradients. The proposed approach solves the problem related to potential distortion of the object's geometry, such as normal inversion or self-intersecting triangles. The paper presents the results of applying the proposed method to reconstruct test objects from image sets and demonstrates the increase in accuracy of geometry reconstruction.

3D Gaussian Splatting has marked a significant milestone in novel view synthesis, particularly in the real-time level rendering speed. Despite its remarkable contributions, this technique encounters substantial challenges when reconstructing dynamic scenes due to motion blur, which leads to inaccurate point clouds and low-resolution vision performance. To address this issue, this paper proposes an event camera assisting 3D Gaussian Splatting generation, which provides high-accuracy vision cues for reliable motion description. Because moving objects attract more attention, the entire scene is rendered at different precision scales. Specifically, the proposed solution provides a detailed rendering of the space captured by the event camera and reduces its rendering accuracy for static backgrounds. Our comparative experiments demonstrate that the adoption of event camera imagery significantly improves the fidelity of the novel view synthesis, thereby enhancing the overall reconstruction quality in dynamic scenes.

Digital twin technology has been widely used in modern industrial and scientific research fields, and its application scenarios are increasingly diversified. However, the current applications have not yet effectively combined stable and efficient rendering methods with flexible and complete interactive functions. In this paper, we investigate the application of 3D visualization technology based on Three.js and Quasar framework in digital twin system, in order to find an effective merging method. Firstly, this study uses a optical 3D scanner to capture high-precision 3D model data in OBJ format, which provides a real and accurate data base for the digital twin system. Secondly, within the Quasar framework, the model was efficiently reconstructed and rendered in 3D using the Three.js library to achieve a smooth interactive experience and realistic visual effects. Detailed tests on the response time, rendering speed and stability of the visualization system were conducted to analyse the performance, and the test results showed that the system can ensure the smoothness and real-time performance of the user in the process of handling large and complex models. In addition, the interaction function of the system is optimised and tested, which enables users to observe and operate the model more conveniently by providing multiple viewpoint switching and detail zoom functions. Finally, the experimental results indicate that this study demonstrates the strong potential and application value of 3D visualization technology in the field of digital twin, which is of great significance in promoting the application of digital twin technology in various industries.

With the development of space satellite remote sensing technology, the demand of high-resolution imaging is increasing day by day. The aim of this paper is to explore a new kind of optical imaging, high-resolution rotating pupil optical imaging. By introducing rectangular rotating pupil and super-resolution reconstruction algorithm into the imaging system, this technology can significantly improve the resolution of the system, reduce aberrations and improve the image quality. Firstly, the theoretical basis of rotating pupil optical imaging is analyzed, including the effect of pupil rotation on the performance of the imaging system. The degradation images with different rotation angles are obtained, and the degradation mechanism and characteristics are analyzed. Then, a set of simulation model and method for image quality improvement of rotating pupil optical imaging system are designed and constructed. The final high-resolution image is obtained by the super-resolution reconstruction algorithm, and the theoretical analysis and performance index of this technology are verified and tested. Compared with traditional active super-resolution imaging system, rotating pupil optical imaging adopts passive imaging mode, which can significantly improve imaging resolution and reduce aberrations, especially in dynamic imaging and long-distance imaging. Finally, the future development trend of rotating pupil optical imaging is prospected, and the possible improvement direction and further research suggestions are put forward. This technology is expected to play an important role in the future field of space optical imaging, providing new solutions for high-resolution imaging.

Mixed reality systems that integrate the real and digital worlds have recently gained popularity. The development of mixed reality devices in the form of wearable devices has been a significant trend in recent years. Key challenges in designing wearable mixed reality devices include reducing power consumption, enhancing productivity, and minimizing the size of the device. To address these challenges, neural processors are being incorporated into mixed reality systems. These are specialized hardware accelerators for convolutional neural network (CNN) processing. The use of neural processors is due to the fact that most environmental analysis and visualizations in mixed reality rely on CNNs. The use of these energy-efficient, high-performance processing devices enables increased comfort in using the wearable device through the high-performance processing of environmental data on a compact device.

The paper is devoted to the design of the microarchitecture of a neural processor for hardware acceleration of CNN processing based on the author's architecture of processor. The paper presents various microarchitectural solutions that can be used to accelerate CNN processing. We explore methods to optimize hardware resources and reduce time required for CNN processing. To achieve high throughput in pipelined computation, different algorithms for convolution calculations in a systolic array are examined. Based on the results of this research, we provide estimates of the characteristics of the neural processor with the proposed microarchitecture.

This paper presents a computer model of an optically transparent augmented reality system built using a configuration with two diffraction elements. A distinctive feature of this device is that the second diffraction element is a two-dimensional diffraction grating that combines functions of beam expansion and image extraction from the light guide. The technology for manufacturing two-dimensional diffraction gratings is presented. Formulas for calculating the direction of beams diffracted by the diffraction gratings, including the two-dimensional grating, are proposed. Two ray-tracing algorithms for image formation by optical augmented reality systems containing diffraction gratings, including two-dimensional ones, are proposed. Based on these algorithms, virtual prototyping of several augmented reality systems was conducted. Design features, advantages, and disadvantages of these systems were identified. Recommendations for the design and optimization of augmented reality systems using a dual diffraction element configuration are provided.

Under low-light conditions, existing RGB frame-based fall detection methods suffer from a significant decline in accuracy. To address this challenge, this paper proposes an innovative fall detection approach that exclusively utilizes RGB frames while incorporating adaptive image enhancement to improve performance. The proposed method leverages the Deep Deterministic Policy Gradient (DDPG) algorithm to estimate illumination conditions in the captured frames. By learning illumination characteristics, the DDPG algorithm accurately predicts parameters for image enhancement. Advanced techniques are then applied to adjust brightness and contrast, producing high-quality visuals even in dim environments. The effectiveness of this approach is validated using the YOLOv5 object detection algorithm to detect falls in both the original low-light images and their enhanced counterparts. Experimental results show that the proposed method significantly outperforms baseline approaches in low-light settings while maintaining real-time performance and robustness, offering a promising solution for fall detection.

In recent years, end-to-end depth map prediction from single-shot fringe modulation images in structured light 3D measurement(F2D) has drawn widespread attention, which has significantly reduced measurement times and eliminated the complex intermediate steps in the traditional method. However, F2D is a long-distance ill-conditioned prediction problem, and it is difficult for existing regression networks to achieve high-precision pixel-by-pixel prediction over long distances in space and time. For the challenge, we propose APS-UNet(Absolute Phase aided Supervision UNet), an endto- end depth maps prediction network supervised by an absolute phase branch. With the core physical process, absolute phase branch as auxiliary supervision, can decompose the one challenging long-distance prediction into two easier shortdistance prediction tasks. Moreover, in the training process, the two branches provide feedback to each other, enhancing the accuracy and robustness of depth prediction. Compared to Res-UNet, APS-UNet demonstrates a 32% decrease in mean absolute error (MAE) based on the real dataset, highlighting the effectiveness of this network.

Single-shot structured light depth prediction based on deep learning has wide application demands in intelligent manufacturing and defect detection. In the depth prediction area, existing CNN has poor long-range global modeling capabilities, and Transformer architecture requires large computational resources. For the challenge, we first propose and adopt DPM-UNet (Depth Prediction Mamba UNet), an end-to-end structured light depth prediction model from singleshot grating integrating Mamba and U-Net architectures, which fully leverages the long-range capturing capability and low computational performance of the State Space Model, as well as the pixel-by-pixel reconstruction capability of the UNet. Compared with pure CNN and Transformer architecture, DPM-UNet is able to achieve more accurate prediction with a 26.4% error reduction on public datasets. Experiments showed that DPM-UNet is effective in improving the accuracy and robustness of depth maps, and demonstrates remarkable potential in structured light depth prediction tasks.

The program allows you to perform the following tasks: fast loading (without a converter) of OCT images (images); visualization of statistical image analysis conducted in dialogue with the user; saving research results in formats.bmp, .jpg, .tmg, .dat, .png and txt. The «Lumen stratis» software is used to analyze OCT images of the skin during a screening examination in order to promptly identify the nature of pathological skin changes. Certificate No. 2024615567 dated 03/11/2024 on the state registration of the computer program "Visual statistical analysis of skin structure images obtained by non-invasive methods (Lumen stratis)" was received. The possibility of using the program for diagnostic purposes and to objectify the evaluation of the effectiveness of cosmetic procedures is shown.

A trend in structured light 3D measurement systems is miniaturization. To address this trend, this paper presents a highspeed structured light vision imaging and storage system based on Xilinx Zynq-7000 SoC. The system employs a hardwaresoftware co-design approach, with the hardware facilitating high-speed image capture and real-time display from a CMOS camera through programmable logic (PL), while the software manages camera configuration and image storage through the processing system (PS). In comparison to traditional FPGA-based image acquisition systems, the system proposed in this paper utilizes Zynq SoC based on ARM+FPGA architecture. By transitioning the SCCB configuration of the CMOS camera from FPGA to ARM processor control using Zynq, the system reduces hardware resources utilization and enhances camera configuration flexibility. Additionally, through the design of BMP image storage and PL-PS interrupt control programs, the system enables convenient camera calibration using stored images. Experimental results confirm the camera's ability to capture images at 1280×720 resolution with a frame rate of 30fps. Calibration of the CMOS camera using the system yields an average reprojection error of 0.09 pixels, meeting calibration accuracy requirements. This system serves as a foundation for embedded structured light 3D measurement systems.

On the stage of the design of any video surveillance system, important parameters are the technical characteristics of the equipment and the conditions of its placement. The focus of this research is to investigate the effect of the illumination conditions and camera position on the quality of face detection and recognition algorithms for biometrical video surveillance system with facial recognition function.

DIALux software was used for modelling the room with camera and simulate illumination conditions in different zones. For the analysis, the YOLOv8 neural network architecture, widely used for detection, trained on several datasets and standard recognition models from the DeepFace library, were used. Other parameter that became the subject of this research was the camera position (front and tilted), which also has a significant impact on the face detection and recognition algorithms results. As a rule, recognition libraries have strict requirements for the maximum values of the distance from camera to the recognition object, the maximum angle of the camera and its position relative to the visitor's face. Without knowledge of these limitations, it is impossible to design a video surveillance system with object recognition function.

The research results demonstrates that the level of illumination significantly affects the quality of the algorithms. There are also maximum permissible camera angles at which it becomes impossible to detect and recognize faces. The first step in designing a facial recognition system should be to study the conditions of its placement (illumination conditions) and the characteristics of the equipment (camera resolution, focal length, angle of view).

Compared to traditional RGB images, light field images fulfill the demand for high-dimensional information in salient object detection. In this paper, we propose a salient object detection method based on light field depth estimation. In particular, we adopted a supervised learning approach for the design of the light field depth estimation algorithm. Subaperture images in each of the 4 viewing directions are constructed into epipolar plane images (EPI) as inputs to the multistream network. The multi-stream network consists of 4 branches, each containing a specific number of convolutional modules to extract the depth information in the epipolar plane images of the corresponding directions. The features extracted from the 4 branches are fed into the merged network to compute the correlation between the different epipolar plane images. The obtained disparity maps and RGB images are simultaneously inputted into a two-stream convolutional neural network for training. The trained model can achieve highly generalized and robust light field salient object detection. Experiments with real-world light field images indicate the superior performance of our method for clearly plotting the boundaries of saliency objects.

This research presents a comprehensive exploration of point cloud processing, with a particular emphasis on its application in mixed reality systems. It delves into the intricacies of semantic understanding, which is a sophisticated dimension of point cloud analysis that transcends basic segmentation by assigning meaningful labels to each segmented region. This process imparts semantic awareness to segments, providing a deeper understanding of the 3D scene captured.

The study further discusses the process of mesh generation and replacement, which is a pivotal element in computer graphics and 3D modeling. This process involves the creation of a mesh—a three-dimensional representation of a scene or object comprised of connected triangles. The significance of meshes extends beyond mere presentation, as they serve as the bedrock for dynamic, interactive 3D worlds.

The research introduces a novel framework for point cloud completion that leverages attention mechanisms to capture the structural information of 3D shapes. This framework eliminates the need for explicit local region operations, alleviating the influence of data density distribution and achieving high-quality complete shapes with precise geometrical details. The findings from this research provide a holistic understanding of point cloud processing, contributing significantly to tasks such as object recognition, tracking, real-time mesh building, and completing 3D shapes from partial 3D point clouds. These tasks are essential for enhancing the user experience in mixed reality systems, where understanding the semantic meaning of elements in a three-dimensional scene is paramount.

This paper presents a neural network designed for light field (LF) disparity estimation. We improve the network's capability to use spatial and geometric information from light field data by:1. Incorporating positional encoding;2. Adding edge attention mechanisms. The positional encoding aids in deciphering the 3D structure of scenes, which is crucial for accurate LF disparity estimation. Meanwhile, edge attention directs the network to prioritize edge details, enabling the construction of more precise disparity maps. Additionally, edge attention ensures global consistency in disparity estimates, particularly in areas with prominent object edges and limited texture, where it can minimize estimation uncertainty. The attention mechanism also selectively refines features from each view, further boosting the accuracy of disparity estimation. Experiments demonstrate our model's improved accuracy, underscoring the effectiveness of our approach in enhancing LF disparity estimation techniques. The proposed method not only enhances performance but also streamlines the network architecture, making it more scalable and suitable for diverse computer vision scenarios.

Depth estimation and semantic segmentation are crucial for visual perception and scene understanding. Multi-task learning, which captures shared features across multiple tasks within a scene, is often applied to depth estimation and semantic segmentation tasks to jointly improve accuracy. In this paper, a deformable attention-guided network for multi-task learning is proposed to enhance the accuracy of both depth estimation and semantic segmentation. The primary network architecture consists of a shared encoder, initial pred modules, deformable attention modules and decoders. RGB images are first input into the shared encoder to extract generic representations for different tasks. These shared feature maps are then decoupled into depth, semantic, edge and surface normal features in the initial pred module. At each stage, effective attention is applied to depth and semantic features under the guidance of fusion features in the deformable attention module. The decoder upsamples each deformable attention-enhanced feature map and outputs the final predictions. The proposed model achieves mIoU accuracy of 44.25% and RMSE of 0.5183, outperforming the single task baseline, multi-task baseline and state-of-the-art multi-task learning model.

Traffic events can be carried out by traffic surveillance cameras and are paramount in ITS. Adverse weather conditions restrict the camera’s function by reducing the qualities of videos and images and increase the probability of the false detection to target vehicles. So, the research for vehicle detection under typical complex weather conditions is crucial for the development of ITS.

A vehicle detection methodology composed with image enhancement+Yolov5 network under typical complex weather conditions is developed in this paper. First, the specified three image enhancement methods are researched, and their efficacy is evaluated by objective evaluation methods to adverse weather conditions. After that, appropriate enhancement algorithms are chosen for different degraded images, the enhanced images are put into Yolov5 network for training and the high detection precision has achieved on the validation dataset. Then, the comparison research is performance among Faster R-CNN, YoLov3 and the methodology proposed in this paper. It is found that the methodology has higher detection precision and lower time cost, and offers a better balance between the accuracy of detection and the velocity of execution in the three approaches.

Single-pixel imaging has gained prominence for its wide working wavelength and high sensitivity. Deep learning-based single-pixel imaging shows superiority in real-time reconstruction, particularly with limited resources. In this work, we report a novel encoder-decoder method for single-pixel imaging, which aims at enhancing imaging quality from extremely low measurement amounts. First, we encode the high-dimensional target information into one-dimensional measurements using globally optimized modulation patterns, implemented by a fully connected or convolutional layer. Second, we integrate a U-Net neural network with an advanced multi-head self-attention mechanism and a pyramid pooling module to decode the measurements and reconstruct high-fidelity images. Under such a strategy, the skip connections within the U-Net structure enhance the preservation of fine image features, and the incorporation of the multi-head self-attention mechanism and pyramid pooling module effectively captures contextual dependencies among low-dimensional measurements, thereby extracting significant image features and enhancing reconstruction quality. The simulation results conducted on the STL-10 dataset validate the efficiency of the reported technique. With a resolution of 96 × 96 pixels and an ultra-low sampling rate of 1%, we consistently achieved the highest image fidelity compared to traditional single-pixel reconstruction methods for both grayscale and color images.

Near-space surveillance is emerging as a pivotal tool to address the challenges of climate observation, resource exploration, and disaster evaluation. Large-scale multi-modal data enables to improve analysis accuracy, which however faces the challenge of limited downlink bandwidth and storage resources in the near-space platform. In this work, we developed a near-space multi-modal surveillance system, which not only enables multi-modal video acquisition but also realizes efficient data storage control and low-latency, low-bandwidth data transmission. Specifically, a snapshot hyperspectral camera (with 2048×2048 spatial resolution, 5 nm spectral resolution, covering a wide spectral range from 400 to 1000 nm), an infrared camera (with 640×512 pixel resolution), and an RGB camera (with 2448×2048 pixel resolution) were equipped together, to enable synchronous wide-spectrum multi-modal data acquisition with a maximum frame rate of 24 fps. To handle the massive heterogeneous data sets generated by the multiple cameras, a B+Tree index was constructed with the data acquisition time as the primary key, which reduces the time complexity of data retrieval from linear level to logarithmic level. A UDP based image transmission protocol was designed to reduce communication latency by eliminating head-of-line blocking and handshake delays caused by TCP. Remote resource management, on-demand image transmission selection and flexible acquiring control of multi-camera were implemented to further reduce storage space and transmission bandwidth usage. Experiments validated the system’s capability to operate normally under conditions of -50 degrees Celsius temperature and 5 kPa pressure, concurrently affirming the enhanced stability, reliability, and efficiency brought by the aforementioned design.

Extreme ultraviolet (EUV) lithography technology is a key technology used to manufacture very small layout patterns on advanced integrated circuits. Concurrently, computational lithography plays a crucial role in advanced lithographic processes. In computational lithography, the establishment of an EUV lithography imaging model and the consideration of factors are crucial for enhancing the image fidelity and computational efficiency. Therefore, we propose a novel calculation method in the field of EUV lithography imaging, which considers optical aberrations and partially coherent illumination conditions in the calculation process of EUV lithography imaging to significantly reduce the lithography patterning error, and maintain high computational efficiency. This work focuses on the research of the 0.33 numerical aperture (NA) EUV lithography imaging system to verify the feasibility and effectiveness of the proposed calculation method. The calculation and simulation results demonstrate that the proposed calculation method can significantly reduce the errors of lithography patterns while maintaining high computational efficiency. Our future work will focus on the establishment and optimization of EUV lithography imaging models under the conditions of high-NA (NA=0.55) and complicated mask 3D effects. This endeavor will further improve the image fidelity and computational efficiency of high-NA EUV lithography systems to fulfill the escalating manufacturing demands of forthcoming integrated circuits.

Recently, deep neural networks have been successfully applied for removing rain streaks from images. However, these methods did not consider the relationships between the skip layers. In order to improve the deraining performance and use the information of different layers, we propose to construct connections between different layers and propose a residual skip connection neural network for single image deraining. Then we train the network on synthesized data. Both experiments on synthesized and real-world images show the proposed method outperforms state-of-the-art methods in terms of rain streak removal and image information preservation.

Chemical reagents are frequently used in traditional histological staining workflow bringing many drawbacks such as environment pollutions and health damages. Based on automated algorithm and computer computing power, virtual staining with non-pollution, strong robustness and high efficiency is proposed to refine it. However, traditional approaches still exist shortages on reliability due to unawareness of physical basis. In this paper, we fuse optical prior information into the virtual staining pipeline to realize highly robust staining. The total staining process is divided into three parts: (1) spectral staining, (2) optical imaging and (3) color correction. An end-to-end neural network oriented to visual staining is constructed for precise and automatic staining. Multi-scale convolution residual block (MultiResBlock) is designed to better handle with abundant information of spectral cubes while both channel attention and spatial attention modules are adopted to pay more attention to histopathological features. Experimental results demonstrate that generated stained images are visually equivalent with histologically stained. Our virtual staining method gives more robust results replying medical concerns of high reliability, and realizes full link co-optimization from front-end spectral staining to rear-end color correction. It is expected to play an important role in relieving the pressure of pathologist, achieving precision medicine and revealing the nature of life, etc.

Tunable metasurfaces are capable of unprecedented modulation of physical parameters such as polarization, amplitude, and phase of electromagnetic waves. However, existing solutions rarely combine the polarization manipulation ability of liquid crystals with the spatial and polarization multiplexing principles of metasurfaces. Here, we demonstrate an electrically tunable polarization dependent spatially multiplexed metasurface integrated with liquid crystals. Through the combination of polarization and space-dependent metasurface and birefringent liquid crystal, the tuning and switching of the multi-dimensional optical channels with two helicity states are realized, and the brightness and focal length of each optical channel can be different due to spatial reuse. The proposed modulation and integration scheme paves the way for dynamic metasurface devices in various applications such as depth estimation, light detection, ranging systems, and optical chips.

Disparity estimation is crucial for light field applications. How to improve the accuracy of disparity estimation in occluded areas, textureless regions, and non-Lambertian surfaces remains a pressing issue. Disparity estimation methods generally consist of two stages: initial disparity estimation and disparity optimization. Disparity optimization methods can be divided into guided filtering-based methods and deep learning-based methods. The first type of method is based on the boundary information of the RGB image to locate the disparity boundaries, while other regions are constrained by piecewise smooth priors. The second type of method relies on deep learning to learn the mapping relationship between the light field and the real disparity from labeled datasets, and can obtain more accurate disparity images. The prior assumptions of the first kind of methods for disparity images are not accurate enough and are greatly affected by RGB images. The second type of method depends on labeled datasets and lacks generalization ability for data collected from different systems. In recent years, the ability of generative models to mine and represent prior information in data has become increasingly prominent. This paper proposes a disparity optimization method based on a conditional diffusion model. This method learns prior information about disparity images from existing public datasets and uses a conditional diffusion model to generate a disparity image with higher accuracy, conditioned on the initial disparity image estimated from the light field. Experimental results show that the prior information about scene disparity learned by this method is more comprehensive than the piecewise smooth property, is not affected by the RGB image, and has stronger generalization ability.

Fringe Projection Profilometry (FPP) faces challenges with objects of varying surface reflectivity, as projected light can exceed the camera’s dynamic range, hindering effective fringe capture. Current solutions using repeated projections with varying exposures increase measurement time, limiting real-time applicability. This study validates deep neural networks that transform traditional multi-frequency, multi-step, multi-exposure methods into a single-step, multi-exposure format, significantly reducing measurement time while maintaining accuracy. Experimental results demonstrate that deep learning methods can effectively extract phase information from modulated fringe images, unwrap it, and reconstruct 3D point clouds. On high-reflectivity metal datasets, the accuracy of the deep learning approach closely matches that of the traditional six-step method, while using only 16.7% of the time. For standard objects, the accuracy reaches up to 60 microns. These findings confirm that various deep learning methods can efficiently resolve phase information in modulated fringe patterns, significantly enhancing measurement speed.

Achieving a wide field-of-view (WFOV) is an essential performance criterion for optical imaging systems, crucial for applications ranging from military reconnaissance to biomedical imaging. Metasurfaces, two-dimensional arrays of sub-wavelength nano-atoms, offer exceptional light field control and compact design, presenting a novel solution for WFOV optical systems. This approach addresses the size and cost constraints associated with complex multi - element optical assemblies traditionally used for WFOV imaging. We present a configuration combining a aperture stop with a single-layer metasurface to achieve WFOV operation. Through algorithmic optimization and polynomial fitting techniques for phase distribution, we designed a metalens operating at 803 nm wavelength with a 1.25 mm focal length. This design enables near diffraction-limited monochromatic imaging over a 70° field-of-view. Simulation results validate the efficacy of this method for WFOV imaging, demonstrating significant structural simplification compared to multi-lens systems and offering a novel perspective on metasurface applications in WFOV imaging.

Fringe projection profilometry (FPP) technology, renowned for its stable and high-precision characteristics, is widely employed in three-dimensional surface measurements of objects. Whether utilizing deep learningbased methods or traditional multi-frequency, multi-step fringe analysis techniques, both require acquiring highquality stripe patterns modulated by the three-dimensional surface of the object. However, the limited dynamic range of cameras makes it difficult to capture effective fringe information in a single exposure, and while multiexposure methods can address this issue, they are inefficient. To address this, this study proposes an end-to-end neural network approach for generating high dynamic range (HDR) fringe patterns from projected gratings. Additionally, an end-to-end network is employed to solve the fringe phase. Experimental results demonstrate that this method significantly improves fringe pattern recovery on metallic surfaces with overexposed or underexposed regions. On a high dynamic range reflectivity dataset, the method achieved a phase error of 0.02072, successfully reconstructing 3D objects with only 8.3% of the time required by the 12-step Phase Shifting Profilometry (PSP) method. Furthermore, on standard spherical and planar objects, the method achieved a radius accuracy of 53.1 μm and flatness accuracy of 61.7 μm, demonstrating effective measurement precision without the need for additional steps. This method is effective for both high dynamic range reflective and non-high dynamic range reflective objects.

In recent years, the application of Extreme Ultraviolet (EUV) technology has been expanding rapidly across various fields, particularly in lithography, nanoscale imaging, and spectroscopy. Metasurfaces, on the other hand, offer a promising route for miniaturized and lightweight optical devices, with the potential to become one of the directions for future EUV device advancements.

In this study, we present a design for a transmissive phase-type metasurface lens for EUV photolithography. After careful consideration of various materials and pore shapes, we selected molybdenum as the optimal substrate material due to its superior optical properties in the EUV range. Finally, we achieved 2π phase modulation while maintaining high transmittance through a patterned vacuum-guided approach, enabling high-precision wavefront modulation. This choice maximizes the performance and efficiency of the metasurface in the EUV regime.

Numerical simulations demonstrate that the metasurface lens exhibits excellent optical performance. Our findings contribute to the ongoing exploration of photolithography and hold potential for transforming future EUV technologies, particularly in applications requiring high-precision optical components.

CNN-Tranformer Hybrid models, combining the strengths of Transformers in capturing global context and CNNs in local feature extraction, have become an appealing direction in vision perception. However, hybrid models still face the significant challenge of minimizing computing expenses and balancing computational throughput and accuracy. This paper proposes an efficient CNN-Transformer hybrid model that improves throughput and memory consumption with high accuracy, named HTViT. Based on the three-stage architecture of LeViT, HTViT introduced a sparse cascaded group attention mechanism and global-local downsampling modules. The sparse cascaded group attention mechanism compresses the key and value in each group attention by the local aggregation to improve throughput and memory consumption. The global-local downsampling module introduces multi-scale convolution downsampling to enhance the local features and retain more valuable information to improve model performance. Comparison experiments with SOTA efficient hybrid models are conducted separately on CIFAR-10, STL-10, and Imagenette datasets. The experimental results demonstrate that HTViT significantly outperforms the baseline model LeViT and better balances the model size, throughput, memory consumption, and accuracy than other hybrid models.

Fourier ptychography is a promising computational imaging technique that has been successfully applied in various fields, such as quantitative phase imaging, three-dimensional imaging and remote imaging. It scans a series of low-resolution images and then combines them in the Fourier domain to reconstruct a high-resolution image, achieving both wide field of view and high resolution. In this work, we report a high-fidelity Fourier ptychographic reconstruction technique based on a complex-valued channel-wise attention network (CANet) in the plug-and-play (PnP) optimization framework. Following the PnP framework, the optimization objective is decomposed into two sub-problems, including physical model constraint and statistical prior regularization. We utilize physical model constraints to ensure fidelity, and apply CANet to attenuate noise and recover fine details for the prior regularization, achieving both high fidelity and high efficiency. In the network, we introduce the channel attention module that treats amplitude and phase features as a token and calculates self-attention along the channel dimension, thereby exploiting the intrinsic relationship between amplitude and phase information. Meanwhile, we employ multi-stage channel attention modules to extract multi-resolution contextual information, improving the network’s robustness. Experiments validate that the reported technique achieves an average improvement of 1.2dB in PSNR over the compressive sensing method under different scenarios.

Fringe projection profilometry (FPP) has been widely applied in industrial 3D measurement due to its high precision and non-contact advantages. However, FPP may encounter fringe saturation in high-reflective scenes, consequently impacting phase computation and introducing measurement errors. To address this problem, an efficient exposure fusion method is proposed in this paper. We propose incorporating complementary gray codes into the multi-exposure fusion method to improve measurement efficiency for high-reflective scenes. After obtaining high-quality wrapped phase by the fused phase-shifting patterns, complementary Gray code patterns are used to assist phase unwrapping to achieve 3D measurement. This method takes advantage of the fast projection speed and edge error elimination capability of complementary Gray code patterns, requiring only one set of patterns to complete phase unwrapping. Compared to the common multi-frequency method, our method reduces the number of projected images and exposure time. Experiments are conducted to demonstrate the feasibility and efficiency of the proposed method.

Optical flow estimation is widely used in various fields, such as motion scene understanding, autonomous driving, and object tracking. Despite great advances over the last decade, handling illumination variations in optical flow remains an open problem. Conventional optical flow estimation techniques build upon the brightness constancy assumption. Thus, variation in the lighting within the scene can affect the accuracy of optical flow algorithms. To tackle this challenge, here we report a plug-and-play illumination correction technique for robust optical flow estimation under variant illumination scenarios. This technique includes a motion-illumination decoupling strategy for the two images used to compute the optical flow, and an image brightness correction strategy. We trained a UNET-based neural network with Swin Transformer layer as basic block to decouple the motion and luminance information of the two images. Then we applied the decoupled luminance information from the reference image to the source image and adjusted its brightness. This technique makes both images maintain the same luminance for robust optical flow estimation under variant illumination conditions. We applied the reported technique on both traditional optical flow algorithms and deep learning-based optical flow algorithms, and the experiment results validated that it enables to enhance the algorithms’ illumination robustness, and achieves competitive evaluation results on the MPI-Sintel dataset and real-captured data.

Video archives play a crucial role in various industries, from media and entertainment to security and surveillance. Managing these archives efficiently and securely is a growing challenge due to the increasing volume of video data being generated. In this paper, a video archive management method is presented to solve security and sensibility of the video files on blockchain. By encrypting video files and generating cryptographic hashes, the presented method can protect the integrity of the data and prevent unauthorized modifications. The method based on blockchain technology offers a compelling solution to the challenges faced in managing video archives. By combining the inherent security and transparency of blockchain with innovative data management algorithms, organizations can ensure the integrity, security, and accessibility of their video archives.

The authors of this work propose a method for recovering the optical properties of real-world objects with relatively low error. The recovery algorithm is based on the differential rendering method, where each object in the scene has its own optical model, whose parameters are adjusted by an optimizer that compares the original image with the synthesized image on a per-pixel basis, finds the error, and modifies the parameters accordingly to bring the initial model closer to reality. To optimize the calculations, the authors suggest selecting a limited number of the most significant points from different camera viewpoints, and then only analyzing these data points during calculations, which reduces the computation time without sacrificing accuracy.

The paper is devoted to high-level design approaches of hardware accelerators for high-performance execution of spiking neural networks (SNNs). SNNs are considered promising for energy-efficient image segmentation, detection and classification, pattern recognition, optical signal processing and other related tasks. However, for efficient execution of SNNs, hardware should be tightly adapted to workload and co-designed with software. This raises the need for rapid development of deeply customizable, complex hardware, which is not sufficiently supported by mainstream design tools.

We explore application of an original hardware design methodology based on explicit decoupling of computations and transaction flow control to neuromorphic processor microarchitecture. We have developed “Neuromorphix” class library that provides a custom abstraction level for synthesizable neuromorphic processors descriptions. This abstraction level is based on transactions traversing through pipelined structures, computing data while being in-flight within the hardware structures, and managed by automatically generated control infrastructure. This control infrastructure includes a block of sequential neuron selector, spikes scheduling buffers, control flow protocols, memory architecture for static and dynamic neuron parameters, which together orchestrate data processing and communication within the hardware.

The library is integrated in ActiveCore hardware generation framework previously developed by the authors. Experimental designing of FPGA-based prototypes shows that the proposed approach allows to efficiently separate non-intuitive aspects of hardware microarchitecture (requiring highly specific competence for implementation) from trivial application-specific logic and reuse of complex design decisions in hardware microarchitecture, while preserving full control over internals of synthesized hardware and obtaining competitive characteristics of generated designs.

The monocular 3D object detection methods based on Transformer have recently progressed significantly. However, most existing methods struggle to effectively handle fine-grained objects and complex scenes, particularly when capturing the features of occluded or small objects. To tackle these issues, we propose a monocular 3D object detector, CU-DETR, based on the MonoDETR framework. CU-DETR introduces the local-global fusion encoder to enhance local feature extraction and fusion and applies an uncertainty perturbation strategy in position encoding to enhance the model’s performance in handling complex scenes. Experimental results on the KITTI public dataset demonstrate that CU-DETR outperforms the MonoDETR.

It is difficult for traditional CMOS camera to obtain clear images under extremely low-light conditions for example the new moon or the quarter moon because the photons generated are so few that the signal-to-noise ratio (SNR) is much lower than what is necessary to resolve finer details. Being different from traditional CMOS camera, intensified CMOS, named as ICMOS camera can greatly amplify the very limited arriving photons through external photoelectric effect and thus the corresponding SNR could be improved a lot for low-light conditions. In previous studies, by fusing a series of low-light images having sub-pixel displacement between each other through classical iterative back projection (IBP) reconstruction algorithm, not only the resolution is enhanced but also the SNR increases as well. However why the SNR can be improved through super-resolution reconstruction is not theoretically answered yet. Therefore in this manuscript two contributions are made. In the first place, the characteristics of sub-pixel super-resolution low-light imaging are firstly further investigated. By introducing the concept of spectral SNR, the analytical expression of the SNR before and after super-resolution reconstruction is established, based on which it is concluded that the MTF boosting generated by super-resolution reconstruction is one important factor that can bring in the SNR increment besides inherent noise reducing characteristic of the super-resolution reconstruction algorithm itself. In the second place, by combing the IBP based super-resolution reconstruction algorithm, the FFT (Fast Fourier Transform) based single image amplification and image enhancement methods together, better reconstruction results could be obtained.

In recent years, with the rapid development of emerging technologies such as virtual reality (VR) and augmented reality (AR), 3D display technology has attracted widespread attention. As a promising true 3D display technology, multi-layer 3D display is not only able to reproduce the light field, provide complete depth cues, and restore the real scene, but also has excellent viewing and interactive experience, which is widely used in the fields of virtual reality, medical imaging, and game development. However, although multilayer 3D display technology has made significant progress in displaying static scenes, it still faces challenges in displaying dynamic 3D scenes. To overcome these challenges, this study adopts OpenGL as the core technology to achieve dynamic drawing, real-time updating and rendering of graphics and models. At the same time, CUDA technology is used to combine the parallel computing capability of GPU to achieve high frame rate light field decomposition, which achieves 16.7 frames per second display of dynamic scenes at 1920×1080 resolution. This provides users with a more stunning and realistic viewing experience. Through the application of these innovative methods, the further development of multi-layer 3D display technology has been promoted, providing new possibilities and technical support for the display of dynamic 3D scenes. These results not only help to improve the user experience, but also open up a new research direction for the development of virtual reality technology.

To address the issue of reconstruction distortions caused by using non-unified RPC models in 3D reconstruction with multi-view satellite imagery. In this paper, we propose a method termed Rotation Correction Based Neural Radiance Fields for Multi-view Satellite Images. Our method optimizes the satellite camera model and neural radiance field by integrating multi-axial rotational correction into the reconstruction task. By separately correcting the attitude angles along the X, Y, and Z axes, the geometric distortions in multi-view satellite images can be corrected. Furthermore, by setting the ray projection in the neural radiance field under the effect of variable correction rotation variables, our method allows simultaneous optimization of the correction rotation variables and the neural radiance field during back propagation. We conducted experiments using WorldView-3 images from different locations, comparing the reconstruction results of images corrected with traditional BA and our method. Our method achieved the best results.

Novel view synthesis has made significant progress in the field of 3D computer vision. However, the rendering of view-consistent novel views from imperfect camera poses remains challenging. In this paper, we introduce a hybrid bundle-adjusting 3D Gaussians model that enables view-consistent rendering with pose optimization. This model jointly extract image-based and neural 3D representations to simultaneously generate view-consistent images and camera poses within forward-facing scenes. The effective of our model is demonstrated through extensive experiments conducted on both real and synthetic datasets. These experiments clearly illustrate that our model can effectively optimize neural scene representations while simultaneously resolving significant camera pose misalignments.

With the rapid development of security surveillance and autonomous driving, the demand for wide field-of-view imaging is increasingly growing. Traditional wide field-of-view (FOV) single-camera systems often suffer from significant distortion, making it challenging to achieve comprehensive environmental perception. Compound eyes, with tightly packed multiple sub-eyes, have the advantages of being compact, lightweight, and possessing a wide FOV with high sensitivity to moving targets. This paper analyzes the characteristics of wide FOV imaging of biomimetic compound eyes and integrates the imaging features of multiple cameras to design a multi-camera imaging system based on a biomimetic compound eye imaging system. The proposed system comprises nine cameras with a resolution of 640×480 arranged in a spherical multirow configuration to emulate the compound eye structure. Given that feature matching significantly impacts the effectiveness of video stitching, we analyze and compare existing typical feature-matching algorithms from the aspects of both matching accuracy and robustness. Consequently, we employ the LightGlue on top of SuperPoint features for feature point matching to ensure the accuracy of subsequent video stitching. Through imaging experiments, the results demonstrate that this video stitching algorithm achieves satisfactory stitching results with minimal mismatches. The proposed multicamera wide field-of-view compound eye imaging can realize wide wide-field imaging of 170°×120° with 8 frames per second.

Hyperspectral imaging provides rich spectral and spatial data but often collects unnecessary information, leading to inefficiencies. To address this, we developed the Explorative Spectral Acquisition Guide (ESAG), which optimizes the selection of essential wavenumbers, with or without priors, avoiding redundant spectral bands irrelevant to downstream tasks. In hyperspectral stimulated Raman scattering imaging, ESAG significantly reduced the number of spectral frames needed. Notably, ESAG achieved a lower error rate on unmixing tasks using only 60% of the spectral frames compared to the full spectrum. In segmentation tasks involving polymer mixtures, ESAG attained 93.96% average accuracy with only 50% of the spectral window. Interestingly, for pure polymer segmentation, the accuracy of ESAG using 70% of the spectral window surpassed that of the full spectrum. Overall, ESAG enhances both the efficiency and accuracy of hyperspectral imaging, with broad potential applications in chemistry and biomedicine.

The paper presents the development of a high-precision system for processing images of the physical model of the impeller obtained by the optical system in transmitted and reflected light. The paper presents the development of a high-precision system for processing images of the physical model of the impeller obtained by the optical system in transmitted and reflected light. The basis for increasing the accuracy of impeller geometry control is the assumption that there exist a correlation between the accuracy of the entire blade surface and the accuracy of the optically accessible zone of the blade if the conditions for shaping the blade surface are met in the optically accessible for measurements zone. In this work, the controlling trajectory for the measuring machine during the formation of a set of control images on the base of numerically relationships between the positioning coordinates of the cutter during 5-axis processing and conditions of the contact of the cutter . Thus, the image processing system allows to establish, based on a set of focal zones in images, the relationship between the coordinates of a physical object. The developed method allows for not only a significant increase in the measurement speed, but also for the identification and elimination of error in the contact of the cutter with the workpiece surface. A method for setting up a measuring machine is proposed, which ensures the required accuracy of product control during multi-coordinate high-speed milling. Based on the operation of the proposed system, the results of geometric accuracy measurements, linear and angular measurements and studies of the properties of machined surfaces of products equipped with slot of various shapes between blades are obtained.

In this study, a new method for identifying the key indicators and dependencies in processing images of helical surfaces of conical cutters at the main stages of the production cycle, which is based on the relationships between the shape of the cutting edges, the angle of inclination of the helical flute, clearance angle, color intensity and image brightness, has been developed. Design and geometric parameters were exported from the simulations with the model created by the automated system for designing conical cutters, which determines the dimensions, shape and geometric parameters of the cutting part of the tool. The identified dependencies allow determining the active zone and the transfer coefficient at the interface between the rear surface and the measuring machine and a stable connection is carried out with a group of cutting part parameters obtained from the results of the CAD system. A high-resolution image of the flank surface boundary zone is scanned with an optical camera on a specialized machine, and the improved method was used to quantitatively calculate the boundary The controlled measurements of a set of point coordinates and geometric parameters of the helical surface of a conical cutter forms a system for a comprehensively studying the designs of conical cutters and the physical mechanisms of their production using image analysis based on the developed indicator interpretation system, which is a foundation for a unified digital cyber-physical production system to be developed. The combination of improved performance or expanded functional capabilities with increased rigidity and reliability, which enables the processing of a wider range of structural designs of part surfaces, is the primary competitive advantage of the new generation of cutting tools with unique working surface geometries. These tools are designed and manufactured using a developed cyberphysical system from contemporary tool materials. With the least amount of frames, the new method's application enables the quickest possible identification of findings that are appropriate for monitoring the back surface of the cutting tool class under consideration.

Image super-resolution is one of the key problems of image restoration. While a lot of methods focus on working with known degradation (like bicubic downsampling), in real-world use cases the degradation models are complex and unknown and therefore difficult to prepare for. To train such a model in supervised manner paired data with real-world degradations is needed. As only unpaired high- and low-resolution data is usually available due to the high cost of real paired data collection and alignment, methods that tackle the blind super-resolution task face the problem of degradation choice for training data generation. Some of existing methods provide a degradation pipeline that include noise injection, jpeg compression, downsampling with different kernels, etc. These methods may be effective in some cases but offer no mechanism of the pipeline to adapt to real-world scenario therefore lacking the performance. The other approach presented in this paper is simulating the degradation directly without trying to construct it from a predefined list. This can be done using modern generative models such as diffusion models. These models has strong generalization capabilities and are known to simulate the data distributions well. The proposed method uses a diffusion model trained for low-resolution image generation to simulate the degradations and construct paired data given high-resolution data. We compare proposed diffusion-based method with the existing paired data generation techniques and show the performance boost for it.

Automatic restoration of damaged or missing pixels is a key problem in image reconstruction for various applications such as retouching, image restoration, image coding, and computer vision. This paper presents a novel approach for reconstructing texture and edge regions, focusing on achieving fine detail in image completion. The proposed method employs spatial reconstruction based on a geometric model, incorporating contour and exemplar-based texture analysis. We propose a technique for restoring object boundaries in images by constructing composite curves using cubic splines and anisotropic gradients. The shape-dependent gradients utilize the distinct forms in the structural pattern to encode both textural and contour information. Additionally, we search for similar patches, fuse them, and apply a deep neural network. We evaluate our model end-to-end on publicly available datasets, demonstrating that it outperforms current state-of-the-art techniques both quantitatively and qualitatively.

The goal of image enhancement is to improve specific features or details of an image and enhance its overall visual quality. We introduce a novel image enhancement algorithm based on block-rooting processing combined with multi-scale exposure image fusion. The proposed method integrates both local and global transform domain-based feedback mechanisms for imaging applications. The core concept of the local alpha-rooting method involves applying it to disjoint blocks of varying sizes, followed by the decomposition of the weight map and multi-scale enhanced images into Gaussian and Laplacian pyramids. Fusion is achieved by multiplying the multi-scale images and their corresponding weights. A new stage is introduced to obtain a local-global estimate of high-contrast images, which is also employed in the general artificial fusion model. Computer simulations conducted on image datasets demonstrate that the new enhancement algorithm outperforms state-of-the-art techniques.

Industry 4.0 marks a shift toward fully automated digital production, where intelligent systems manage processes in realtime and interact continuously with their environment. Central to this evolution is robotic technology, which enhances productivity and precision in manufacturing. A key aspect of this advanced production model is human-robot interaction, where operators and robots work together on complex tasks. Ensuring safe collaboration between humans and robots is a primary objective. This paper proposes a method for human gesture recognition based on multi-sensor data fusion. By incorporating data from multiple sensors, we achieve a more complete and robust representation of gestures. Our approach involves an algorithm that classifies human movements in real-time using visual data. The process consists of several steps: data preprocessing, feature extraction, data integration, and gesture classification. By employing machine learning and deep learning techniques for feature extraction and analysis, we aim to achieve high accuracy in recognizing gestures.

Important goals in the application of various technologies designed to increase crop yields are to reduce costs and reduce environmental impact. The advent of agricultural robots can improve the quality of fresh produce, reduce production costs, reduce manual labor and, in some countries, offset labor shortages in some agricultural sectors. Modern mobile robots use technologies for constructing the most optimal path for its movement. It uses simultaneous navigation and mapping techniques known as SLAM. The problem with all mapping methods is the presence of lost areas in the depth map. This paper presents an approach to reconstruct depth maps using a generative adversarial network (GAN) structure augmented with a two-path discriminator for separate texture and color analysis. The proposed GAN architecture consists of two main components: a generator and a discriminator. The generator strives to create an image that is as close to the original as possible, without aliases or extraneous objects, while the discriminator attempts to determine how successful the restoration was. As part of the study, an experiment was conducted on a dataset to test the effectiveness of the proposed architecture. The results showed that the new GAN model exhibits good depth map reconstruction performance, outperforming most existing methods.

Grape bushes are one of the most important cultivated plants, which are subject to various diseases leading to significant losses in agriculture. The use of robotic systems and neural networks helps improve the diagnosis and management of these problems, reducing time and resources, increasing accuracy and efficiency. New technologies open up new prospects for agriculture, improving crop yields and quality, reducing costs and improving the economic performance of enterprises. This paper presents a deep learning-based model for leaf lesion recognition in grape plants. As part of the study, an experiment was conducted on a dataset to test the effectiveness of the proposed architecture. The results showed that the new model exhibits effective recognition, outperforming most existing methods.

Research in the field of steganographic methods of information hiding in order to remain undetectable both statistically and for human perception is relevant, since using such files it is possible to hide various information using such content as container. During the investigation, the algorithms for embedding information in video stream were considered. Based on this, the method of embedding information into video file was chosen, specifically the process of writing characters of source text into the pixels of frames obtained from the video file. Key application in the software is defined also. The implementation of software tool for the process of hiding and extracting information in video stream to ensure the safe transmission of hidden information is the result of this article.

Today, there are many technologies for mounting semiconductor crystals and microcircuits on substrates. Some of them require preheating the substrate, which can cause its deformation. Due to the high precision of installation, it is necessary to compensate for the thermal displacement error of the substrate and other positioning errors. To do this, reference marks are made on a heated assembly table. In addition, fiducial marks are located directly on substrates and printed circuit boards.

The main problem of introducing technical vision systems is their integration into numerical control systems of installations in order to increase the speed of technological operations. Thus, the development of real-time algorithms that determine the position of semiconductor crystals relative to the substrate, their implementation in the form of programs for specialized controllers integrated directly into CNC systems, is an urgent task.

The paper presents the implementation of a system for adjusting the position of a semiconductor crystal or microcircuit by processing a video stream from high-resolution cameras, which determines the true coordinates, position relative to the reference marks of the substrate and heating table, and outputs them directly to the CNC programmable logic controller. system. The basis of the system is the 64-bit RISC-V core.

Increasing demands for versatility, ease of deployment and operation, increasing the degree of automation and energy saving necessitate the use of new approaches to the design of manipulators for moving goods. The weight and cost of handling equipment are significantly reduced, as is the degree of protection of more advanced structures based on flexible mechanical connections.

One of the problems of machine that arises when carrying loads, is the impact of factors, such as pendulum fluctuations of the load, compression of suspension cables. In conventional manipulators, such as cranes, this is determined by the skill of the operator. This is not possible in fully controlled non-orthogonal manipulators with multiple actuators.

To solve these problems, the authors of the article use two stereo cameras, the data from which, through mathematical transformations, is fed into the control system. Using a stereo camera is the simplest way to go from object coordinates in pixels an image, in an actual offset expressed in units of length.

The article discusses the technical vision system of a machine for mounting semiconductor crystals. The disadvantage of existing systems is that the moment of installation of a semiconductor crystal on the substrate is not recorded by a camera and is in no way controlled by the operator. This makes it impossible to manually control such installations and complicates the learning process.

The authors of the article eliminated this drawback by using two non-orthogonally located cameras, the focuses of which coincide with the location of the crystal on the working tool. The resulting two images are then processed. The contours of the reference marks on the substrate, already installed elements and the mounted crystal are highlighted. After geometric transformations of the selected contours, the operator receives a real-time map of elements and can adjust the position of the chip in the horizontal plane. All operations are performed in real time. This approach has been practically tested on industrial equipment developed with the direct participation of the authors.

The work examines the problem of improving the quality of data obtained in the IR range, their primary processing for possible analysis by the operator. To perform these operations, the work considers an approach based on the use of a group of methods that are unified for a wide range of tasks and, as a consequence, allow processing on computationally simple devices. We also propose iterative primary processing algorithms based on the multi-criteria smoothing method and the extraction of local features on two-dimensional data. The estimated number of elementary operations required for its implementation is shown, processing examples are given, parameters and errors are errors. An algorithm for localizing closed areas according to the temperature threshold class is presented. An algorithm has been developed for parallel analysis of streaming information from a pair of IR sensors that generate data in different ranges. An approach to parallel data extraction and the formation of a new information field with combined data is considered. Using the example of field test data, the results of improving their quality and examples of the formation of combined IR data are shown.

Significant advancements in Mixed Reality (MR) systems, which blend physical and digital worlds, have been observed in recent years. However, accurate detection and recognition of objects in diverse and dynamic environments remain a key challenge. Traditional methods often struggle to adapt to the variability and complexity of real-world scenarios. This paper proposes a novel approach to address this challenge by leveraging zero-shot models, specifically a Large Multimodal Model (LMM) known for its promising capabilities in Visual Question Answering (VQA), object detection, and segmentation mask generation.

Zero-shot capabilities, the ability to identify objects without prior training, present a potential game-changer for MR systems. However, these abilities can be limited in specific domains, leading to recommendations for fine-tuning the model for optimal performance.

This paper presents a method to fine-tune the LMM for MR systems, focusing on improving object detection and recognition in diverse environments. This approach is demonstrated in a case study involving object detection in MR environments, a domain where foundational models typically do not perform well.

Results show significant improvements in the performance of the MR system, with the fine-tuned LMM demonstrating superior object detection and recognition capabilities. This research opens up new possibilities for the application of zero-shot models in MR, paving the way for more immersive, interactive, and accurate mixed reality experiences. The implications of this research extend beyond MR, offering insights into how zero-shot models can be optimized for various specific domains.