This PDF file contains the front matter associated with SPIE Proceedings Volume 13072, including the Title Page, Copyright information, Table of Contents, and Conference Committee information.

Convolutional Neural Networks have become an important tool for various Computer Vision tasks. Yet, increasing complexity of such architectures drives computational costs. To this end, we propose two measures to achieve similar classification results as state-of-the-art architectures while at the same time reducing model complexity significantly. Firstly, we describe a novel type of non-linear parameter-efficient morphological layers inspired by concepts that are well-known and widely used with convolutions. Secondly, we present a set of simple network architectures, organized as optimization framework, which is enhanced by neural architecture search and hyperparameter optimization. In experiments with hyperspectral remote sensing data, we demonstrate that the identified optimal morphological architecture produces results not only comparable with other architectures from the optimization framework, but also comparable or better than selected state-of-the-art neural network architectures for image classification. Depending on the performed task, the proposed optimized architecture requires up to 25 times fewer parameters than actual state-of-the-art networks.

We propose a novel active learning framework for image classification - sMoBYAL. Our contribution is modifying MoBY - one of the highly effective self-supervised learning algorithms to utilize both labeled and unlabeled data for the active learning pipeline. Finally, we thoroughly evaluate and analyze the robustness and performance of our pipeline in image classification tasks. Our approach attains comparative outcomes, surpassing recent AL methods in terms of results. Our code available at: https://github.com/thanhdh-3030/sMoBYAL

In the paper, we present a quantization method for bipolar morphological neural networks. Bipolar morphological neural networks use only addition, subtraction, and maximum operations inside the neuron and exponent and logarithm as activation functions of the layers. These operations allow fast and compact gate implementation for FPGA and ASIC, which makes these networks a promising solution for embedded devices. Quantization allows us to reach an additional increase in computational efficiency and reduce the complexity of hardware implementation by using integer values of low bitwidth for computations. We propose an 8-bit quantization scheme based on integer maximum, addition, and lookup tables for non-linear functions and experimentally demonstrate that basic models for image classification can be quantized without noticeable accuracy loss. More advanced models still provide high recognition accuracy but would benefit from further fine-tuning.

The initial steps of many computer vision algorithms are local feature extraction and matching. However, in the problem of recognizing objects in images with complex backgrounds, this approach has a weak point since keypoints may be found not only in the object of interest, but also in the background. This leads to redundant calculations and can cause mismatches. In this paper, we propose a keypoints filtering method applicable to the problem of classification and localization of ID documents in the wild. Using a light-weight deep learning model, keypoints are divided into ”document” and ”background” classes, after which the keypoints of the background are removed. Experimental results show that adding the proposed filtering step gives an average speedup of 3.14% on the entire MIDV-500 dataset and 14.77% on MIDV-2020. At the same time, the acceleration on target images with complex backgrounds reaches 81%.

This paper presents two deep learning models using a multi-perspective convolutional neural network (CNN) for classifying objects in the context of intelligent transportation systems (ITS). The proposed model categorizes objects accurately, enabling them to make well-informed decisions in multi-object (such as Persons, Trucks, Motorbikes, Cars, and Cyclists.) detection in complex scenarios for automotive applications. The custom backbone model is designed based on experimentation with the VGG backbone network based on the VGG backbone network, incorporating a multilayer prediction head and custom feature extraction blocks for classifying multiple objects in complex scenes. The model is to extract abstract features and features at multiple scales with a custom-designed feature extraction backbone with multiple blocks. The proposed models are lightweight and require fewer computational resources for high classification performance. The automotive publicly available dataset with 19800 images and labels has been used. Results show that when we experimented with the VGG backbone CNN model, the classification accuracy of 99.64% is achieved, and on the other hand, the classification accuracy of custom backbone CNN is 99.46%. The performance of the proposed custom model is also compared to those of pre-trained benchmark models. The experimental findings presented in this paper show that the proposed models achieve higher accuracy than the pre-trained models.

Drone detection has become an essential task in object detection as drone costs have decreased and drone technology has improved. It is, however, difficult to detect distant drones when there is weak contrast, long range, and low visibility. In this work, we propose several sequence classification architectures to reduce the detected false-positive ratio of drone tracks. Moreover, we propose a new drone vs. bird sequence classification dataset to train and evaluate the proposed architectures. 3D CNN, LSTM, and Transformer based sequence classification architectures have been trained on the proposed dataset to show the effectiveness of the proposed idea. As experiments show, using sequence information, bird classification and overall F1 scores can be increased by up to 73% and 35%, respectively. Among all sequence classification models, R(2+1)D-based fully convolutional model yields the best transfer learning and fine-tuning results.

Recent studies have shown that Convolutional Neural Networks (CNNs) achieve impressive results in crop segmentation of Satellite Image Time-Series (SITS). However, the emergence of transformer networks in various vision tasks raises the question of whether they can outperform CNNs in crop segmentation of SITS. This paper presents a revised version of the Transformer-based Swin UNETR model adapted specifically for crop segmentation of SITS. The proposed model demonstrates significant advancements, achieving a validation accuracy of 96.14% and a test accuracy of 95.26% on the Munich dataset, surpassing the previous best results of 93.55% for validation and 92.94% for the test. Additionally, the model’s performance on the Lombardia dataset is comparable to UNet3D and superior to FPN and DeepLabV3. Experiments of this study indicate that the model will likely achieve comparable or superior accuracy to CNNs while requiring significantly less training time. These findings highlight the potential of transformer-based architectures for crop segmentation in SITS, opening new avenues for remote sensing applications.

The human olfactory bulb (OB), an important part of the brain responsible for the sense of smell, is a complex structure composed of multiple layers and cell types. Studying the OB morphological structure is essential for understanding the decline in olfactory function related to aging, neurodegenerative disorders, and other pathologies. Traditional microscopy methods in which slices are stained with solutions to contrast individual elements of the morphological structure are destructive. Non-destructive high-resolution technique is the X-ray phase-contrast tomography. However, manual segmentation of the reconstructed images are time-consuming due to large amount of data and prone to errors. U-Net-based model to optimize the segmentation of OB morphological structures, focusing specifically on glomeruli, in tomographic images of the human OB is proposed. The strategy to address overfitting and enhance the model's accuracy is described. This method addresses the challenges posed by complex limited data containing abundant details, similar grayscale levels between soft tissues, and blurry image details. Additionally, it successfully overcomes the limitations of a small dataset containing images with extremely dense point clouds, preventing the models from overfitting.

In modern agriculture, livestock monitoring plays a vital role in ensuring animal health, welfare, and production efficiency. Leveraging computer vision and deep learning, this paper presents an innovative framework aimed at enhancing livestock monitoring. Specifically, we address two crucial challenges: denoising and segmentation of cattle in livestock images. The denoising task is fundamental in preprocessing noisy images affected by adverse environmental conditions and equipment limitations. To tackle this, we introduce an encoder-decoder model that effectively denoises cattle images while preserving critical anatomical details. Our framework incorporates a segmentation module inspired by the U-Net architecture. Notably, both denoising and segmentation tasks share a common encoder, optimizing computational efficiency. The segmentation model employs hybrid loss functions and leverages the Grad-CAM technique to provide interpretable insights into the decision-making process. Our approach stands as one of the pioneering joint solutions for cattle denoising and segmentation, particularly focusing on top-view cattle images.

The paper is devoted to the training of binary neural networks. They reduce the requirements for computing power and memory, which is especially important in conditions of limited resources. To date, binary networks do not provide sufficient recognition quality comparable to the quality of traditional floating-point networks, so the development of more efficient methods of training networks are highly relevant. In this paper, we propose a probabilistic model of a neural network that can be transformed into a binary network and consider a way of binarization. Experimental results have shown that our model with incremental binarization and subsequent fine-tuning makes it possible to achieve recognition accuracy of 97.5% for MNIST image classification problem when the accuracy of the binary model trained by Straight Through Estimation was 87.5%.

Handwritten text recognition (HTR) is a challenging task that requires a large amount of diverse training data. One of the possible approaches to this problem is the adoption of CNNs. The key challenge is that the CNN requires geometrically labeled training data, which may increase the cost and time consumption of labeling. To overcome these limitations we propose the method, based on Generative Adversarial Network (GAN), which transfers handwriting styles to printed style images, preserving the Same geometrical Annotation as Input - SAIGAN. Taking printed style image as an input, it produces the handwritten image with the same text content located in the same positions. Our method operates on the character-level and can produce sequences of an arbitrary length and any content. Once trained, it is also possible to generate new handwriting styles by simply manipulating latent vectors. Proposed character style supervision allowed our model to surpass the basis method.

In this work, a probabilistic approach to assessing the quality of a system for determining the authenticity of documents in the images is considered. The considered system is based on the aggregation of responses of various checks of the security elements of the document. We propose a probabilistic model of the document authentication system and the functional for evaluating the quality of the system. Based on them, we offer an approach for assessing the quality of the separate part and the whole system. Finally, the approach to constructing an optimal function of making a final decision is obtained.

Datasets are de facto the only way to test the recognition pipelines and to compare them with each other. To avoid the manual gathering of documents and, moreover, to avoid problems with the law in the case of ID documents researchers create synthetic datasets or datasets of fake documents, but this process is also time-consuming. In this paper, we present a simple method to use when you need to test a recognition pipeline or some part of it. The method employs only the information that the developers of such pipelines use in their work and allows them to create natural-looking images. The quantitative experiments show that the recognition accuracy of the synthesized images corresponds with the recognition accuracy of the MIDV-2020 dataset. The qualitative comparison also demonstrates that such images can be helpful in recognition systems’ development.

We present FARA, a novel approach for fast approximation of RFD-like descriptors in the context of document retrieval systems. RFD-like descriptors are widely used for document representation, but their computation is expensive, especially for large document collections. Our method is a CPU-friendly gradient maps computation approximation with sequential memory access and integer-only calculations. There are three types of operations that we use: addition, subtraction, and absolute values. It allows us to effectively use SIMD extensions, resulting in an additional increase in the running speed. Experimental results demonstrate that FARA achieves the same accuracy as RFDoc descriptors and significantly reduces the computational overhead. The proposed approach achieves a twofold speed improvement of gradient maps computation and 25% acceleration of overall descriptor computing time compared to the most efficient RFDoc implementation.

Object detection is one of the most common problems solved by computer vision systems. Even though neural network methods have become a standard tool for solving the problems, these methods have many disadvantages, which include high computational power requirements both for training and inference stages and tremendous training sets. This paper considers such a classical method for object detection as the Viola and Jones method and proposes an enhanced soft cascade calibration method based on Multiple-Instance Pruning to increase detection performance. The proposed method considers a response of the classifier to an image region as a random variable and follows a statistical approach to provide robust detectors. In addition, the paper addresses the problem of non-conformity of detection parameters at training and inference stages and studies performance decline. The performance of the proposed methods is demonstrated in a variety of practical tasks, including identity document detection and document fraud detection.

Anomaly detection is one of the most popular fields for computer vision in industrial applications. The idea of training machine learning only on defect-free objects saves enormous amounts of integration effort. The state of the art shows that current methods on public data sets (e.g. MVTec AD data set) have already solved the problem with AUROC segmentations scores of more than 99%. But how accurate are these methods really? In this paper, one current method from the field of supervised learning and anomaly detection is evaluated on two problems. Each problem contains a defect pattern that grows in 11 steps. This work shows that the defect is already reliably detected from a relative size of 0.03% of the pixels in the image.

The ability of autonomous vehicles (AVs) to detect three-dimensional objects is crucial for motion planning, object tracking and safe driving. This task is especially challenging for systems using only monocular cameras, for which depth estimation presents special difficulties. In this paper, we discuss the subsystem of 3D object detection in bird’s-eye-view (BEV) for a single camera in an AV system. The subsystem consists of two parts. First, it estimates the contour of the object’s projection polygon in BEV based on 2D detection and drivable area segmentation (a planar ground model is used). Second, it simplifies the object’s projection by fitting the obtained polygon to a rotated bounding box. For this part we propose a new L-shape model-based fitting algorithm. It assumes that the vertices of the input polygon belong to two adjacent sides of the fitted bounding box. We compared this algorithm with a naive approach which minimizes the bounding box’s area and with adaptations of algorithms from a paper solving a similar problem with LiDAR point clouds. The L-shape algorithm outperformed the alternatives.

A robust object detection algorithm is essential while detecting objects in videos and real time scenarios, where false positives might result in unwanted outcomes. Our goal here is to observe how Simple Online and Real-time Tracking with a Deep association metric (Deep SORT) algorithm for Multi-Object Tracking (MOT) can be used to minimize false positives, from a state of the art detection algorithm like You Only Look Once (YOLO), by using the Kalman filter approach. An auto encoder based feature extractor has been used, instead of the standard CNN networks like ResNet-50 to further improve speed of the detector. There have been other MOT algorithms in the recent times which give good results, but are not as real time efficient as the simple yet efficient Deep SORT method. Experimental analysis has shown how Autoencoder based Deep SORT performs in contrast to native Deep SORT and YOLO, in eliminating false positive detections.

The amount of image forgery strongly increases recently. There are different ways to fake an image, one of the common ones is a copy-move manipulation. There are numerous methods for detecting copy-move manipulations on natural images. However, they are difficult to adapt for document images due to their features. This work proposes an algorithm for detecting and localizing copy-move manipulations on digital images of documents. The main idea is to use JPEG artifacts in order to find the target region area and then localize the source and target regions precisely. For the efficient application of the proposed method, firstly, the original image must have been subjected to JPEG compression, and secondly, after the manipulation the image must have been saved in a lossless format. The experiments were carried out on an open set of document images CMID; in the detection task, the recall was 0.992, the specificity was 1.0; in the localization task, the recall was 0.923, the false discovery rate was 0.021, which means that the proposed algorithm successfully detects more than 99% of copy-move manipulations, similar to manipulations in the CMID and does not give false positives.

One-class incremental learning is a special case of class-incremental learning, where only a single novel class is incrementally added to an existing classifier instead of multiple classes. This case is relevant in industrial defect detection scenarios, where novel defects usually appear during operation. Existing rolled-out classifiers must be updated incrementally in this scenario with only a few novel examples. In addition, it is often required that the base classifier must not be altered due to approval and warranty restrictions. While simple finetuning often gives the best performance across old and new classes, it comes with the drawback of potentially losing performance on the base classes (catastrophic forgetting [1]). Simple prototype approaches [2] work without changing existing weights and perform very well when the classes are well separated but fail dramatically when not. In theory, null-space training (NSCL) [3] should retain the basis classifier entirely, as parameter updates are restricted to the null space of the network with respect to existing classes. However, as we show, this technique promotes overfitting in the case of one-class incremental learning. In our experiments, we found that unconstrained weight growth in null space is the underlying issue, leading us to propose a regularization term (R-NSCL) that penalizes the magnitude of amplification. The regularization term is added to the standard classification loss and stabilizes null-space training in the one-class scenario by counteracting overfitting. We test the method’s capabilities on two industrial datasets, namely AITEX and MVTec, and compare the performance to state-of-the-art algorithms for class-incremental learning.

This paper proposes an algorithm for detecting finger areas in document images captured in an uncontrolled environment. The main idea of the proposed algorithm is to segment the image on the chromaticity plane and search for a segment that crosses the boundary of the displayed document in the image. It is proposed to use segmentation in combination with the edge analysis to prevent false merging of two segments belonging to the document and the background respectively, being different in the original RGB space, but acquiring similar characteristics on the chromaticity plane. Testing was performed on document images from the MIDV-2020 open dataset. The precision of the proposed algorithm was evaluated as 91.2%, and the recall as 73.5%.

Selecting representative data is a key factor in improving the performance of machine learning algorithms. In this paper we focus on out-of-distribution (OoD) methods evaluation, which can be integrated into ML project lifecycle in a nonintrusive way, without changing a model architecture. Considered methods are applicable to image classification datasets analysis. In addition to commonly used AUROC metric, we evaluate the number of out-of-distribution samples misclassified with high confidence. Case studies were conducted on benchmark and production datasets. As a result, we provide practical guidance for data evaluation and recommendations on which method to use to detect different types of OoD images.

We introduce a novel approach for image edge detection based on calculating pseudo-Boolean polynomials on image patches whose resulting polynomial degrees determine whether a patch lies over an edge or a blob. In this paper we show that patches covering edge regions within the image result in pseudo-Boolean polynomials of higher degrees compared to patches that cover blob regions. The proposed approach is based on reduction of polynomial degree and equivalence properties of penalty-based pseudo-Boolean polynomials.

Object detection is a critical task in computer vision with applications across various domains, ranging from autonomous driving to surveillance systems. Despite extensive research on improving the performance of object detection systems, identifying all objects in different places remains a challenge. The traditional object detection approaches focus primarily on extracting and analyzing visual features without considering the contextual information about the places of objects. However, entities in many real-world scenarios closely relate to their surrounding environment, providing crucial contextual cues for accurate detection. This study investigates the importance and impact of places of images (indoor and outdoor) on object detection accuracy. To this purpose, we propose an approach that first categorizes images into two distinct categories: indoor and outdoor. We then train and evaluate three object detection models (indoor, outdoor, and general models) based on YOLOv5 and 19 classes of the PASCAL VOC dataset and 79 classes of COCO dataset that consider places. The experimental evaluations show that the specialized indoor and outdoor models have higher mAP (mean Average Precision) to detect objects in specific environments compared to the general model that detects objects found both indoors and outdoors. Indeed, the network can detect objects more accurately in similar places with common characteristics due to semantic relationships between objects and their surroundings, and the network’s misdetection is diminished. All the results were analyzed statistically with t-tests.

There are various techniques for decreasing the computational complexity of neural networks, and a number of them use neuron approximations. A bipolar morphological neuron is an approximation of a classical neuron that can be used on FPGAs and ASICs to enhance computational efficiency. It uses 4 distinct computational pathways utilizing addition and maximum functions, in contrast to the traditional neuron which employs multiplication and addition. In this paper, we introduce bipolar morphological YOLO network for object detection task. To train the network, we employ an iterative approach that combines knowledge distillation for backbone and fine-tuning of the network’s head. Our experiments, which were conducted using the COCO dataset, yield results that are on par with classical networks. Specifically, the average recall for large images is 0.393 for the BM network and 0.371 for the classical network. Additionally, the average precision values are 0.088 for the BM network and 0.097 for the classical network. These outcomes establish a baseline for object detection using bipolar morphological networks.

Semantic scene reconstruction from sparse and incomplete point clouds is a critical task for point scene understanding. It aims to recognize semantic labels for objects and recover their complete shapes as meshes. Existing methods often fail to realize high-quality instance reconstruction due to inadequate shape representation and underutilization of proposal point clouds. To address these issues, we optimize the previous BSP/occupancy-to-mesh reconstruction framework to points-to-mesh and accomplish multi-level utilization of proposals. We chose point cloud as the representation of completion to reduce the difficulty of restoring curved shallow parts. Benefiting from the optimization, we can match and merge proposal point clouds with the restored ones, avoiding missing parts existing in inputs. We design an effective pose normalization module to extract point-based features from normalized proposals, which are fused with features extracted from voxelized proposals, avoiding the detailed geometry lost in voxelization and enhancing the reconstruction's robustness to different input postures. The suitable points-to-mesh reconstruction framework and full utilization of proposals make our method improve reconstruction results efficiently. Detailed experiments on the challenging ScanNet dataset of the semantic scene reconstruction benchmark show that our network outperforms state-of-the-art methods in both completion and mapping metrics.

Recently, we presented the SCA-2023 dataset, which had been developed specifically to evaluate the quality of various image precompensation algorithms for observers with imperfect vision. Such precompensation makes it possible to bring their image perception closer to that of an observer with the ideal vision. While experimenting with various image quality metrics, we realized that it was not so easy to evaluate the quality provided by different algorithms, since the metrics ''voted'' for different things, and their choice often seemed to contradict the human perception. This is a key motivation for our study, in which we set out to select the metric best correlated with the human perception of precompensated images. We selected a suitable subdataset from our SCA-2023 dataset and, based on it, created 90 grayscale images, which were shown to our colleagues in a pairwise comparison way. More than 2,000 pairwise comparison results were collected from 24 study participants. Further, according to our original methodology, these results were compared with the ''opinion'' of some popular quality metrics, which made it possible to rank these metrics according to their adequacy within the framework of this task. Finally, we showed how to use these results in optimization procedures aimed at improving the quality of precompensation.

U-Net similar architectures are widely used in the task of document image binarization. However, despite the good quality of binarization, they also have high computational complexity, which greatly limits their use on mobile and embedded devices. The performance bottleneck of U-Net architectures is the first encoder layers and the last decoder layers, which operate on high-resolution input data and contain the largest number of operations. Based on this, in this paper we propose a new Threshold U-Net model: instead of predicting the final image, Threshold U-Net predicts a low-resolution adaptive threshold map, with which the input image is binarized. The proposed architecture naturally combines the ideas of classical algorithms that calculate the binarization threshold for a specific image region with an approach based on a deep learning model with a large receptive field and context understanding. Threshold U-Net demonstrates quality of binarization of historical documents comparable to U-Net on the DIBCO-2017 dataset. At the same time, depending on the resolution of the threshold map, Threshold U-Net is up to 2 times faster, requires up to 26% less RAM and consists up to 10% fewer parameters.

It is critical to make full use of the information of the backbone to improve the performance of object tracking. A common way to mine useful information is to add attention to features. However, most trackers only use single attention to mine the features and fail to utilize the effective information in the backbone. To leverage the useful information of features in multiple ways, this paper proposes an integrated channel attention mechanism based on all kinds of commonly used channel attention methods. First, we used ResNet50 as the backbone, and then we used four attention methods to process the fourth-stage features that were extracted from the backbone to obtain attention factors. Then, through adaptive weighting, we added the four attention methods to the original features. It adaptively adjusts the importance of each channel attention, suppresses redundant information, and better captures key features of the tracked object in different channels. The effectiveness of our approach is validated on five tracking benchmarks.

In this paper, we consider the problem of clothes compatibility for total look recommendation systems by means of deep neural networks. This task has become very popular in recent years, primarily due to the growth of online retail sales of clothing. Unlike the existing solutions, we developed a comprehensive model of clothes compatibility evaluation based on color characteristics as well as on the characteristics of the style. As a rule, neural networks are robust to the color characteristics of an image, but color is an extremely important component in the task of a total look evaluation, so such additional branch with color characteristics is well justified. The proposed model uses both: color embedding obtained from color clustering and histograms, and style embedding as an output tensor of ResNet-50 encoder. The paper shows that color embeddings significantly improve the quality of the total look evaluation. The model was trained on Polyvore dataset, which was pre-processed and cleaned from the items not related to the topic of total look compatibility.

Nowadays, mobile robots solve various tasks realted to navigation in an unknown or changing environment. In such conditions, mapping the environment is crucial for mobile robot navigation. Conventionally, maps are built as 2D or 3D dense metric structures which require much memory for storage and much computational time for path planning, especially in large environments. Representing a map as a topological structure (i.e. a graph of locations) allows fast path planning with low memory consumption. In this paper, we present a method of real-time topological map building and updating from odometry measurements and local point clouds. The proposed method guarantees the topological graph connectivity and adds shortcuts to the graph to optimize paths. We tested our method in a simulated environment and measured efficiency of path planning in the obtained graphs. In all the tests, the SPL value exceeded 81%, with 100% success rate. The source code of our method is available at https://github.com/KirillMouraviev/simple_toposlam_model.

Spectral reconstruction (recovering spectra from RGB measurements) is a vital problem of computational photography. As a matter of curiosity, modern mobile devices open a new opportunity to improve the quality of spectral reconstruction by utilizing images from several cameras at once. This leads to the idea of creating a mobile hyperspectral camera for the general public. In this paper we investigate the achievable accuracy when using several identical cameras simultaneously in combination with different spectral filters. To find optimal filters, two algorithms are proposed: one learns spectral transmittance functions simultaneously with spectral reconstruction, the other learns only spectral transmittances by information loss minimization. As a result of numerical experiments, 4 cameras and 4 filters allow us to perform spectral reconstruction two times accurately than from a single RGB image.

Currently, color image formation is based on the CIE 1931 and CIE 1964 standard colorimetric observer models. Expansion of color gamuts of displays requires narrow spectral power distributions functions of their primary colors. In this case, the use of CIE 1931 and CIE 1964 models leads to color reproduction errors, as these models do not take into account the peculiarities of color vision of a particular user. To reduce color rendering errors caused by individual vision features, a method of color image formation based on the categorical observer model is proposed. The basis of the method is the evaluation of the categorical observer type using binary tests, which can be implemented outside the laboratory conditions on conventional displays with three basic colors. The effectiveness of the method has been confirmed experimentally.

Computed tomography (CT) is a powerful tool for reconstruction and analysis of inner structure of objects applied in various fields. Although many classes of objects of interest may have highly absorbent inclusions, leading to a certain type of distortions on reconstructed volume images (metal-like artifacts). The correction of this type of artifacts can’t be considered a solved task, despite all the efforts in this direction. The development and research of methods for suppressing CT artifacts require high-quality synthetic data which allow for numerical assessment of the accuracy of the metal-like artifacts reduction methods and training of neural networks. Although simplified methods considering only beam hardening and Poisson photon distribution are commonly used to simulate the data with type of distortions. In present work we design experiments using the tomographic scanner of the Federal Research Center “Crystallography and Photonics” of the Russian Academy of Sciences to demonstrate that in some cases beam hardening may not be the dominant reason for the arising of metal-like artifacts. These experiments are closely analyzed and modeled within different approaches. The problems in both simplified and state of the art approaches are emphasized and discussed. The provided results show the importance of paying attention to the dark current modeling for synthesized data generation under the conditions of total photon absorption.

In this paper, we introduce HoughToRadon Transform layer, a novel layer designed to improve the speed of neural networks incorporated with Hough Transform to solve semantic image segmentation problems. By placing it after a Hough Transform layer, ’inner’ convolutions receive modified feature maps with new beneficial properties, such as a smaller area of processed images and parameter space linearity by angle and shift. These properties were not presented in Hough Transform alone. Furthermore, HoughToRadon Transform layer allows us to adjust the size of intermediate feature maps using two new parameters, thus allowing us to balance the speed and quality of the resulting neural network. Our experiments on the open MIDV-500 dataset show that this new approach leads to time savings in document segmentation tasks and achieves state-of-the-art 97.7% accuracy, outperforming HoughEncoder with larger computational complexity.

Super-resolution is crucial in computer vision and digital image processing, aiming to enhance low-quality images’ resolution and visual quality. This paper focuses on correcting the distortion introduced by fisheye lenses and improving the resolution of images for better detail representation. Specifically, we propose an evaluation approach that benchmarks three state-of-the-art models in different categories: Real-ESRGAN (convolutions), SwinIR (transformers), and SR3 (diffusion). We evaluate their performance in super-resolution and distortion correction tasks using metrics such as PSNR and SSIM. To facilitate this evaluation, we create and release a new dataset of lunar surface images with fisheye distortion applied. Our experiments demonstrate the effectiveness of each model in handling distortion and improving image resolution. The results show that large models generally outperform medium models, and PSNR models achieve higher PSNR and SSIM scores than GAN models. Additionally, we evaluate the distortion correction by comparing the corrected images with ground truth. Our findings contribute to understanding different model categories and their performance in super-resolution and distortion correction tasks. The proposed dataset and evaluation approach can be valuable resources for future research.

The rotation axis position is an important parameter of classical reconstruction algorithms in X-ray computed tomography (CT). The use of incorrect values of the axis position parameters during the reconstruction leads to the appearance of various artifacts distorting the reconstructed image. Therefore, to obtain a reconstruction of better quality, automatic rotation axis position determination and misalignment correction methods are of use. Most of the existing high-precision automatic rotation axis position determination methods are either fast, but suitable only within a parallel-beam geometric scheme, or indifferent to the geometric scheme, but computationally laborious. In this paper, we propose a method for auto-detection of two scalar parameters of rotation axis position — axis shift and tilt in the plane parallel to the detector window plane — using a pixel-wise arithmetically averaged projection image. The described method is highly accurate within both parallel-beam and cone-beam geometric schemes whereas it is characterized by robustness to noise in projection data. The method has performed an increase in reconstruction quality when compared with some well-known and still used in practice methods both on synthetic data and on real data obtained in real laboratory conditions.

This paper presents a 3D multiple object detection and tracking framework for identifying and quantifying changes in fish behaviour through tracking the 3D position, distance and speed of fish with respect to an underwater stereo camera. The framework consists of six essential modules based on 3D object detection to identify fish and multiple object tracking algorithms to track the fish in sequential frames. In particular, the latest version of Yolo (Yolov7) is utilised for object detection and the deep SORT algorithm is used for multiple object tracking. The framework was tested using videos captured from an underwater stereo camera in an industrial-scale sea-based fish farm. The results showed that the framework was able to accurately detect and track multiple fish in 3D. The fish position, distance and speed relative to the camera were also successfully detected. The results of this study demonstrate the effectiveness of this framework in identifying and quantifying changes in fish behaviour. The proposed novel framework has the potential to greatly enhance our understanding of fish behaviour in their natural habitats, leading to new insights into fish ecology and behaviour, while at the same time, it can enable researchers to study fish behaviour in a more detailed and accurate way.

The right ventricular (RV) function deterioration strongly predicts clinical outcomes in numerous circumstances. To boost the clinical deployment of ensemble regression methods that quantify RV volumes using tabular data from the widely available two-dimensional echocardiography (2DE), we propose to complement the volume predictions with uncertainty scores. To this end, we employ an instance-based method which uses the learned tree structure to identify the nearest training samples to a target instance and then uses a number of distribution types to more flexibly model the output. The probabilistic and point-prediction performances of the proposed framework are evaluated on a relatively small-scale dataset, comprising 100 end-diastolic and end-systolic RV volumes. The reference values for point performance were obtained from MRI. The results demonstrate that our flexible approach yields improved probabilistic and point performances over other state-of-the art methods. The appropriateness of the proposed framework is showcased by providing exemplar cases. The estimated uncertainty embodies both aleatoric and epistemic types. This work aligns with trustworthy artificial intelligence since it can be used to enhance the decision-making process and reduce risks. The feature importance scores of our framework can be exploited to reduce the number of required 2DE views which could enhance the proposed pipeline’s clinical application.

In the recent years, there has been a significant growth of interest in real-world systems based on deep neural networks (DNNs). These systems typically incorporate multiple DNNs running simultaneously. In this paper we propose a novel approach of multi-DNN execution on a single GPU using multiple CUDA contexts and TensorRT, state-of-the-art DNN inference framework. We show that it can lead to more efficient scheduling of multiple DNNs, especially in case when a lightweight and a heavy DNNs are inferred together. We show that our approach can provide an almost 7x increase in the throughput of a lightweight DNN at the cost of neglible throughput drop of a heavy DNN, compared to the baseline. Moreover, we compare two ways of improving throughput of a single DNN by processing multiple images together: standard batching and implicit batching by processing multiple images simultaneously using several TensorRT execution contexts. We show that meanwhile standard batching outperforms implicit batching at larger batch sizes, implicit batching can provide up to 43% more throughput for a smaller DNN using smaller batch size.

Remote sensing images often suffer from different types of haze. Its presence significantly complicates remotely sensed image analysis that is crucial for monitoring of land state and precision agriculture. Currently existing remote sensing dehazing methods are designed for achromatic haze, but in cases such as smoke from fires or sandstorms, the haze may have its own pronounced coloration. In this paper we propose a new hazed image formation model that considers chromatic haze. Using this model we propose a new single image dehazing method CADCP that is based on color attenuation and dark channel priors. For quality assessment of the proposed method we generated a dataset of remotely sensed images with simulated chromatic haze. The generated dataset includes data with various haze spatial distribution and density. Quality evaluation results including qualitative and quantitative approaches demonstrated better results of the proposed method comparing with other existing methods.

Anomaly detection is one of the most popular fields for computer vision in industrial applications. The idea of training machine learning only on defect-free objects saves enormous amounts of integration effort. The state of the art shows that current methods on public data sets (e.g. MVTec AD data set [1]) have already solved the problem with AUROC segmentation scores of more than 99%. In real-world applications training data is not as ”clean” as in public data sets. This work investigates the changes in detection performance when outliers end up in the training data. For this purpose, the training data is enriched step by step with images of defective objects. The AUROC score and the anomaly score is used as a quality criterion for performance measurement. We show that state of the art methods can be very robust, but that in some scenarios a draw down of 15 percentage points is possible.

Tracking objects can be a difficult task in computer vision, especially when faced with challenges such as occlusion, changes in lighting, and motion blur. Recent advances in deep learning have shown promise in challenging these conditions. However, most deep learning-based object trackers only use visible band (RGB) images. Thermal infrared electromagnetic waves (TIR) can provide additional information about an object, including its temperature, when faced with challenging conditions. We propose a deep learning-based image tracking approach that fuses RGB and thermal images (RGBT). The proposed model consists of two main components: a feature extractor and a tracker. The feature extractor encodes deep features from both the RGB and the TIR images. The tracker then uses these features to track the object using an enhanced attribute-based architecture. We propose a fusion of attribute-specific feature selection with an aggregation module. The proposed methods are evaluated on the RGBT234 [1] and LasHeR [2] datasets, which are the most widely used RGBT object-tracking datasets in the literature. The results show that the proposed system outperforms state-of-the-art RGBT object trackers on these datasets, with a relatively smaller number of parameters.

Portfolio management is an essential part of investment decision-making. However, traditional methods often fail to deliver reasonable performance. This problem stems from the inability of these methods to account for the unique characteristics of multivariate time series data from stock markets. We present a two-stage method for constructing an investment portfolio of common stocks. The method involves the generation of time series representations followed by their subsequent clustering. Our approach utilizes features based on Topological Data Analysis (TDA) for the generation of representations, allowing us to elucidate the topological structure within the data. Experimental results show that our proposed system outperforms other methods. This superior performance is consistent over different time frames, suggesting the viability of TDA as a powerful tool for portfolio selection.

The availability of large datasets has significantly contributed to recent advancements in deep Convolutional Neural Network (CNN) models. However, training a large CNN model using such datasets is a time-consuming task. This issue has been addressed by the parallelization and distribution of data/model during the training process. There are two ways to implement distributed deep learning processes: data parallelism and model parallelism. Data parallelism involves distributing the dataset across multiple workers, allowing them to process different portions simultaneously. While increasing the number of workers can reduce computation time, it also introduces additional communication time. In some cases, the increased communication time can outweigh the benefits gained from reduced computation time. In this paper, our focus is on reducing the overall computation time of data parallel approach by employing two strategies. First, we emphasize the preservation of dataset distribution across all workers, ensuring that each worker has access to representative data. Second, we explore the localization of parameters and the quantization of gradients to three levels: {-1, 0, 1} to reduce communication delays between the server and workers, as well as between workers themselves. By adopting these two strategies, we aim to enhance the performance of data parallel approach in the distributed deep learning processes. As a result of preserving the distribution of the data while sampling the entire data, each partition retains a similar mean and variance (capturing important first and second-order statistics). This approach guarantees that all worker machines train their local models on uniformly distributed data instead of random distribution. Additionally, localizing parameters limits the communication between the server and workers to gradients only. Furthermore, by quantizing gradients to 2-bits, we successfully achieve our objective of reducing computation time by enabling faster convergence without compromising test or validation accuracy. The experimental results demonstrate that employing these strategies in distributed deep learning effectively reduces communication overhead and leads to faster convergence when compared to methods that utilize random data sampling. These improvements were observed across multiple datasets such as MNIST, CIFAR-10, and Tiny ImageNet.

Merger of Quantum computing and Machine learning explores a new shift in artificial intelligence. Quantum neural networks and parameterized quantum circuits are tools that enable the merger of these two branches. Here, we use Quantum circuit enabled long short term memory (QLSTM) neural network to forecast time series data. We show the efficacy of quantum computing by conducting experiments on two different time series data sets. In our first experiment, we are predicting the amount of rainfall fall and the second experiment has electric load (power) prediction. Our dataset for rainfall prediction includes hourly information on the weather conditions i.e., wind speed, wind direction, minimum and maximum temperatures, and pressure with the amount of rain falls. For electric load (power generation) dataset, few of the features include amount of biomass, amount of fossil brown coal/ignite, amount of fossil coal derived gas, nuclear power, solar power, and the corresponding wind velocity forecasts. We compare the training as well as the test loss of classical Bidirectional LSTM (BILSTM) and the Quantum BILSTM and observe that LSTM based on quantum approach reduces both the training and test loss considerably when compared to its classical part with very few epochs of training.

The field of affective computing has seen significant advancements in exploring the relationship between emotions and emerging technologies. This paper presents a novel and valuable contribution to this field with the introduction of a comprehensive French multimodal dataset designed specifically for emotion recognition. The dataset encompasses three primary modalities: facial expressions, speech, and gestures, providing a holistic perspective on emotions. Moreover, the dataset has the potential to incorporate additional modalities, such as Natural Language Processing (NLP) to expand the scope of emotion recognition research. The dataset was curated through engaging participants in card game sessions, where they were prompted to express a range of emotions while responding to diverse questions. The study included 10 sessions with 20 participants (9 females and 11 males). The dataset serves as a valuable resource for furthering research in emotion recognition and provides an avenue for exploring the intricate connections between human emotions and digital technologies.