Machine Learning has played a major role in various applications including Visual Slam and themal image process. In this paper, we discussed the possibility of generating a thermal map using LWIR images and a deep learning-based visual slam network and the value that the thermal map can create. We summarized the advantages and applicability of various deep learning-based visual slams and confirmed the results of nice slam, which generates the most curious Dense map. In order to apply Visual SLAM technology, time series, scene repetition, and images from various angles for one scene are required. However, most LWIR data sets consist of one shot for each scene or are unidirectional driving data. To solve this, we created a scenario using the LWIR driving dataset and created a repetitive route through repetition. RGB-Depth SLAM Mapping was performed on the constructed data set, and the results were evaluated and the limitations of the current approach were discussed. Finally, we summarized future directions for creating stable 3D thermal maps in indoor and outdoor environments by resolving the limitations.
In this paper, we proposed a template matching technique using deep learning to match pairs of wide fields of view and narrow field of view infrared images. The Deep Learning network has a similar structure with the Atrous Spatial Pyramid Pooling (ASPP) module and both wide and narrow fields of view images are input to the same network, so the network weights are shared. Our experiments used the Galaxy S20 (Qualcomm Snapdragon 865) platform and show that the trained network has higher matching accuracy than other template matching techniques and is fast enough to be used in real time.
In recent times, target recognition techniques based on deep learning in the optical domain have exhibited impressive performance. Because of these promising results, there has been a surge in research centered around deep learning in the field of Synthetic Aperture Radar (SAR) target recognition. Most of the contemporary studies directly adopt or modify the deep learning model structures used in optical image target recognition. A primary limitation of this approach is the large amount of data required for training. However, collecting real SAR data entails significant time and cost, and the availability of publicly accessible SAR target datasets are also insufficient. As a solution, studies have been undertaken to generate synthetic SAR data using CAD models and electromagnetic simulations. Yet, a discrepancy in recognition performance emerges due to domain differences between synthetic and real data, especially variations in speckle intensity and side-lobes. In this paper, we propose a novel domain randomization technique to mitigate these inter-domain disparities. Utilizing adversarial generative networks as a foundation, we preserve the core characteristics of SAR targets while minimizing domain differences, applying random transformations to extraneous elements (e.g., Clutter, Speckle). Through this method, we can diversify a single SAR data into various data, effectively augmenting the dataset. This considerably enhances the recognition performance and robustness of deep learning-based target recognitions models.
Short wave infrared (SWIR) is the reflected radiance in the 1-2.5 μm range. Mid wave infrared (MWIR) is the radiated radiance in the 3-5 μm range. This paper proposes a paint detection method using two infrared bands with different characteristics. Object detection is one of the issues in hyperspectral image (HSI). We use one dimensional convolution neural network (1D-CNN) and guided gradient-weighted class activation mapping (Guided Grad-CAM) for band selection. We make a 1D-CNN architecture and select bands using Guided Grad- CAM from well-trained 1D-CNN. Finally, paint is detected using selected bands. We use datasets included short wave infrared band (SWIR) and mid wave infrared band (MWIR).
The basic computational module of the technique is an old pattern recognition procedure: the mean-shift. In case of gray level feature domain, the spatial information of the target is lost when the background brightness histogram is the same as the target histogram. In this paper, we propose a new algorithm that is independent of background contrast by changing features from a conventional brightness based histogram to a temperature- based histogram. The proposed algorithm can track targets robustly regardless to target-background contrast. The experiment results demonstrate that the temperature-based Mean-Shift outperforms comparing with the brightness-based Mean-Shift when track a object with successive background variations.
Recently, autonomous driving based on multiple sensors has been studied actively in the field of automobiles and unmanned robots. Extrinsic parameter calibration is the first step to integrate the camera with Light Detection And Ranging (LiDAR). This paper proposes an extrinsic parameter calibration method using camera images and single 2D LiDAR points. The removal of infrared cut filter makes the line of laser scan points visible in the camera image. The scan line of the laser points on the calibration target is detected using edge matching in camera images, and the laser points are mapped to the coordinates of image with the initial value of the extrinsic parameters. We estimate the extrinsic parameters using edge matching and top-down method. The proposed method is verified by experiment according to the distance of the target.
In this paper, we propose a data-driven proposal and deep-learning based classification scheme for small target detection. Previous studies have shown feasible performance using conventional computer vision techniques such as spatial and temporal filters. However, those are handcrafted approaches and are not optimized due to the nature of the application fields. Recently, deep learning has shown excellent performance for many computer vision problems, which motivates the deep learning-based small target detection. The proposed data-driven proposal and convolutional neural network (DDP-CNN) can generate possible target locations through the data-driven proposal and final targets are recognized through the classification network. According to the experimental results using drone database, the DDP-CNN shows 91% of train accuracy and 0.85 of average precision (AP) of the target detection.
Automatic target recognition (ATR) is a traditionally challenging problem in military applications because of the wide range of infrared (IR) image variations and the limited number of training images. IR variations are caused by various three-dimensional target poses, noncooperative weather conditions (fog and rain), and difficult target acquisition environments. Recently, deep convolutional neural network-based approaches for RGB images (RGB-CNN) showed breakthrough performance in computer vision problems, such as object detection and classification. The direct use of RGB-CNN to the IR ATR problem fails to work because of the IR database problems (limited database size and IR image variations). An IR variation-reduced deep CNN (IVR-CNN) to cope with the problems is presented. The problem of limited IR database size is solved by a commercial thermal simulator (OKTAL-SE). The second problem of IR variations is mitigated by the proposed shifted ramp function-based intensity transformation. This can suppress the background and enhance the target contrast simultaneously. The experimental results on the synthesized IR images generated by the thermal simulator (OKTAL-SE) validated the feasibility of IVR-CNN for military ATR applications.
This paper presents a novel method to detect the remote pedestrians. After producing the human temperature based brightness enhancement image using the temperature data input, we generates the regions of interest (ROIs) by the multiscale contrast filtering based approach including the biased hysteresis threshold and clustering, remote pedestrian’s height, pixel area and central position information. Afterwards, we conduct local vertical and horizontal projection based ROI refinement and weak aspect ratio based ROI limitation to solve the problem of region expansion in the contrast filtering stage. Finally, we detect the remote pedestrians by validating the final ROIs using transfer learning with convolutional neural network (CNN) feature, following non-maximal suppression (NMS) with strong aspect ratio limitation to improve the detection performance. In the experimental results, we confirmed that the proposed contrast filtering and locally projected region based CNN (CFLP-CNN) outperforms the baseline method by 8% in term of logaveraged miss rate. Also, the proposed method is more effective than the baseline approach and the proposed method provides the better regions that are suitably adjusted to the shape and appearance of remote pedestrians, which makes it detect the pedestrian that didn’t find in the baseline approach and are able to help detect pedestrians by splitting the people group into a person.
A novel hyperspectral image normalization method was developed to make the hyperspectral profile invariant to illumination. The well-known band-ratio method shows unstable spectral profile to shades and highlights. The proposed minimum removal and maximum normalization method is simple but can reduce the spectral variations caused by shades and high-lights effectively, which leads to enhanced abnormal region detection performance in VNIR hyperspectral images.
IR Target detection is one of the key technologies in military applications. However, IR sensor has limitations of passive sensor such as low detection capability to weather and atmospheric effects. In recent years, sensor fusion is active research topic to overcome the limitations. Additional active SAR sensor is selected for sensor fusion because SAR sensor is robust to various weather conditions. The state-of-the-art detector, BMVT, has good performance in clear environment such as sky and sea background for small target. However, it shows poor performance when the target has extended size or the target is located in complex background such as ground-background with dense clutters. Therefore, we presents an improved ground target detection method based on the BMVT and Morphology filter (BMVT-M). The proposed algorithm consists of two parts: The first part is target enhancement based on the BMVT. The second part is clutter rejection and target enhancement based on the Morphology filter. In addition, conventional BMVT is not suitable to SAR image for target detection because SAR image has many shot noises. Therefore we apply a median filter before the BMVT in SAR image to suppress the shot noise. For the verification of the performance, experiments are performed in various cluttered backgrounds, such as ground, sea, and sky generated by the OKTAL-SE tool. The proposed algorithm showed upgraded detection performance than the BMVT in terms of detection rate and false alarm rate. Moreover, we discuss the applicability of the proposed method to the SAR and IR sensor fusion research.
This paper presents a novel camouflaged target detection method using spectral and spatial feature fusion. Conventional unsupervised learning methods using spectral information only can be feasible solutions. Such approaches, however, sometimes produce incorrect detection results because spatial information is not considered. This paper proposes a novel band feature selection method by considering both the spectral distance and spatial statistics after spectral normalization for illumination invariance. The statistical distance metric can generate candidate feature bands and further analysis of the spatial grouping property can trim the useless feature bands. Camouflaged targets can be detected better with less computational complexity by the spectral-spatial feature fusion.
This paper presents a novel coastal region detection method for infrared search and track. The coastal region detection is critical to home land security and ship defense. Detected coastal region information can be used to the design of target detector such as moving target detection and threshold setting. We can detect coastal regions robustly by combining the infrared image segmentation and sensor line-of-sight (LOS) information. The K-means-based image segmentation can provide initial region information and the sensor LOS information can predict the approximate horizon location in images. The evidence of coastal region is confirmed by contour extraction results. The experimental results on remote coasts and near coasts validate the robustness of the proposed coastal region detector.
This paper presents a concealed target detection based on the intersection kernel Support Vector Machine (SVM).
Hyperspectral imagers are widely used in the field of target detection and material analysis. In military applications, it
can be used to border protection, concealed target detection, reconnaissance and surveillance. If disguised enemies not
detected in advance, the damage of allies will be catastrophic by unexpected attack. Concealed object detection using
radar and terahertz method is widely used. However, these active techniques are easily exposed to the enemy. Electronic
Optical Counter Counter Measures (EOCCM) using hyperspectral imagers can be a feasible solution. We use the band
selected feature directly and the intersection kernel based SVM. Different materials show different spectrums although
they look similar in CCD camera. We propose novel concealed target detection method that consist of 4 step, Feature
band selection, Feature Extraction, SVM learning and target detection.
Infrared search and track is an important research area in military applications. Although there are a lot of works on small infrared target detection methods, we cannot apply them in real field due to high false alarm rate caused by clutters. This paper presents a novel target attribute extraction and machine learning-based target discrimination method. Eight kinds of target features are extracted and analyzed statistically. Learning-based classifiers such as SVM and Adaboost are developed and compared with conventional classifiers for real infrared images. In addition, the generalization capability is
also inspected for various infrared clutters.
Two-point correction-based nonuniformity correction of a scan-based long wave infrared camera can be successful in normal working environments. However, the method produces strong fixed-pattern noise in cluttered aerial environments. We analyze the mechanism of such phenomena and propose a nonuniformity correction method by combining a reference-based approach and a nonlinear filter-based residual offset estimation method. The nonlinear filtering process of row directional minimum filtering after two-point correction can remove the nonuniformity around cluttered aerial infrared images. Experimental results validate the feasibility of the proposed method.
This paper presents a separate spatio-temporal filter based small infrared target detection method to address the sea-based
infrared search and track (IRST) problem in dense sun-glint environment. It is critical to detect small infrared targets such
as sea-skimming missiles or asymmetric small ships for national defense. On the sea surface, sun-glint clutters degrade
the detection performance. Furthermore, if we have to detect true targets using only three images with a low frame rate
camera, then the problem is more difficult. We propose a novel three plot correlation filter and statistics based clutter
reduction method to achieve robust small target detection rate in dense sun-glint environment. We validate the robust
detection performance of the proposed method via real infrared test sequences including synthetic targets.
Infrared search and track pursues the detection of sea-skimming infrared targets incoming from long distance. This paper presents a realistic synthetic target simulator for the development of infrared target detection and tracking algorithms. The proposed simulator consists of a 2-D background modeling part and a 3-D infrared target modeling part. Real infrared background images are used for the realistic modeling of background. Synthetic infrared target images are obtained by the consecutive processing of 3-D geometric modeling and radiometric modeling of targets according to target types, target distances, and atmospheric transmissivity. The experimental results validate the realistic modeling of the proposed method by comparing real observation sequence data.
In an infrared search and tracking (IRST) system, the clustering procedure which merges target pixels into one cluster
requires larger computational load according to increasing clutters. In this paper, we propose a novel clustering method
based on weighted sub-sampling to reduce clustering time and obtain suitable cluster in cluttered environment. A
conventional sub-sampling method can reasonably reduce clustering time but cause large error, when obtaining cluster
center. However, our proposed clustering method perform sub-sampling and assign specific weights which is the number
of target pixels in sampling region to sub-sampled pixels to obtain suitable cluster center. After performing clustering
procedure, the cluster center position is properly obtained using sampled pixels and their weights in the cluster.
Therefore, our proposed method can not only reduce clustering time using a sub-sampling method, but also obtain proper
cluster center using our proposed weights. To validate our proposed method, experimental results for several infrared and
noise images are presented.
A mean shift algorithm has gained special attention in recent years due to its simplicity to enable real-time tracking.
However, the traditional mean shift tracking algorithm can fail to track target under occlusions. In this paper we propose
a novel technique which alleviates the limitation of mean shift tracking. Our algorithm employs the Kalman filter to
estimate the target dynamics information. Moreover, the proposed algorithm performs the background check process to
calculate the similarity which expresses how similar to target the background is. We then find the exact target position
combining the motion estimation by Kalman filter and the color based estimation by the mean shift algorithm based on
the similarity value. Therefore, the proposed algorithm can robustly track targets under several types of occlusion, while
the mean shift and mean shift-Kalman filter algorithms fail.
This paper presents a new small target detection method using scale invariant feature. Detecting small targets whose
sizes are varying is very important to automatic target detection in infrared search and track (IRST). The conventional
spatial filtering methods with fixed sized kernel show limited target detection performance for incoming targets. The
scale invariant target detection can be defined as searching for maxima in the 3D (x, y, and scale) representation of an
image with the Laplacian function. The scale invariant feature can detect different sizes of targets robustly. Experimental
results with real FLIR images show higher detection rate and lower false alarm rate than conventional methods.
Furthermore, the proposed method shows very low false alarms in scan-based IR images than conventional filters.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.