|
1.IntroductionHyperspectral remote sensing data provide the advantage of detailed spectral information. However, the spectra of objects are often altered by interference from various noise sources during spectral measurement. Noise in the spectral dimension tends to conceal the true spectral characteristics of ground objects, affecting the accuracy of quantitative applications of hyperspectral images.1–3 Thus, spectral noise is one of the principal obstacles to further the application of hyperspectral remote sensing data. The need to develop effective methods to eliminate noise interference and recover the intrinsic spectral signatures of objects is urgent and significant. In recent years, several algorithms have been proposed to remove spectral noise. Based on their various background theories, these denoising algorithms can be generally divided into three categories: Savitzky–Golay (SG) filters, wavelet-transform (WT)-based filters, and mathematical-morphology filters. Based on the principle of least-squares fitting, SG filters steadily reduce noise using polynomials fitted to the original signal with a small error component.4–6 Krishnan and Seelamantula7 applied the SG filter to smooth real-world electrocardiogram signals and addressed the problem of selecting optimal length SG filter. Schettino et al.8 used an SG filter to detect current-transformer saturation in electrical power system engineering and found that the SG filter removed signal noise and improved the robustness of the detection process. However, the SG filter tended to distort the detailed spectral information.9 The WT is a method of localization analysis of time and frequency, which employs a linear combination of wavelet basis functions to represent a signal.10,11 WTs decompose the input signal into a series of distinctive frequencies that represent different characteristics of the signal and have the capability to reflect the nonstationarity of the signal. With the benefits of multiscale and multiresolution operation, WTs have been widely applied in many studies, including for hyperspectral image denoising,12,13 compression,14 classification,15 and image fusion or enhancement.16,17 Guo et al.18 employed a WT filter to the harmonic detection systems. Just once, transform and reconstruction were used to remove a variety of complex noise from harmonic spectrum. Experiments showed that the WT filter could be used for different harmonic detection systems. Rasti et al.12 used three-dimensional wavelets to remove noise from hyperspectral images, enhancing the spectral features and increasing the accuracy of image classification. Huang et al.19 used a wavelet threshold denoising algorithm to eliminate the noise of Pound–Drever–Hall signals, further improving the signal-to-noise ratio (SNR) of signal and static-strain measurement resolution. However, selecting the wavelet functions and the decomposition scale of the signal was complicated and difficult.20,21 In addition, signal drift causes obvious changes in wavelet coefficients,22 and noise removal using a WT generates pseudo-Gibbs phenomena at the spectrum’s endpoints, resulting in signal loss.23 Mathematical-morphology filters are a type of nonlinear filters found on set theory. The basic principle of mathematical-morphology filters is that geometric features and correlations between different parts of the signal are extracted with the aid of a structural element, which can be regarded as a probe acting on the signal. Based on purely mathematical foundations and first principles, these filters have been used extensively in the fields of image analysis and signal processing.24 Li et al.25 used multiscale morphology for weak microseismic signal detection. Their method decomposed a signal into multiscale components and thus obtained more signal information at selected scales. Dhane et al.26 performed five filters using mathematical-morphology operations for removing the impulse/random noise from camera-captured digital wound image. The test results demonstrated that the mathematical-morphology filter could obtain higher peak signal-to-noise ratio (PSNR) between the reference original and the filtered image. However, mathematical-morphology filters are unable to remove white noise effectively.27 The measured vegetation spectrum usually contains large- and small-amplitude noise, especially in the spectral range of water absorption. Single filter usually does not remove different kinds of noise effectively.27,28 To remove spectral noise, including small-amplitude noise (such as white noise) and large-amplitude noise (such as pulse noise) effectively, a hybrid method that combines GM filter with WT filter was used to remove noise from spectra in this study. The simulated noise spectrum based on standard vegetation spectrum collected from the United States Geological Survey (USGS) spectral library and the measured canopy spectra of winter wheat were denoised. The performance of the combination filter (CF) was assessed comprehensively through qualitative and quantitative analyses. 2.Materials and Methods2.1.Experimental Data2.1.1.Simulated noise spectrumOne standard vegetation spectrum of grass was collected from USGS spectral library and added with noise in the simulated experiments. The spectrum covers a spectral range of 395 to 2560 nm at spectral resolutions of 2 and 5 nm in the visible and near-infrared spectral range, respectively. To obtain simulated spectrum containing large- and small-amplitude noise, the salt-and-pepper noise and multiplicative noise29 were added to the standard vegetation spectrum together. The standard vegetation spectrum and simulated noise spectrum are shown in Fig. 1. 2.1.2.Measured canopy spectra of winter wheatThe canopy spectra of winter wheat and corresponding biophysical and biochemical parameters were used in this experiment. Measurement was conducted in 2001 at Beijing Precision Farming Experimental Station, located in the town of Xiao Tangshan, Changping District, Beijing (40°11′N, 116°27′E), China.30 All canopy spectra were measured from a height of 130 cm under clear sky conditions between 10:00 and 14:00 Beijing Local Time. An ASD FieldSpec Pro spectrometer (Analytical Spectral Devices, Boulder, Colorado) fitted with 25-deg field-of-view fiber optics was used to measure the canopy spectra of winter wheat. The spectral range of this spectrometer is from 350 to 2500 nm with sampling intervals of 1.4 nm between 350 and 1050 nm, and 2 nm between 1050 and 2500 nm, and with spectral resolutions of 3 nm at 700 nm and 10 nm at 1400 nm.31 To reduce the influence of random noise on spectrum, the final spectrum of each sample was determined from the mean of 20 measurements. Winter wheat samples were obtained immediately after completing wheat spectrum measurements. Then, biophysical and biochemical parameters were determined in the laboratory:32 wheat leaf water content (LWC) was calculated by drying fresh samples at 60°C in an oven. Leaf area index (LAI) was calculated using the dry weight method, by drying and weighing 50 to 100 leaves after measuring their leaf areas. Leaf area was then estimated according to the dry weight of leaves and calibrated using a CI-203 laser instrument. Chlorophyll (CHL) concentrations were measured using spectrophotometry. In total, 76 samples and the corresponding biophysical and biochemical parameters were used for the experiment. The specific measurement time and growth stages of wheat are shown in Table 1. Table 1Measurement time and corresponding wheat growth stages.
2.2.Theory of Mathematical MorphologyThe basic operational steps of mathematical morphology include dilation and erosion, whereas all other operations, such as opening and closing, are derived from these two operations. In addition, diverse combinations of opening and closing operations are used to form the traditional morphology filter and the GM. Suppose the discrete input signal is defined as and the structural element is defined as , then the erosion and dilation operations33 are defined as follows: Dilation: Erosion: where and .After determination of both erosion and dilation, the opening and closing operations can be obtained. The opening and closing operations are defined as follows: Opening: Closing: where ∘ stands for the opening operation and • for the closing operation. The erosion and opening operations retain negative impulses and remove positive impulses, and the dilation and closing operations follow the opposite pattern.34,35 Due to shrinkage of the opening operation, the output amplitude of the opening–closing filter process is small. However, expansion of the closing operation increases the output value of the closing–opening process.36 To obtain better denoising effects and eliminate the statistical deviation present in a single instance of opening–closing or closing–opening the filter, the following combination is usually adopted:37 where is the opening and closing filter, with the subscript OC representing an opening operation before a closing operation, and is the closing and opening filter, with the subscript CO representing an opening operation after a closing operation.The traditional morphology filter uses a single structural element to remove noise from the signal. However, the denoising effect of a single structural element is limited when there are multiple types and intensities of noise in the signal; therefore, GM filters were proposed.38 A GM filter is constructed by means of different structural elements, rather than just one kind of structural element, the definitions of which can be expressed as25 The average value of these two types of GM filters is thus defined as39 where and refer to the different structural elements, is the opening and closing filter with different structural elements, and is the closing and opening filter with different structural elements.2.3.Theory of the Wavelet-Transform-Based FilterIf the generating function satisfies Eq. (9) and belongs to , which is the space made up of all quadratic integration functions, is called a “base wavelet” or “mother wavelet.” The wavelet basis function is a set of function sequences obtained by calculating the scale and transition of the same mother function For the continuous case, the wavelet sequence is defined as where and , here, and are the scaling and translation factors, respectively.For the discrete case, the parameter takes the discrete value , and the parameter takes the discrete value , and thus, the wavelet sequence is defined as For arbitrary , continuous WT is defined as the inner product of the signal and wavelet function , with the specific expansion: where and is a complex conjugate function. In mathematics, the inner product represents the similarity of two functions, and therefore, the above equation can be illustrated as the degree of similarity between and .For , where meets the conditions of Eq. (9), and , , then the discrete WT is defined as if and , then the above equation represents the dyadic discrete WT.Based on multiresolution analysis, WT decomposes signal into two components, including detail and approximation coefficients.20 At each scale, the input signal is decomposed by high-pass filters to record high-frequency components and low-pass filters to extract the low-frequency components for the next scale decomposition. For example, as shown in Fig. 2, the signal was decomposed to four layers, and the signal can be represented as the sum of , , , , and , that is, . The , , , and are the approximate coefficients that represent the low-frequency components. The , , , and are the detail coefficients that represent the high-frequency components. The signal denoising process based on WT-based filter with soft or hard thresholding is as follows:
Hard threshold estimation: Soft threshold estimation: where , is the signal length, and is the standard deviation of noise (Fig. 3).2.4.Proposed MethodAs the opening operation can remove positive pulse noise and the closing operation can remove negative pulse noise, the GM filter effectively removes large-amplitude noise from the signal but it has a poor denoising effect on small-amplitude noise.28,41 The WT uses multiscale analysis of local signal features and enlarges subtle trends, removing small-amplitude noise (such as white noise) effectively. However, when dealing with large-amplitude noise, the WT decomposes partial noise into low-frequency information, and thus the noise is not removed completely.27 For these reasons, the mathematical-morphology filter and WT were combined for this study. A flowchart of the proposed method is presented in Fig. 4. The parameter settings have a strong influence on the effectiveness of filter. Previous studies have shown that ball or disk structural elements for mathematical-morphology filters,42,43 and the Symlet wavelet and Daubechies wavelet methods for WTs, effectively remove noise from the signal.44,45 In addition, a structural element with a small size can detect detailed signal information but has a weak denoising capacity. Although a large structural element has strong noise-removal ability, it lacks the ability to detect detailed information.46 The size of the structural element is usually determined according to experimental results and the characteristics of the original signal.47 In our experiments, ball, disk, diamond, rectangle, and square structural elements were used for comparative analysis. In addition, based on the Daubechies wavelet and Symlet wavelet methods, denoising results of the WT with four-layer wavelet decomposition are discussed below. To evaluate the denoising effects of different filters used in the simulated experiments, five evaluation indices between simulated noise spectrum and standard vegetation spectrum were calculated. To evaluate the application effectiveness of three filters on measured spectra, a subset of 51 canopy reflectance spectra with the corresponding measured biophysical and biochemical parameters were selected randomly from 76 total samples to develop models between vegetation indices and biophysical and biochemical parameters. The remaining 25 samples were used for accuracy validation. 2.5.Evaluation CriteriaIn the simulated experiments, the normalized correlation coefficient (NCC),48,49 coefficient of determination (), root mean square error (RMSE), SNR, and PSNR50 between the smoothed spectrum and standard vegetation spectrum were calculated to evaluate the denoising results. For measured canopy spectra of winter wheat, NCC was used to estimate the waveform similarity of the spectra before and after denoising, to assess the results qualitatively, whereas wheat biophysical and biochemical parameters were retrieved using different hyperspectral vegetation indices derived from denoised spectra to quantitatively assess the results. From seven bands in the visible and near-infrared (VNIR) spectral range and four bands in the shortwave infrared (SWIR) spectral range, seven hyperspectral vegetation indices were calculated. These vegetation indices are commonly used to retrieve biophysical and biochemical parameters. The normalized difference vegetation index (NDVI),51 structure-insensitive pigment index (SIPI),52 and modified chlorophyll absorption in reflectance index (MCARI2)53 were used for retrieval of LAI and CHL. The LWC was retrieved from the normalized difference spectral indices (),54 difference spectral indices ( and ),54 and modified normalized difference water index (NDWI*), which is obtained by multiplying the enhanced vegetation index (EVI) and normalized difference water index ().55–57 Specific definitions of the vegetation indices and evaluation indices used are as follows: where EVI is and is . where represents the spectral reflectance and its subscript indicates the wavelength (nm). For example, represents reflectance at a wavelength of 800 nm. where and are the wheat spectral reflectance of band before and after denoising, respectively. They also represent the spectral reflectance of band for smoothed spectrum and standard vegetation spectrum, respectively. Here, denotes the band number of spectrum. where and represent the vegetation index and measured biophysical and biochemical parameters, respectively. They also represent the retrieved and measured biophysical and biochemical parameters, respectively. Here, denotes the mean value of measured wheat biophysical and biochemical parameters. where and represent the measured and retrieved wheat biophysical and biochemical parameters, respectively. They also represent the standard vegetation spectrum and the smoothed spectrum, respectively. Here, is the number of wheat samples or band number of standard vegetation spectrum.3.Results and Analysis3.1.Experimental Results on Simulated Noise SpectrumThe simulated spectrum with noise was denoised by different filters. The comparison of denoising effects for different filters is shown in Table 2 and Fig. 5. SNS represents the simulated noise spectrum. and stand for a WT using three-layer Symlet wavelet function; and stand for a WT using three-layer Coiflet and Daubechies wavelet function, respectively. , , , and stand for a GM using ball-diamond, ball-square, disk-rectangle, and ball-line structural elements, respectively. stands for a combination filter using and , and so on. Table 2Comparison of denoising effects of different filters.
Note: SNS represents the simulated noise spectrum; WT1 and WT4 stand for a WT using three-layer Symlet wavelet function, WT2 and WT3 stand for a WT using three-layer Coiflet and Daubechies wavelet function, respectively. GM1, GM2, GM3, and GM4 stand for a GM using ball-diamond, ball-square, disk-rectangle, and ball-line structural elements, respectively. CF1 stands for a combination filter using GM1 and WT1, and so on. As shown in Fig. 5, although WT removed small-amplitude noise effectively and the denoised spectra from WT are relatively smooth, the large-amplitude noise was not removed completely. GM removed large-amplitude noise, but the smoothed spectrum showed many broken lines. The denoised spectrum from CF removed both large- and small-amplitude noise, maintaining the waveform and smoothness of spectrum better. As shown in Table 2, based on the standard vegetation spectrum, the SNR and PSNR calculated using simulated noise spectrum are 13.769 and 19.223 dB, respectively. Moreover, the , RMSE, and NCC are 0.901, 0.077, and 0.979 before denoising, respectively. The NCC and were improved slightly by the three filters, which demonstrate that the denoised spectrum becomes more similar to standard vegetation spectrum. Compared with GM and WT, the SNR and PSNR calculated using CF were highest, reaching 28.886 and 34.593, respectively, with 25.723 and 31.411 for GM, 21.525 and 27.136 for WT. In addition, the RMSE for CF was lowest with 0.013, whereas RMSE for GM and WT were 0.019 and 0.031. Thus, CF has better denoising ability, improving SNR and PSNR of simulated noise spectrum significantly and maintaining the waveform and smoothness of standard vegetation spectrum in the meantime. 3.2.Qualitative Analysis of Noise Removal from Measured SpectraThe sources of spectral noise include variations in light intensity and sensor errors.58 When incident energy is strongly absorbed by atmospheric water vapor and becomes too weak, the sensor is not able to detect the signal. Thus, field spectra usually contain more noise than spectra measured in the laboratory.59 The original canopy spectrum (OS) of winter wheat is shown in Fig. 6. This wheat spectrum contains little noise in the VNIR spectral range (400 to 1100 nm) and the spectrum is relatively smooth, as the spectrometer has a high SNR in this spectral range. However, due to the low SNR of the spectrometer and strong water absorption of regions around 1400, 1900, and 2000 nm,60 the spectrum in the SWIR spectral range (1100 to 2500 nm) contains large-amplitude noise, which appears as “bulges” or “burrs.” Since the OS of winter wheat contains less noise in the VNIR spectral range, NCC values for the CF, GM, and WT are very close to 1 between the original and denoised spectra. For all three filters, the denoised and original spectra overlap in the VNIR spectral range, indicating that none of the filters have a negative influence on the original data in the VNIR spectral range. A comparison of the spectra before and after denoising in different spectral ranges is shown in Fig. 7. Due to the spectra before and after denoising overlay in the VNIR spectral range, to present the result clearly, only the smoothed spectrum from the CF and original spectrum are compared and analyzed in Fig. 7(a). As shown in Fig. 7(a), the smoothed spectrum from the CF and original spectrum almost align within non-noisy spectral regions, reflecting the ability of the CF to maintain spectral signatures. After denoising, the red-edge region (680 to 760 nm) of the wheat spectrum was well preserved. Furthermore, the CF also removed obvious burrs in the form of peaks and valleys in the SWIR spectral range. Figure 7 shows that all filters could remove large-amplitude noise in the SWIR spectral range, restoring the intrinsic features of original spectrum. However, there were some differences between the denoised spectra obtained using different filters. Compared with the original spectrum, the denoised spectra from the CF and WT are relatively smooth. On the other hand, the denoised spectra from the GM showed many broken lines, especially in the spectral range of 1800 to 1950 nm. The GM with a disk structural element produced more obvious broken lines under various parameter settings. In addition, as shown in Figs. 7(b) and 7(c), the spectral values of the smoothed spectrum obtained using the WT became negative at some wavelengths, which is inconsistent with the measured reflectance spectrum whose values are always positive. Therefore, although all denoising methods maintained the original spectrum stably in the VNIR spectral range, the CF shows better performance with large-amplitude noise in the SWIR spectral range, removing such noise while maintaining or restoring the spectral shape and value. 3.3.Quantitative Analysis of Noise Removal from Measured SpectraThe biophysical and biochemical parameters of winter wheat were retrieved using the denoised spectra. Regression analysis was conducted between the measured and retrieved biophysical and biochemical parameters of winter wheat to verify the denoising accuracy. The experimental results are listed in Tables 3Table 4–Table 56, where different subscripts on the filter names represent different parameters settings. OS means that the original spectra were used to calculate related vegetation indices. and stand for a GM using disk-rectangle and ball-square structural elements, respectively. and stand for a WT using the four-layer Daubechies and Symlet wavelet function, respectively. stands for a CF using with . stands for a CF using with . Table 3Accuracy of the models developed between vegetation index, LAI, and CHL using different filters.
Note: OS represents the original spectra; WT4 and WT5 stand for a WT using four-layer Daubechies and Symlet wavelet function, respectively. GM4 and GM5 stand for a GM using disk-rectangle and ball-square structural elements, respectively. CF4 stands for a combination filter using GM4 and WT4, and so on. Table 4Retrieval accuracy of LAI and CHL using different filters.
Note: OS represents the original spectra; WT4 and WT5 stand for a WT using four-layer Daubechies and Symlet wavelet function, respectively. GM4 and GM5 stand for a GM using disk-rectangle and ball-square structural elements, respectively. CF4 stands for a combination filter using GM4 and WT4, and so on. Table 5Accuracy of developed model between vegetation index and LWC for different filters.
Note: OS represents the original spectra; WT4 and WT5 stand for a WT using four-layer Daubechies and Symlet wavelet function, respectively. GM4 and GM5 stand for a GM using disk-rectangle and ball-square structural elements, respectively. CF4 stands for a combination filter using GM4 and WT4, and so on. Table 6Retrieval accuracy of LWC for different filters.
Note: OS represents the original spectra; WT4 and WT5 stand for a WT using four-layer Daubechies and Symlet wavelet function, respectively. GM4 and GM5 stand for a GM using disk-rectangle and ball-square structural elements, respectively. CF4 stands for a combination filter using GM4 and WT4, and so on. 3.3.1.Retrieval of leaf area index and chlorophyllSome models of the regression analysis between NDVI and CHL are shown in Table 3 and Fig. 8. The relationships between the retrieved and measured values of CHL are presented in Table 4 and Fig. 9. Based on regression analysis, NDVI-LAI, MCARI2-LAI, NDVI-CHL, and SIPI-CHL exhibit a significant logarithmic relationship.61–64 As shown in Fig. 8, the distribution of 51 sample points after denoising was nearly the same as that before denoising. The accuracy of the model was not enhanced significantly when using spectra denoised with the generalized morphology or WT, and the remained basically unchanged from that before denoising. The relationship between retrieved and measured LAI and CHL showed the same pattern. However, for the CF, the accuracy of the developed model was slightly improved. As shown in Table 3, the of NDVI-LAI, MCARI2-LAI, SIPI-CHL, and NDVI-CHL using denoised spectra from the CF reached 0.5235, 0.4726, 0.3442, and 0.4801, respectively, whereas the corresponding values were 0.5100, 0.4390, 0.3249, and 0.4587 using the original spectra. The relationship between the retrieved and measured values of biophysical and biochemical parameters was also enhanced, as shown in Table 4 and Fig. 9. The coefficients of determination between the vegetation indices and biophysical and biochemical parameters remained unchanged or improved slightly, primarily for the following reasons: (1) the bands used in NDVI are in the VNIR spectral range, which is rarely influenced by noise and (2) the CF removed small-amplitude noise in the VNIR spectral range, so the was slightly improved. This finding shows that the proposed algorithm is capable of removing small-amplitude spectral noise, and thereby restoring spectral information to some extent. 3.3.2.Retrieval of wheat leaf water contentThe specific results of retrieving LWC were listed in Tables 5 and 6. Taking the as an example, the regression models were established between and LWC, as shown in Fig. 10. Figure 11 shows the LWC retrieval results using . The and LWC exhibited a significant logarithmic relationship, as shown in Fig. 10.65–68 The distribution of sample points after denoising was more concentrated than before. Before denoising, the between and LWC was 0.428. After denoising with three filters, the reached 0.549, 0.555, and 0.622, corresponding to the WT, GM, and CF, respectively. Among these, the relationship between and LWC was most obviously improved after denoising with the CF. As shown in Fig. 11, there was no strong relationship between retrieved and measured values of LWC before denoising, as the and RMSE were only 0.364 and 0.027, respectively. After denoising with the three filters, this relationship improved. For the WT, the maximum value was 0.478 with an RMSE of 0.023. Using the GM, the values of and RMSE obtained were 0.504 and 0.022, respectively. For the CF, and RMSE between the retrieved and measured values of LWC were 0.611 and 0.018, respectively, representing a strong improvement over those obtained before denoising. These results show that the regression analysis models obtained using the CF were more reliable than those obtained before denoising and that the CF can provide higher retrieval accuracy for LWC. Furthermore, the relationships between LWC and other vegetation indices, including , , and , were also improved after denoising with the CF. The reason for this improvement is that these vegetation indices are calculated mainly using bands in the SWIR spectral range, which contain large-amplitude noise. After denoising with the CF, large-amplitude noise is removed effectively and the spectral characteristics are restored. 4.DiscussionFiltering parameters have a strong influence on the performance of a filter. According to the characteristics of different signals, many experiments have been carried out to select the optimal parameters for the three major filter types. For the mathematical-morphology filter, structural elements are important parameters. Our experimental results show that mathematical-morphology filters with ball, square, rectangular, or disk structural elements can remove large-amplitude noise, but the denoised spectra are distorted to varying degrees, appearing as many broken lines. Optimization of the WT is focused on selection of wavelet functions and the number of wavelet decomposition layers. Two different wavelet functions, the Daubechies and Symlet wavelet, were used to denoise the wheat canopy spectra. Each wavelet function utilized four-layer wavelet decomposition of the original spectra. As shown in Figs. 12(a) and 12(b), canopy spectra denoised using the WT with certain parameters become negative around 1380 and 1850 nm. This result does not agree with the measured reflectance spectra of winter wheat, which always have values . Moreover, this phenomenon is relatively rare in previous research on noise removal from simulated data.69 This finding also indicates that there are differences between the simulated data and measured field spectra. The same parameters used with the WT were adopted for the CF. As shown in Figs. 12(c) and 12(d), when the CF was used to remove spectral noise, the denoised spectra were always positive. Also, the spectral curves are smoother than those obtained with the GM, without the appearance of broken lines in denoised spectra. This result shows that the CF enhanced the accuracy of denoised spectra and achieved more effective noise removal. No single filter can eliminate noise from a signal completely, particularly in the case of complex noise patterns. However, the CF is able to remove large-amplitude spectral noise in our experiments, which is consistent with previous research. For example, the CF could denoise photoplethysmography signals efficiently, retaining the wave form of the signal well.22 This filter removes not only large-amplitude noise but also small-amplitude noise.70 Therefore, it is suitable to apply the CF to removal of spectral noise from winter wheat for restoration of the original spectral characteristics. 5.ConclusionSpectral noise interferes with the absorption characteristics of objects, limiting the quantitative applications of hyperspectral remote sensing data. A CF was proposed in this study to remove spectral noise. Two commonly used filters, the GM and WT, were used for comparative analysis. The simulated experiments were conducted using simulated spectrum obtained by adding noise to USGS standard vegetation spectrum. Five evaluation indices were calculated to make comparison of different denoising results. Moreover, the measured canopy spectra of winter wheat were denoised by three filters. To evaluate the denoising effects of different filters, regression analysis models were established between vegetation indices and measured biophysical and biochemical parameters. From the viewpoint of practical application of hyperspectral data, the accuracy of the models was validated by retrieving the biophysical and biochemical parameters. In the simulated experiments, compared with the GM and WT, the CF removed both large- and small-amplitude noise, improving SNR and PSNR of simulated noise spectrum greatly and maintaining the waveform and smoothness of standard vegetation spectrum in the meantime. For measured canopy spectra of winter wheat, the VNIR spectral range (400 to 1100 nm) contained little noise, and all three filters were able to maintain the detailed features of the original spectra. The NCCs in this range were close to 1. In contrast, large-amplitude noise is present in the SWIR spectral range (1100 to 2500 nm). All three filters can suppress large-amplitude noise effectively, confirming their ability to remove spectral noise. However, there are differences between the denoising results of the three filters. The spectra denoised using the GM lack smoothness, exhibiting many twists and turns. Although the WT can produce relatively smooth spectra, negative values present in the denoised spectra are inconsistent with typical vegetation spectra. The CF can remove noise in the SWIR spectral range effectively and exhibits smooth denoised spectra, improving the retrieval accuracy of LWC. Compared with the of 0.428 before denoising, the of the model developed between and LWC is improved to 0.622 using the CF, to 0.555 with the GM, and to 0.549 with the WT. In addition, the and RMSE values between retrieved and measured LWC are 0.364 and 0.027, respectively, before denoising, whereas the corresponding values are 0.504 and 0.022 after denoising with the GM, and 0.478 and 0.023 with the WT, respectively. Compared with the GM and WT, the CF obtains the highest retrieval accuracy of LWC, with an of 0.611 and RMSE of 0.018. However, this research also identified some shortcomings that must be further explored and addressed. The proposed method focuses on spectral noise removal. Further applications of the CF should be studied and more validation experiments designed. The applicability of this method to other applications, such as hyperspectral image denoising, needs to be confirmed, including whether it might improve the accuracy of spectral unmixing or image classification through spectral noise removal. AcknowledgmentsThis work was supported by the National Key R&D Program on Monitoring, Early Warning and Prevention of Major National Disaster (No. 2017YFC1502802), the National Natural Science Foundation of China (No. 41671360), the Central Public Welfare Project (No. 2018SYIAEZD1), and the National Key R&D Program (No. 2016YFB0500502). ReferencesQ. Wang et al.,
“Low rank constraint and spatial spectral total variation for hyperspectral image mixed denoising,”
Signal Process., 142 11
–26
(2018). https://doi.org/10.1016/j.sigpro.2017.06.012 Google Scholar
W. Xie and Y. Li,
“Hyperspectral imagery denoising by deep learning with trainable nonlinearity function,”
IEEE Geosci. Remote Sens. Lett., 14
(11), 1963
–1967
(2017). https://doi.org/10.1109/LGRS.2017.2743738 Google Scholar
Y. Chen et al.,
“Denoising of hyperspectral images using nonconvex low rank matrix approximation,”
IEEE Trans. Geosci. Remote Sens., 55
(9), 5366
–5380
(2017). https://doi.org/10.1109/TGRS.2017.2706326 IGRSD2 0196-2892 Google Scholar
H. L. Kennedy,
“Maximally flat IIR smoothers with repeated poles and a prescribed delay,”
IEEE Trans. Signal Process., 64
(19), 4975
–4986
(2016). https://doi.org/10.1109/TSP.2016.2593689 ITPRED 1053-587X Google Scholar
Y. Liu et al.,
“Applications of Savitzky–Golay filter for seismic random noise reduction,”
Acta Geophys., 64
(1), 101
–124
(2016). https://doi.org/10.1515/acgeo-2015-0062 Google Scholar
J.-H. Kim et al.,
“Performance evaluation of a two-dimensional Savitzky–Golay filter for image smoothing applications,”
in Proc. 5th Int. Conf. Electron. Commun. Networks,
309
–316
(2016). Google Scholar
S. R. Krishnan and C. S. Seelamantula,
“On the selection of optimum Savitzky–Golay filters,”
IEEE Trans. Signal Process., 61
(2), 380
–391
(2013). https://doi.org/10.1109/TSP.2012.2225055 ITPRED 1053-587X Google Scholar
B. M. Schettino, C. A. Duque and P. M. Silveira,
“Current-transformer saturation detection using Savitzky–Golay filter,”
IEEE Trans. Power Delivery, 31
(3), 1400
–1401
(2016). https://doi.org/10.1109/TPWRD.2016.2521327 ITPDE5 0885-8977 Google Scholar
P. J. Curran et al.,
“Reflectance spectroscopy of fresh whole leaves for the estimation of chemical concentration,”
Remote Sens. Environ., 39
(2), 153
–166
(1992). https://doi.org/10.1016/0034-4257(92)90133-5 Google Scholar
A. S. Antolín and R. A. Zalik,
“Some smooth compactly supported tight framelets associated to the quincunx matrix,”
J. Math. Anal. Appl., 437
(1), 35
–50
(2016). https://doi.org/10.1016/j.jmaa.2015.12.022 JMANAK 0022-247X Google Scholar
S. Mallat and W. L. Hwang,
“Singularity detection and processing with wavelets,”
IEEE Trans. Inf. Theory, 38
(2), 617
–643
(1992). https://doi.org/10.1109/18.119727 IETTAW 0018-9448 Google Scholar
B. Rasti et al.,
“Hyperspectral image denoising using 3D wavelets,”
in Int. Conf. Geosci. Remote Sens. Symp.,
1349
–1352
(2012). https://doi.org/10.1109/IGARSS.2012.6351286 Google Scholar
D. Fang et al.,
“A sparsity-based InSAR phase denoising algorithm using nonlocal wavelet shrinkage,”
Remote Sens., 8
(10), 830
–848
(2016). https://doi.org/10.3390/rs8100830 Google Scholar
B. Li, R. H. Jiao and Y. C. Li,
“Fast adaptive wavelet for remote sensing image compression,”
J. Comput. Sci. Technol., 22
(5), 770
–778
(2007). https://doi.org/10.1007/s11390-007-9086-7 JCTEEM 1000-9000 Google Scholar
X. Huang and L. Zhang,
“A multiscale urban complexity index based on 3D wavelet transform for spectral-spatial feature extraction and classification: an evaluation on the 8-channel WorldView-2 imagery,”
Int. J. Remote Sens., 33
(8), 2641
–2656
(2012). https://doi.org/10.1080/01431161.2011.614287 IJSEDK 0143-1161 Google Scholar
K. Pyka,
“Wavelet-based local contrast enhancement for satellite, aerial and close range images,”
Remote Sens., 9
(1), 25
(2017). https://doi.org/10.3390/rs9010025 Google Scholar
N. Han, J. Hu and W. Zhang,
“Multi-spectral and SAR images fusion via Mallat and À trous wavelet transform,”
in Int. Conf. Geoinformatics,
1
–4
(2010). https://doi.org/10.1109/GEOINFORMATICS.2010.5567653 Google Scholar
Y. Guo et al.,
“The noise filtering and baseline correction for harmonic spectrum based on wavelet transform,”
Spectrosc. Spectral Anal., 33
(8), 2172
–2176
(2013). https://doi.org/10.3964/j.issn.1000-0593(2013)08-2172-05 Google Scholar
W. Huang et al.,
“-phase-shifted FBG for high-resolution static-strain measurement based on wavelet threshold denoising algorithm,”
J. Lightwave Technol., 32
(22), 4294
–4300
(2014). https://doi.org/10.1109/JLT.2014.2354413 JLTEDG 0733-8724 Google Scholar
V. D. Hoang,
“Wavelet-based spectral analysis,”
TrAC Trends Anal. Chem., 62 144
–153
(2014). https://doi.org/10.1016/j.trac.2014.07.010 Google Scholar
Y. M. Chen et al.,
“A data-driven threshold for wavelet sliding window denoising in mechanical fault detection,”
Sci. China Technol. Sci., 57
(3), 589
–597
(2014). https://doi.org/10.1007/s11431-013-5451-7 Google Scholar
T. Bai et al.,
“A PPG signal de-noising method based on the DTCWT and the morphological filtering,”
in Int. Conf. Signal-Image Technol. Internet-Based Syst.,
503
–506
(2016). https://doi.org/10.1109/SITIS.2016.85 Google Scholar
H. Z. M. Shafri and P. M. Mather,
“Wavelet shrinkage in noise removal of hyperspectral remote sensing data,”
Am. J. Appl. Sci., 2
(7), 1169
–1173
(2005). https://doi.org/10.3844/ajassp.2005.1169.1173 Google Scholar
A. Hu and L. Xiang,
“An optimal selection method for morphological filter’s parameters and its application in bearing fault diagnosis,”
J. Mech. Sci. Technol., 30
(3), 1055
–1063
(2016). https://doi.org/10.1007/s12206-016-0208-4 1738-494X Google Scholar
H. Li et al.,
“Weak signal detection using multiscale morphology in microseismic monitoring,”
J. Appl. Geophys., 133 39
–49
(2016). https://doi.org/10.1016/j.jappgeo.2016.07.015 JAGPEA 0926-9851 Google Scholar
D. M. Dhane et al.,
“Selection of optimal denoising filter using quality assessment for potentially lethal optical wound images,”
Procedia Comput. Sci., 58 438
–446
(2015). https://doi.org/10.1016/j.procs.2015.08.059 Google Scholar
Y. Chan and X. Miao,
“Signal filtering method for early detection of short circuit fault in low voltage power distribution system,”
Electr. Eng., 17
(8), 9
–13
(2016). https://doi.org/10.3969/j.issn.1673-3800.2016.08.002 Google Scholar
F. Wang and Y. Gan,
“Study of traveling-wave singularity detection algorithm based on mathematical morphology,”
Electr. Eng., 16
(4), 26
–29
(2015). https://doi.org/10.3969/j.issn.1673-3800.2015.04.007 Google Scholar
Z. Chen et al.,
“Denoising method of spectral signal with multiplicative and additive mixed random noises,”
Acta Opt. Sin., 37
(7), 0730001
(2017). https://doi.org/10.3788/AOS GUXUDC 0253-2239 Google Scholar
L. Liu et al.,
“Estimating winter wheat plant water content using red edge parameters,”
Int. J. Remote Sens., 25
(17), 3331
–3342
(2004). https://doi.org/10.1080/01431160310001654365 IJSEDK 0143-1161 Google Scholar
W. Huang et al.,
“Managing grain protein content by remote sensing in winter wheat,”
Proc. SPIE, 6742 67420W
(2007). https://doi.org/10.1117/12.737713 PSISDG 0277-786X Google Scholar
X. Zhang et al.,
“Study on spectral indices of MODIS for wheat growth monitoring,”
J. Image Graphics, 10
(4), 420
–424
(2005). https://doi.org/10.3969/j.issn.1006-8961.2005.04.004 Google Scholar
J. Serra,
“Morphological filtering: an overiew,”
Signal Process., 38
(1), 3
–11
(1994). Google Scholar
S. Osman and W. Wang,
“A morphological Hilbert–Huang transform technique for bearing fault detection,”
IEEE Trans. Instrum. Meas., 65
(11), 2646
–2656
(2016). https://doi.org/10.1109/TIM.2016.2598019 IEIMAO 0018-9456 Google Scholar
B. Li et al.,
“A weighted multi-scale morphological gradient filter for rolling element bearing fault detection,”
ISA Trans., 50
(4), 599
–608
(2011). https://doi.org/10.1016/j.isatra.2011.06.003 ISATAZ 0019-0578 Google Scholar
W. Zhang, X. Zhou and Y. Lin,
“Application of morphological filter in pulse noise removing of vibration signal,”
in Congr. Image Signal Process.,
132
–135
(2008). https://doi.org/10.1109/CISP.2008.202 Google Scholar
X. Mao et al.,
“Three-dimensional morphological analysis method for geologic bodies and its parallel implementation,”
Comput. Geosci., 96 11
–22
(2016). https://doi.org/10.1016/j.cageo.2016.07.004 Google Scholar
J. F. Liu et al.,
“The noise suppression of magnetotelluric sounding data based on multiple structural elements mathematical morphology filtering,”
Geophys. Geochem. Explor., 38
(1), 109
–114
(2014). https://doi.org/10.11720/j.issn.1000-8918.2014.1.20 Google Scholar
P. Zhang et al.,
“Self adjusting composite cascade morphology filter algorithm and its application,”
J. Vib. Meas. Diagn., 35
(3), 459
–463
(2015). https://doi.org/10.16450/j.cnki.issn.1004-6801.2015.03.009 Google Scholar
H. He and Y. Tan,
“A novel adaptive wavelet thresholding with identical correlation shrinkage function for ECG noise removal,”
Chin. J. Electron., 27
(3), 507
–513
(2018). https://doi.org/10.1049/cje.2018.02.006 CHJEEW 1022-4653 Google Scholar
R. Hu, S. Jiang and D. Li,
“A mathematical morphological filtering approach to remove the noise of MCG-signals,”
Mod. Sci. Instrum., 15
(2), 54
–55
(2005). https://doi.org/10.3969/j.issn.1003-8892.2005.02.026 Google Scholar
J. Li et al.,
“Magnetotelluric data processing based on combined generalized morphological filter,”
J. Cent. South Univ., 45
(1), 173
–185
(2014). Google Scholar
L. Tang et al.,
“The application of mathematical morphological filtering to the noise suppression of audio magnetotelluric sounding data,”
Chin. J. Eng. Geophys., 42
(11), 33
–37
(2013). https://doi.org/10.3969/j.issn.1672-7940.2013.04.018 Google Scholar
X. Yang, W. Zhang and Y. Yang,
“Denoising technology of radar life signal based on lifting wavelet transform,”
Acta Opt. Sin., 34
(3), 0328003
(2014). https://doi.org/10.3788/AOS GUXUDC 0253-2239 Google Scholar
H. Jun and M. You-Liang,
“Quantitative study on the selection of wavelet functions for the de-noising of ECG signal,”
Inf. Electron. Eng., 8
(3), 286
–289
(2010). https://doi.org/10.3969/j.issn.1672-2892.2010.03.010 Google Scholar
F. Wang and L. H. Xia,
“Research on application of mathematical morphology in edge detection of high resolution remote sensing image,”
Geomatics Spat. Inf. Technol., 32
(1), 50
–53
(2006). https://doi.org/10.3969/j.issn.1672-5867.2006.02.007 Google Scholar
L. Hui et al.,
“Edge detection method of remote sensing images based on mathematical morphology of multi-structure elements,”
Chin. Geogr. Sci., 14
(3), 263
–268
(2004). https://doi.org/10.1007/s11769-003-0057-9 Google Scholar
X. Wan et al.,
“Illumination-invariant image matching for autonomous UAV localisation based on optical sensing,”
ISPRS J. Photogramm. Remote Sens., 119 198
–213
(2016). https://doi.org/10.1016/j.isprsjprs.2016.05.016 IRSEE9 0924-2716 Google Scholar
G. Bao et al.,
“Multi-template matching algorithm for cucumber recognition in natural environment,”
Comput. Electron. Agric., 127 754
–762
(2016). https://doi.org/10.1016/j.compag.2016.08.001 CEAGE6 0168-1699 Google Scholar
M. F. Zhao et al.,
“Research on denoising of UV-Vis spectral data for water quality detection with compressed sensing theory based on wavelet transform,”
Spectrosc. Spectral Anal., 38
(3), 844
–850
(2018). https://doi.org/10.3964/j.issn.1000-0593(2018)03-0844-07 Google Scholar
X. Zhang, W. C. Qi and W. C. Sun,
“Research on vegetation spectrum denoising method based on mathematical morphology filtering,”
Remote Sens. Technol. Appl., 31
(5), 846
–854
(2016). https://doi.org/10.11873/j.issn.1004-0323.2016.5.0846 Google Scholar
L. Liang et al.,
“Chlorophyll content inversion with hyperspectral technology for wheat canopy based on support vector regression algorithm,”
Trans. Chin. Soc. Agric. Eng., 28
(20), 162
–171
(2012). https://doi.org/10.3969/j.issn.1002-6819.2012.20.022 Google Scholar
D. Haboudane et al.,
“Hyperspectral vegetation indices and novel algorithms for predicting green LAI of crop canopies: modeling and validation in the context of precision agriculture,”
Remote Sens. Environ., 90
(3), 337
–352
(2004). https://doi.org/10.1016/j.rse.2003.12.013 Google Scholar
X. J. Liu et al.,
“Monitoring leaf water content based on hyperspectra in rice,”
Sci. Agric. Sin., 45
(3), 435
–442
(2012). https://doi.org/10.3864/j.issn.0578-1752.2012.03.004 Google Scholar
S. Wang et al.,
“Responses of net primary productivity to phenological dynamics in the Tibetan Plateau, China,”
Agric. For. Meteorol., 232 235
–246
(2017). https://doi.org/10.1016/j.agrformet.2016.08.020 0168-1923 Google Scholar
Z. Tang et al.,
“Assessing Nebraska playa wetland inundation status during 1985–2015 using Landsat data and Google Earth Engine,”
Environ. Monit. Assess., 188
(12), 654
(2016). https://doi.org/10.1007/s10661-016-5664-x EMASDH 0167-6369 Google Scholar
X. J. Cheng et al.,
“Estimating canopy water content in wheat based on new vegetation water index,”
Spectrosc. Spectral Anal., 34
(12), 3391
–3396
(2014). https://doi.org/10.3964/j.issn.1000-0593(2014)12-3391-06 Google Scholar
C. Vaiphasa,
“Consideration of smoothing techniques for hyperspectral remote sensing,”
ISPRS J. Photogramm. Remote Sens., 60
(2), 91
–99
(2006). https://doi.org/10.1016/j.isprsjprs.2005.11.002 IRSEE9 0924-2716 Google Scholar
A. K. Skidmore,
“Smoothing vegetation spectra with wavelets,”
Int. J. Remote Sens., 25
(6), 1167
–1184
(2004). https://doi.org/10.1080/0143116031000115085 IJSEDK 0143-1161 Google Scholar
J. Wang et al.,
“A wavelet-based area parameter for indirectly estimating copper concentration in Carex leaves from canopy reflectance,”
Remote Sens., 7
(11), 15340
–15360
(2015). https://doi.org/10.3390/rs71115340 Google Scholar
Z. H. Du,
“Study on the scaling and the establishment of the hyper-spectral inversion model of chlorophyll content in rice,”
Beijing
(2015). Google Scholar
J. Jiang et al.,
“Analyzing the spatial scaling bias of rice leaf area index from hyperspectral data using wavelet-fractal technique,”
IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 8
(6), 3068
–3080
(2015). https://doi.org/10.1109/JSTARS.2014.2346251 Google Scholar
G. Lin et al.,
“Retrieving winter wheat leaf area index based on unmanned aerial vehicle hyperspectral remote sensing,”
Trans. Chin. Soc. Agric. Eng., 32
(22), 113
–120
(2016). https://doi.org/10.11975/j.issn.1002-6819.2016.22.016 Google Scholar
H. L. Jiang et al.,
“Research on spectral scale effect in the estimation of vegetation leaf chlorophyll content,”
Spectrosc. Spectral Anal., 36
(1), 169
–176
(2016). https://doi.org/10.3964/j.issn.1000-0593(2016)01-0169-08 Google Scholar
L. Xuan,
“Vegetation spectral feature analysis and water content inversion based on hyperspectral remote sensing image,”
(2016). Google Scholar
X. Liu et al.,
“Prediction of moisture content in loess using continuum-removed method,”
Wuhan Daxue Xuebao, 42
(5), 661
–668
(2017). https://doi.org/10.13203/j.whugis20150006 Google Scholar
J. Li, W. Song and N. F. University,
“Water content model for strawberry leaves with spectral signature,”
J. Northeast For. Res., 44
(1), 72
–74
(2016). https://doi.org/10.13759/j.cnki.dlxb.2016.01.002 Google Scholar
X. Cheng et al.,
“Inferred water content of winter wheat based on ground hyperspectral and remote sensing data of TM5,”
J. Triticeae Crops, 34
(2), 227
–233
(2014). https://doi.org/10.7606/j.issn.1009-1041.2014.02.14 Google Scholar
W. Zheng et al.,
“Partial discharge signal obtain based on adaptive wavelet with mathematical morphology,”
Phys. Procedia, 24 912
–917
(2012). https://doi.org/10.1016/j.phpro.2012.02.137 PPHRCK 1875-3892 Google Scholar
H. Li et al.,
“Research on spectrum denoising methods based on the combination of wavelet package transformation and mathematical morphology,”
Spectrosc. Spectral Anal., 30
(3), 644
–648
(2010). https://doi.org/10.3964/j.issn.1000-0593(2010)03-0644-05 Google Scholar
BiographyXia Zhang is a professor at the Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences (CAS). Her current research interests include hyperspectral image information extraction and classification. |