2.1.

Study Area

This study was conducted at the Corteva Agriscience⁴⁸ grain sorghum nursery in Manhattan, Kansas (39°9′13.69″ N, 96°40′1.2″ W). The soil type is classified as a reading silt loam with a slope between 0% and 2%. Twenty pioneer sorghum hybrids with different released years were planted in a randomized complete block design at a density of 13 plants ${\mathrm{m}}^{-2}$. Sorghum genotypes were pioneer hybrids spanning six decades of genetic selection (from 1963 until 2017). Each hybrid was replicated three times for a total of 60 plots. The hybrids were planted on June 8, 2019, in blocks with eight rows 5.3 m in length, with a row spacing of 0.76 m. Fertility and pest control measures were taken when necessary to ensure plant health throughout the season. Figure 1 shows the layout of the experiment. Both aerial imagery and ground truth data were taken, with aerial imagery corresponding to the flowering (August 9, 2019), soft dough (August 29, 2019), hard dough (September 11, 2019), and physiological maturity (September 26, 2019) stages and ground truth data being taken at flowering, soft dough, and physiological maturity. Ground truth measurements were collected over 2 to 3 days for each growth stage (flowering, August 5, August 9, August 14, 2019; soft dough, August 19, August 22, August 29, 2019; physiological maturity, September 25, October 3, 2019), as sorghum hybrids differed in maturity groups, therefore, reaching each growth stage at slightly different dates. For more information on flights and ground measurements, see Secs. 2.2 and 2.4, respectively.

Fig 1

Overview of field study in Manhattan, Kansas. Twenty sorghum genotype plots (indicated by the red boxes) were planted randomly within three blocks, indicated by the white squares.

2.2.

Data Collection

Image collection, processing, and data extraction followed a framework presented in Fig. 2. To collect aerial imagery, the UAS used for this study was a DJI Matrice 200⁴⁹ outfitted with a MicaSense RedEdge-MX multispectral camera.⁵⁰ The Matrice 200 design includes a DJI SkyPort integration kit, allowing the sensor to be quickly mounted on the camera gimbal. The RedEdge-MX sensor is a five-banded multispectral sensor, able to simultaneously capture five narrow bandwidths on both the visible and invisible electromagnetic spectrum (blue, 465 to 485 nm; green, 550 to 570 nm; red, 663 to 673 nm; RE, 712 to 722 nm; NIR, 820 to 860 nm). The camera can capture a spatial resolution of $8\mathrm{cm}{\text{pixel}}^{-1}$ at an altitude of 122 m and is equipped with a downwelling light sensor to detect changes in ambient lighting conditions during flight.

Fig 2

Workflow diagram of image collection, image processing, data extraction, and statistical analysis.

Prior to flights, ground control points (GCPs) were set around the perimeter of the study, and real time kinematic (RTK) GPS points were taken at each target. Throughout the growing season, four flights were flown in accordance with grain sorghum growth stage, primarily focusing on the reproductive period of the crop: flowering, soft dough, hard dough, and physiological maturity. To ensure lighting conditions were the same for all flights, each mission was flown under clear, sunny conditions within $\pm 2.5\mathrm{h}$ of solar noon. Due to close proximity to class D airspace, flight altitude was limited to 30 m, with a forward and side overlap set to 80% to ensure clear image orthomosaic production. Missions were planned and uploaded to the aircraft using the DJI Pilot⁵¹ application, with all missions conducted as GPS waypoint mapping missions. Waypoint missions produce flight plan of GPS points with altitude, flight area, and overlap settings factored into it, creating points to which the aircraft will travel to during flight. The RedEdge-MX camera was set to collect imagery every 2 s to ensure that enough images would be collected for processing. Following recommendations from the manufacturer, gain (ISO) and exposure (i.e., shutter speed and aperture) were left at an automatic setting, allowing for the camera to self-adjust these settings for the conditions of the flights.

To assess image data quality, the signal-to-noise ratio (SNR) was computed. The SNR is an important parameter of remote sensing imagery, determining the quality of imagery that the camera can collect.⁵² The SNR is commonly estimated by computing the mean pixel value to the standard deviation of a homogeneous area.^52,53 To test this, four Masonite boards with dimensions of $0.61\mathrm{m}\times 0.61\mathrm{m}$ were painted with a flat white, flat black, flat brown, and Italian Olive color. Each color was chosen to replicate certain homogeneous features that would be found within a sorghum field (white representing maximum reflectance values, black for shadows, brown for soil, and Italian Olive for vegetation). Images of the boards were captured on a sunny day under the same conditions as the flights, and the ratio was computed for each band. After converting each ratio to dB, each band’s SNR (over each panel) was averaged to produce an overall SNR, which was found to be 26 dB. Cameras over 20 dB are considered high-quality research cameras.^52,54

2.3.

Image Processing

To process the images, we used structure-from-motion photogrammetry. This technique results in the production of three-dimensional surface models and orthomosaic image generation by processing a series of overlapping images.⁵⁵ The main steps in processing the imagery included image orthorectification, data extraction, and exportation for statistical analysis. Images taken via UAS were saved on an onboard SD card as 16-bit TIFF files and were tagged with GPS coordinates. UAS images were uploaded into Agisoft Metashape (Version 1.5.3, St. Petersburg, Russia)⁵⁶ where images were layered into multispectral images. Radiometric calibration converting digital numbers into reflectance values was conducted using a MicaSense calibration panel. These images were then aligned by matching common features shared between photos, and a series of tie points were then computed into a sparse point cloud. RTK points were then placed onto each GCP to provide ground-reference points for more accurate alignment. A dense point cloud was then computed, followed by a digital elevation model, and finally an orthomosaic photo was produced. The end orthomosaic was produced as an unsigned 16-bit image.⁵⁷ This process was repeated for each flight during this study. Each orthomosaic had a spatial resolution between 2.2 and $2.3\mathrm{cm}/\text{pixel}$, and orthomosaics were then exported to ArcGIS Pro (Version 2.5, Redlands, California)⁵⁸ for further analysis.

2.4.

Ground Truth Measurements

Plant traits specifically related to biomass were chosen to be sampled through a destructive sampling process. These traits include fresh and dry aboveground whole plant biomass at varying growth stages during the reproductive period. Plant fractions were separated in leaves and stem during vegetative stages; and leaves, stem, and panicle (plus grain) during the reproductive stages. In addition, LAI at each sampling stage was measured, along with end-of-season yields.

Fresh biomass samples were taken at the flowering, soft dough, and physiological maturity growth stages, in the third and fourth rows of each plot. At each stage, all above-ground sorghum biomass within a 0.45-m length was sampled. Fresh weight of the samples was recorded, with subsequent partitioning of leaves, stems, and panicles. To determine LAI, three plants were chosen at random from the sampled plants, and each leaf from these plants was measured with a Licor 3100C leaf area meter.⁵⁹ To obtain dry biomass measurements, all samples were oven-dried at 65°C until constant weight was achieved. At physiological maturity, rows 6 and 7 were machine harvested for final grain yield. Fresh and dry biomass measurements were adjusted to $\mathrm{g}{\mathrm{m}}^{-2}$, LAI to leaf area ${\mathrm{m}}^{-2}$, and grain yield to $\mathrm{kg}{\mathrm{ha}}^{-1}$.

For senescence, a subsample of five consecutive plants were set aside for ground-truth measurements in each plot. These plants were set aside in the seventh row of each plot. Leaf senescence was defined as the decrease in green color due to the breakdown of leaf chloroplasts,⁶⁰ resulting in a certain percentage of leaf greenness loss for any given measurement stage. Each leaf on each plant was scored visually for leaf senescence on a scale of 100 (no visible senescence) to 0 (complete leaf mortality). Measurements were taken at soft dough, hard dough, and physiological maturity and were taken from the first eight leaves of the canopy with the flag leaf being the first leaf measured. To ensure leaf senescence was measured from maximum leaf greenness to maturity, an additional set of ground measurements were taken at $F$, and only leaves that scored 100 at this stage were used for analysis. This proved effective, as nearly all the top eight leaves showed no signs of senescence at this stage.

Prior to flights, elevated ground targets with dimensions of $0.3\mathrm{m}\times 0.3\mathrm{m}$ were placed between the fifth and sixth rows of each plot indicating the location of the ground-measured plants. Targets were placed exactly one row to the north of the plants to avoid casting shadows that would impede UAS data collection. In all cases, the width of the targets corresponded to the length of the measured plants, allowing for their locations to be identified when extracting data from the imagery.

2.5.

Image Data Analysis

From the original orthomosaic data, image background noise was then removed via image classification. A supervised, pixel-based classification was chosen to allow the greatest user input to define classes.⁶¹ Each image was classified into four classes: sorghum leaves, shadows, soil, and grain heads. The only exception to this classification schema was the imagery collected at the $F$ stage, which was classified as leaves, shadows, and soil because grain heads were not yet distinguishable from vegetation. As larger numbers of training samples are expected to increase supervised classification accuracies,⁶² 20 representative training sample polygons for each class were collected from each image.⁶¹ Separate training samples were generated for each orthomosaic to increase classification accuracies. A support vector machine algorithm was chosen for the classification in this case,⁶¹ and a new raster image was created. Accuracy assessments were conducted using a stratified random sampling procedure with 500 total points. Accuracies for sorghum leaf classifications ranged between 88% and 95% are correct for any given flight stage, indicating that vegetation was accurately classified when compared with a threshold of 85%.⁶³

After classification, each spectral band was extracted from the original orthomosaic. As outlined by Potgieter et al.,³⁶ each band was extracted and was normalized between 0 and 1. To establish plot boundaries, shape-file polygons were drawn around both the whole plots and designated senescence plants, and data were extracted from each region. The average pixel value for each spectral band was extracted, and VIs were computed based on significant findings of spectral bands (see Sec. 3). Each band/VI was extracted with a conditional statement, allowing for only data to be extracted from the “leaves” class to mask data from background features. The data were then exported for statistical analysis.

2.6.

Statistical Analysis

Statistical analysis was conducted using $R$.⁶⁴ To analyze spectral band relationships among biomass, yield, and LAI, simple correlation and regression analyses were used. The coefficient of determination ($r^2$) and root-mean-square error (RMSE) were also determined to verify the performance of the regression models.⁴⁵ For bands that were observed to be consistently significant, VIs were computed that composed of these bands to further investigate the relationships with ground-sampled data. From this data, either spectral bands or VIs were chosen for further analysis based on correlation and regression coefficient values. Hierarchical clustering was then used to cluster the significant spectral data into three distinct groups, based on Euclidean distance between clusters. For each growth stage, the mean NIR band value for each hybrid was hierarchically clustered, and each hybrid was divided into three groups (1 to 3). To determine if significant differences existed in ground-sampled traits as defined by each cluster group, a one-way analysis of variance (ANOVA) was conducted using the mean of each cluster, comparing the effect of cluster groups on plant traits. A Tukey honest significant difference test⁶⁵ was used to separate means from each cluster group.

To analyze senescence patterns, both correlation and simple regression analyses were used to determine the relation of spectral data to senescence scores at each growth stage. Each leaf that was scored for senescence was averaged into a “plant” score, with each plant score was averaged to form a “plot” score. The mean spectral band/VI score was correlated and regressed against the mean plot scores to determine significant relationships. In the same manner, the relationship between spectral data and postflowering senescence rates was determined. Senescence rates were determined by subtracting the mean senescence score at maturity from mean scores at flowering, and spectral change rates were determined in the same manner.³⁸ As previously, hierarchical clustering using the $R$ base package was used to divide each hybrid into three groups based on spectral responses, and a one-way ANOVA (comparing the effect of cluster on senescence/senescence rates) and Tukey HSD test conducted to determine significant differences in ground-measured scores. For all analyses, the ANOVAs and mean separations were computed using the “car”⁶⁶ and “agricolae”⁶⁷ $R$ packages, respectively. All significance levels were set at $\alpha =0.05$.

3.1.

Biomass, LAI, and Yield Traits

A variety of significant and nonsignificant results resulted from the correlation and regression analyses of the five spectral bands and field-measured plant traits (Table 1). Regression and residual plots for selected significantly correlated plant traits and spectral bands are shown in Fig. 3. The NIR band was seen to be the most consistently related to these traits across all growth stages. Weak to moderate relationships were observed between the NIR band and fresh biomass traits ($r=0.36$ to 0.61; $r^2=0.10$ to 0.37; $\mathrm{RMSE}=0.01$ to 0.02). Most notably, the highest relationships were seen with fresh leaf biomass at each stage ($r=0.51$ to 0.61; $r^2=0.26$ to 0.37). Weak to no significant relationships were observed with dry biomass at flowering, but total, stem, and leaf dry biomass were significantly related to the NIR band at the soft dough and maturity stages ($r=0.30$ to 0.46; $r^2=0.09$ to 0.21). For LAI, weak correlations were observed between the blue and NIR bands at flowering but were not significantly related at any other stage. As the blue band was significantly related to certain fresh and dry biomass traits for flowering and soft dough, these relationships were not statistically advantageous over the NIR band.

Table 1

Correlation and regression coefficients for spectral bands and sampled biomass traits, LAI, and grain yield.

Fig 3

Regression and residual plots for fresh whole plant biomass, fresh leaf biomass, and fresh whole stem biomass as regressed against the NIR, NIR, and blue bands, respectively. Data shown here are taken from the F growth stage.

Based on these overall results, the blue, green, red, and NIR band was chosen to create VIs to further investigate these relationships. The blue normalized difference vegetation index (BNDVI), green NDVI (GNDVI), NDVI, simple ratio (SR), and enhanced vegetation index (EVI) were all computed (Table 2). Results shown in Table 3 indicate significant relationships between the EVI and fresh biomass traits across all measured growth stages ($r=0.28$ to 0.50, $r^2=0.08$ to 0.25, $\mathrm{RMSE}=0.02$ to 0.04), with the exception of leaf fresh biomass at soft dough. In addition, significant relationships with fresh biomass traits were observed with the GNDVI at flowering ($r=0.39$, $r^2=0.15$ to 0.16, $\mathrm{RMSE}=0.01$) and with the SR at $M$ ($r=0.29$ to 0.34, $r^2=0.09$ to 0.11, $\mathrm{RMSE}=0.43$ to 0.44). Total, leaf, and stem dry biomass were significant for the EVI ($r=0.31$ to 0.39, $r^2=0.10$ to 0.15, $\mathrm{RMSE}=0.03$) and SR ($r=0.30$ to 0.33, $r^2=0.09$ to 0.11, $\mathrm{RMSE}=0.43$ to 0.44) at maturity. As a whole, it was determined that the NIR band was more consistently significant with biomass traits than the VIs, so it was selected for further analysis.

Table 2

Vegetation indices computed for biomass, LAI, and yield comparison.

Table 3

Correlation and regression coefficients for VIs and ground sampled biomass traits, LAI, and grain yield.

The mean ground-sampled trait for each cluster group was then analyzed via ANOVA and separated with the Tukey honest significant difference test for traits found significant in the previous correlation and regression analyses (Table 4). Within the assigned clusters, significant differences were seen with fresh and dry leaf biomass for all growth stages. No comparisons were made with total and stem dry biomass at flowering because it was not significantly related with NIR data at flowering, and no significant differences were seen between groups at soft dough or physiological maturity. For both total and stem fresh biomass, significant differences were only seen between groups at maturity. LAI mean separations at flowering revealed no significant differences, with no further comparisons made due to insignificant relationships with NIR data.

Table 4

Results of ANOVA and Tukey honest significant difference test for significant differences in biomass traits, LAI, and grain yield in clusters defined by hybrid NIR spectral responses.

As demonstrated in Table 4, it was discovered that the GNDVI ($r=0.39$ to 0.56, $r^2=0.15$ to 0.31, $\mathrm{RMSE}=0.01$ to 0.02) and the SR (0.30 to 0.56, $r^2=0.09$ to 0.32, $\mathrm{RMSE}=0.44$ to 0.48) were the most related to final grain yields throughout the postflowering grow stages. As multispectral data taken at flowering have been found to be the most correlated to final grain yield in previous grain sorghum studies,^45,73 the GNDVI was used for further analysis since it was found to be the most related to yield at flowering in our study ($r=0.56$, $r^2=0.31$, $\mathrm{RMSE}=0.01$). The mean GNDVI scores for each hybrid was clustered and divided into three distinct groups (Fig. 4). Through the ANOVA to determine yield differences, we found no significant differences between clusters ($F=0.001$, $p=0.97$). To determine if there were significant differences among yield that the cluster groups may have overlooked, an additional one-way ANOVA was conducted to test the effect of hybrid on yield without clustering. No significant yield differences were found among the hybrids in this study ($F=1.70$, $p=0.09$).

Fig 4

Dendrogram produced by hierarchical clustering the mean GNDVI data for each of the genotypes, resulting in the formation of three distinct cluster groups. Hybrids are denoted by year of release.

3.2.

Senescence

During data analysis, it was discovered that the panel we used to calibrate the images was highly reflective. There was a very high level of saturation in the NIR band in the boundaries drawn around the senescence plants, but there was very little saturation in the RGB bands. Therefore, VIs were computed using the RGB bands, including the excess green index (ExG), normalized difference index (NDI), excess green minus excess red index (ExGR), visible atmospherically resistant index (VARI), and the green leaf index (GLI) (Table 5).

Table 5

RGB VIs computed for comparison with ground-measured senescence scores.

When regressing the individual bands, there were no statistical differences observed with ground-truth scores (data not shown). However, significant relationships were discovered when regressing the RGB VIs chosen for this study (Table 6). At maturity, the most significant VIs were the NDI ($r=0.57$, $r^2=0.33$, $\mathrm{RMSE}=0.03$), VARI (0.60, $r^2=0.36$, $\mathrm{RMSE}=0.05$), and GLI ($r=0.48$, $r^2=0.23$, $\mathrm{RMSE}=0.03$), with the VARI being selected for further cluster analysis. Dividing the resulting dendrogram into three groups, the ANOVA analysis showed no statistical differences between clusters ($F=0.53$, $p=0.47$). Similarly, an ANOVA analysis comparing the effect of hybrid on senescence scores at maturity showed no statistical differences among scores at physiological maturity ($F=1.41$, $p=0.18$). In the same manner, senescence and VI change rates were analyzed, and the VARI was again found to be the most significantly related to postflowering senescence rates ($r=0.42$, $r^2=0.18$, $\mathrm{RMSE}=0.05$). ANOVA analysis showed no difference among clusters ($F=0.21$, $p=0.65$), and ANOVA analysis comparing the effect of hybrid on senescence rate revealed no significant differences in senescence rates in hybrids used for this study ($F=1.88$, $p=0.05$).

Table 6

Correlation and regression coefficients for RGB VIs and senescence scores.