1.

Introduction

One aspiration of precision agriculture is to minimize costs while maximizing crop yield by allowing farmers to identify problem areas in the field and deploy mitigation tactics.¹ Remote sensing increases the efficiency of this process by gathering data associated with crop health quickly and automating processes for crop health visual display and evaluation.² Satellite imagery is not collected with the frequency required for precision agriculture, and atmospheric effects or cloud cover can have dramatic effects on the quality of data produced. Ground-based (tractor-mounted) sensors achieve high resolution, but operational and data efficiency measures may be much reduced compared with other approaches. High-resolution imagery is necessary for more precise data, and unmanned aircraft systems (UAS) are able to fly much lower and can achieve higher temporal and spatial resolutions than satellite imagery or imagery captured using manned aircraft.^1,3

Over the past decade and into the present, an increasing amount of research has been done on the use of remote sensing with UAS and other platforms for detecting measures of crop health. Typically, the calculation of vegetation indices, such as normalized difference vegetation index (NDVI), is used to estimate the amount of photosynthetically active radiation absorbed by plants, which has been found to be a good indicator of overall health.⁴ The potential of remote sensing with UAS to predict crop yield and estimate biomass using vegetation indices has been well explored in the scientific literature. UAS-derived NDVI predicted grain yield in winter wheat at a high level of accuracy.⁵ Similarly, UAS-based monitoring of a sunflower crop found significant linear regressions between NDVI and grain yield as well as biomass and nitrogen content.⁶ Research has also discovered good correlation between NDVI and yield or biomass in rice^7,8 and grain sorghum.⁹

Using NDVI to estimate crop health can be complicated by the effects of ground that is visible through gaps in the canopy. Early in a crop’s growth stage, the canopy can be very sparse and mean NDVI values may be deceptively low due to the amount of soil reflectance included in the NDVI calculation. Vegetation indices can be tuned to reduce soil background effects. One study compared multiple soil-adjusted indices with NDVI in an area with low vegetation cover.¹⁰ The simplest index is the soil-adjusted vegetation index (SAVI), which uses a correcting factor, $L$, an estimated value between 0 and 1 based on the vegetation density.¹¹ More complex indices such as the transformed SAVI utilize the slope and intercept of an equation (also known as the soil line) fitted through a plot of red versus near-infrared (NIR) reflectance for various bare soil conditions. In the study, these indices were not found to outperform soil-unadjusted vegetation indices at vegetation cover below 30%. Furthermore, the requirement of establishing a soil line for applying algorithms for soil adjustment can limit its use for the commercial agricultural applications.

The use of photogrammetric height determination for crops by UAS is not as heavily researched as the use of vegetation indices, but some studies have been done using three-dimensional (3-D) point clouds to obtain an estimate of crop height and correlate it with yield and biomass. These point clouds are generated through a photogrammetric technique called structure-from-motion, or SfM.¹² Research introduced by Ref. 13 using terrestrial laser scanning suggests a reasonably accurate method for deriving plant height from the point cloud data is a difference approach that calculates crop height by taking the difference between a digital surface model (DSM) of the crop canopy generated from the point cloud and a digital elevation model (DEM) of the bare-ground surface generated from a secondary data source or from ground points filtered out of the point cloud. Studies have also applied this approach to derive crop height from UAS-SfM derived point cloud data.^14–16 The combination of vegetation indices and UAS-based canopy surface models have been shown to estimate biomass in barley and corn more accurately than vegetation indices alone.^15,16 The results from these studies suggest that crop height metrics can be used as a complement to NDVI and potentially as a surrogate for assessing crop health under certain conditions.

Small UAS are also becoming increasingly affordable and accessible to the modern farmer with new regulations permitting more flexible use in different countries, and preprogrammed flight plans enable nonpilots to easily conduct flights for mapping purposes.¹ Powerful software can automatically stitch raw imagery into orthomosaics and generate point clouds via SfM with geoferencing based on a ground control point (GCP) network and position information from onboard GPS stored in each image’s exchangeable image file (EXIF) format data. Consumer-grade cameras can be modified with a filter to convert a visible channel to an NIR channel, forgoing the need for more expensive four-band color-infrared cameras. A study¹⁷ comparing field measurements from more advanced multispectral sensors to consumer-grade cameras found that both types were equivalent under ambient light conditions but that unstable illumination and angular variation in reflectance in sunny conditions could impact performance for consumer-grade UAS-mounted cameras. A high percentage of image endlap and sidelap can help reduce this effect. Reference 17 recommends using at least 75% endlap and 60% sidelap. The algorithms for brightness correction, including a bidirectional reflectance distribution function (BRDF), are also recommended for creating reflectance maps on which to base NDVI calculation.¹⁷

Sorghum, Sorghum bicolor (L.) Moench, is grown throughout much of the world in small to large acreage production systems. The grain is used for animal feed, human consumption, and other purposes. Other sorghum varieties are grown for animal forage.¹⁸ The crop is also increasingly grown for the production of ethanol. In South Texas, where the study here was conducted, the sugarcane aphid, Melanaphis sacchari (Zehntner) (Hemiptera: Aphididae), has begun to infest sorghum crops.¹⁹ The insect directly disrupts sorghum physiology and photosynthesis and also produces large amounts of honeydew that serves as a growth media for black mold, which also disrupts the plant’s photosynthesis.²⁰ Through this process, insect infestation may disrupt the spectral response of the plant, potentially allowing remote sensors to detect this form of plant stress.²¹ Insect damage has previously been identifiable via multispectral and hyperspectral reflectance on other types of crops.²² For example, wheat aphid damage has been identified using benchtop-based spectral measurement of individual leaves.²³ Aerial detection of the insect infestation is a more complicated process. One study on UAS-based detection of potato beetle infestation found no correlation between plot-scale vegetation indices, but it was able to detect insect damage using object-based feature extraction.²⁴ A significant negative correlation has been found between plant injury rating due to aphids and NDVI in grain sorghum fields using an airborne multispectral imaging system.²⁵ Unfortunately, aerial detection of plant stress may require substantial, large-scale plant stress after the window of deploying a mitigation measure has passed. Insect detection at the plant level may be capable of detecting damage before it is too severe to correct, but it may also be inefficient and computationally intense. There is the additional problem of differentiating between insect disruption of photosynthesis and other stressors that have a similar impact such as nutrient deficiencies.

The only study of which the authors are aware that used UAS remote sensing to detect insect stress in sorghum investigated impacts of white grubs.²⁶ The method used an unsupervised clustering approach to segment the field into three zones (decimated, transition, and healthy) based on the RGB pixel values. The method showed potential but was developed for severely damaged and readily identifiable stress. Although such a method could be feasible for detection of severe levels of aphid stress on sorghum, at that point it would be too severe for mitigation. Furthermore, the study did not assess the utility of NIR vegetation indices or 3-D crop structure metrics that can be extracted from the UAS acquired imagery; most notably, the study did not examine UAS for detecting variable levels of aphid stress with its unique challenges.

The purpose of this study is to investigate the utility of a small UAS equipped with a modified consumer-grade camera for NIR capture to derive NDVI and crop height measures using SfM photogrammetry for assessment of aphid stress on sorghum grain development. The study area is a small plot field experiment subjected to highly variable aphid pressure. Pregrowth, midseason, and late-season flights were conducted; NDVI and crop height metrics were extracted from acquired imagery, and statistical analysis was performed to assess correlation to aphid presence and yield estimation. NDVI derived from these flights was also compared with NDVI data collected from a manned aircraft flight conducted midseason as well as NDVI collected late in the season from a modified ground crop sprayer tractor-mounted with multispectral sensors. Additionally, the filtering methods for soil signal and lower canopy (height only) were utilized and compared among one another to provide more informative UAS estimates of mean per-plot NDVI and canopy height values, respectively.

2.

Methodology

2.1.

Study Area

The study was conducted at research fields at the Texas A&M AgriLife Research and Extension Center at Corpus Christi, Texas, USA, during the 2015 growing season. The research complex consists of several hundred acres where a variety of crop research experiments are conducted. For this project, the specific study area was an acre field where sorghum was planted at varying times and sugarcane aphid population densities were manipulated with insecticide (Fig. 1). The study consisted of two plantings of a grain sorghum hybrid known to be susceptible to the aphid (DKS 53-67, DeKalb/Monsanto, St. Louis, Missouri). Each planting was planted in its own section, containing 20 plots. Each plot had four rows of sorghum planted 0.96 m apart from one another. The plots were 40 feet (12.19 m) long and 12 feet (3.66 m) wide. The early planting was seeded on April 2, 2015, and the late planting was seeded on May 4, 2015.

Fig. 1

Study area map of sorghum plots. Early- and late-planted plots are highlighted in the center. Source imagery is a false color composite orthomosaic of images captured on June 10, 2015.

2.4.

Image Processing and Metric Extraction

2.4.1.

Photogrammetric processing

Over 250 raw images were captured in each of the four flights covering the entire mapped area including the study site. These were transferred to a hard drive for postprocessing. Following each flight, the raw image data set was processed using SfM photogrammetry. Traditional photogrammetry requires metric cameras precisely calibrated. However, metric cameras are expensive and not conducive for widespread use of small UAS for surveying and mapping applications. In contrast, SfM photogrammetry exploits information from multiple overlapping images to extract 3-D object information and negate the need for precise camera calibration. There are many SfM-based commercial and open-source software solutions for processing of UAS collected imagery for mapping purposes. In this work, Pix4Dmapper Pro (Pix4D SA, 1015 Lausanne, Switzerland) was utilized to process the imagery. The basic SfM processing workflow is summarized as follows:²⁷

• 1. Find corresponding features (key points) from all overlapping images using a feature-matching algorithm, such as a variant of the scale invariant feature transform (SIFT). SIFT is a well-known computer vision algorithm that allows for feature detection regardless of scale, camera rotations, camera perspectives, and changes in illumination.²⁸

• 2. Key points as well as approximate values of the image geoposition provided by the UAS autopilot (onboard GPS) are input into a bundle block adjustment to reconstruct the exact position and orientation of the camera for each image. This process also simultaneously calibrates the camera for lens distortion and other intrinsic parameters. Input of GCPs can be done here and results readjusted.

• 3. Based on this reconstruction, the matching points are verified and their 3-D coordinates calculated. Densification of the sparse point cloud is then performed to increase the point density. The resultant set of 3-D points is used to generate a triangulated irregular network (TIN) and obtain a DSM.

• 4. The DSM is then used to project every image pixel and to calculate a geometrically corrected (orthorectified) image mosaic with uniform scale.

Standard geospatial data outputs from the UAS-SfM processing workflow are a 3-D point cloud, DSM, and orthomosaic. Here, these outputs were used to generate two main products for each flight: reflectance maps from which to extract NDVI and crop height maps. GIS analysis was then used to extract mean NDVI and crop height values from this data by averaging the values within polygons created for each data row in each plot. More details on these products are provided below. Figure 2 summarizes the image processing workflow.

Fig. 2

SfM image processing workflow used to generate NDVI and crop height maps.

2.4.2.

NDVI extraction

When computing a band ratio index such as NDVI, the interest is in the variation of reflectance of the terrain across the different bands. However, the pixel value in the raw imagery and derived orthomosaics are influenced by both the variation in illumination and sensor response. Regarding illumination, the spectral signature of available incoming light can vary within a scene due to atmospheric conditions, sun-view geometry, and shadow of the plants, among other factors. Additionally, the total amount of irradiance can vary due to changing conditions such as moving clouds, time of day, or seasonal variation (e.g., winter versus summer). This effect can be estimated and corrected using radiometric calibration targets placed within the scene with known albedo properties for the wavelength bands of interest captured by the specific sensor or camera. Adjusting for variability in the amount of irradiance can also be performed using an upward looking sensor mounted on the UAS or on the ground. Radiometric calibration is important when the objective is to obtain absolute or stable NDVI measurements for comparison across surveys conducted at different times, which are likely subjected to different spectral illumination conditions. It is generally not as important for NDVI computation within a given flight due to the normalization of the formula and assuming stable illumination conditions; however, variability in local illumination during a flight can potentially impact NDVI in addition to surface reflectance. In regard to sensor effects on pixel brightness, such factors include sensor sensitivity to different wavelengths, ISO/aperture/shutter speed settings, optical vignetting effects (variability in amount of received light on the sensor due to lens and sensor geometry with edges having less brightness), gamma corrections, spectral overlap, and other factors.

It is important to mention that the above factors do not fully account for the reflectance function of a target as viewed by a sensor. For a complete characterization, one would need to look at the BRDF for a given illumination and viewing geometry. In practice, this is not feasible, particularly for small UAS surveys with numerous images, and it is not necessary. Simplified corrections for brightness variability due to illumination and sensor effects provide useful results for deriving vegetation indices.¹⁷ Furthermore, high image overlap from the UAS flight enables weighted pixel averaging to help compensate for differences in viewing geometry.

For this work, reflectance maps were created in Pix4D to correct sensor effects on pixel brightness from image to image and generate a more spatially stabilized NDVI value for a given flight. Raw imagery was captured in 16-bit TIFF format and represented linear encoded values that were not smoothed. To generate a reflectance map, the processing software uses parameters present in the EXIF data of the images to correct for some of the factors mentioned above that may vary from image to image including ISO sensitivity, aperture setting, shutter speed, and vignetting. Although radiometric calibration may have reduced a small degree of variability due to the spectral signature of illumination during the flight, it was determined to not be necessary for the purposes of this study. The objective here is to compare relative NDVI values over the study plots for a given flight extracted directly from the sensor using a standard UAS-SfM processing workflow common to most applications. Furthermore, each UAS flight targeted similar times of day with consistent illumination conditions to minimize variability within and across flights.

Three separate reflectance maps were created for each band captured by the sensor—red, green, and NIR. An NDVI map was then generated using a raster calculation based on the well-known band ratio formula:⁴ $\mathrm{NDVI}=\frac{\mathrm{NIR}-\text{red}}{\mathrm{NIR}+\text{red}}$. NDVI maps were generated for each flight except the bare-ground flight on May 1, 2015.

Three sets of NDVI values were obtained for each flight. The first method used no soil signal filtering; the second method filtered out low NDVI values that might correspond to ground (soil) pixels; and the third method utilized a raster calculation to remove NIR band values below a percentage of the maximum NIR reflectance value before calculating NDVI. For the second method, NDVI values below 0.50 were excluded from the per-plot averages. This threshold will exclude bare ground but may exclude some unhealthy plants as well. The third method excludes pixels with low NIR reflectance values that fall below a certain percentage (threshold) of the maximum NIR reflectance. These maximum values are evaluated as being the peak values found in the entire field reflectance map, which varies with each flight. Finding this threshold requires manual examination by iteratively increasing the percentage cutoff value while assessing how many “mostly soil” pixels are removed and “mostly vegetation” pixels are retained. Differences in ambient conditions (e.g., sunlight and atmosphere), imaging geometry (e.g., camera perspective and overlap), and reflectance of the soil and vegetation influence the choice of an effective threshold for each flight. For the June 10 and July 29, 2015, flights, NIR values below 25% of maximum NIR reflectance were excluded. For the June 23, 2015, flight, the threshold was 50% of maximum NIR reflectance. Figure 3 shows NDVI maps for all three NIR camera flights. NDVI values were extracted from the NDVI maps and scaled to the plot level metrics using pairs of row polygons as explained in Sec. 2.5.

Fig. 3

Colorized crop height and NDVI map products generated by the processes described in Fig. 2. (a), (b), and (c) images are from imagery collected on June 10, June 23, and July 29, 2015, respectively. Crop height and NDVI maps for imagery collected on May 1, 2015, is not included because this flight used an RGB camera to generate a bare-earth DEM. Row polygons are delineated in the NDVI maps on the right side.

2.4.3.

Crop height extraction

Each pixel within the study area had an overlap of at least five images and flights targeted low wind conditions to reduce parallax from moving vegetation. This allowed for robust and dense generation of 3-D point cloud data over the sorghum plots (see Table 1). A noise filtering algorithm was applied after point clouds were generated to reduce spurious points. This algorithm finds points that are isolated or have three or fewer other points within a $3\times 3\times 3$ grid of cells. Points that have three or fewer other points in their surrounding $3\times 3\times 3$ grid (with the respective point in the center cell) are removed. Cell size was set at 25 cm to compensate for average point density and to ensure a reasonable buffer above which we do not expect isolated points.

Points within each SfM photogrammetric point cloud were triangulated using a Delaunay triangulation to create a seamless TIN. The TIN was then rasterized into a DSM using linear spatial interpolation based on the TIN’s triangular facets. Because the average point spacing across flights is around 0.1 m (see Table 1), a step size of 0.1 m was used to obtain the highest usable resolution and ensure a consistent resolution from flight to flight for crop height extraction. Recall that the May 1, 2015, flight was conducted during pregrowth with the ground exposed. This flight was used to generate a bare-earth DEM for computing crop height. Raster calculations were used to differentiate the mid- and late-season flights from the DEM derived from the May 1, 2015, flight to provide an estimate of crop height (see Fig. 3).

The resulting crop surface models were further processed in three methods, similar to the NDVI extraction methods. The first method was not filtered for soil or lower canopy. The second method filtered out values under 0.5 m, and the third method filtered values under 0.25 m. Filtering out low values ensures that only the upper leaves of the canopy are included in the per-plot mean height values. To filter the low values, pixels below a certain threshold were set to null. These null values were not included in the per-plot averages. Crop height values were extracted from the crop height maps and scaled to the plot level metrics using pairs of row polygons as explained below.

3.

Results

3.1.

Comparison between Different Soil Signal Filtering Methods

VMRs in NDVI were compared among the three flights for the late planting [Fig. 4(a)]. The late planting had the most visible ground between the plants and filtering for low values had the most impact on the mean NDVI for this planting. The most effective removal in most cases was using filtering of low NIR values with filtering at 25% of max NIR reflectance for the June 10 and July 29, 2015, flights and 50% for the June 23, 2015, flight. Removing values with low NIR reflectance proved to be about as effective as removing low NDVI values. While the final flight shows a lower VMR for method 2 with filtering at an NDVI value of 0.5, there remained an unacceptable amount of unhealthy vegetation removed with this method. A lower NDVI threshold may remedy this problem.

Fig. 4

Comparison between VMRs in late planting among three methods of soil signal filtering for (a) mean NDVI and (b) mean crop height.

Soil signal filtering was most beneficial in situations where the most soil was visible. Without filtering out soil, a plot’s NDVI will appear very low despite the plants being apparently healthy. This is evident on the June 10, 2015, flight. At this point, the plants were young and the canopy was sparse and fairly uniform among all plots in the late planting. As observed in Fig. 4(a), the VMR for the June 10, 2015, flight is very similar for both the NIR and NDVI filtering methods and much lower than VMR without filtering. This result suggests that these methods are interchangeable for plots with sparse canopies. During the denser canopy of the June 23, 2015, flight, NIR filtering still showed the lowest VMR. Therefore, for the purposes of comparing NDVI to field data, we chose filtered NIR reflectance to extract NDVI.

VMRs were also compared among three methods in measuring mean crop height for the three flights in the late planting [Fig. 4(b)]. The first method used no soil signal filtering. The second filtered out values below 0.5 m in the DSM, and the third filtered values below 0.25 m. The estimated canopy height (based on field observations) ranged from about 0.6 to 1.0 m.

The third method of filtering values below 0.25 m effectively removed soil, but it included lower leaves that were visible through gaps in the canopy. The most effective filtering method was the second method of filtering values below 0.5 m. This removes not only soil but also the lower leaves of the plants to provide a more accurate estimation of mean canopy height. Method 2 had the lowest VMRs, and the difference was most noticeable in the flights over sparse canopies. Therefore, method 2 was used to estimate canopy height for examining correlations between crop height and field data.

3.2.

Comparison between Tractor-Mounted, Manned Aerial, and Unmanned Aerial Surveys

Imagery from the manned flight was not corrected for brightness; therefore, NDVI values appear low compared with those gathered by the UAS. The normalization of the NDVI calculation, however, ensures that relative NDVI measurements are comparable across platforms. The manned aerial flight was conducted 2 days after the June 23, 2015, UAS flight. NDVI per-plot values extracted from the manned flight data correlated very strongly with the UAS-derived NDVI in the early planting ($p<0.0001$, Table 2). A positive linear relationship was detected with higher NDVI values obtained from the UAS NDVI extraction that included soil signal filtering ($\text{slope}=1.6$, $R^2=0.91$, Fig. 5). In the late planting, however, no significant correlation was detected. This is likely due to differing methods of NDVI extraction. While the polygon method employed with the UAS-derived NDVI extraction kept soil between the rows from being included in mean NDVI, the manned flight utilized a method that included soil, causing large variations between per-plot mean values. This explanation is further substantiated by comparing the unfiltered UAS-derived NDVI to the manned aerial survey, which showed an improved positive correlation ($r=0.532$, $p=0.0340$, Table 2).

Fig. 5

Linear regression models comparing different platforms. (a) Comparison of UAS-derived NDVI to manned flight-derived NDVI in the early planting on June 23, 2015. (b) Comparison of UAS-derived NDVI to ground rig-derived NDVI in the late planting on July 29, 2015. Shading represents the 95% confidence interval of the regression.

Table 2

Pearson’s correlation coefficients as well as p values for NDVI comparison among tractor-mounted, manned aerial, and unmanned aerial surveys. Method 2, low NDVI filtered; method 3, low NIR filtered.

	NDVIMNDa		NDVIGNDb
	Early	Late	Early	Late
${\mathrm{NDVI}}_{\mathrm{UAS}}$ (unfiltered)	0.971	0.532	0.123	0.869
$p$ value	$1.20\times {10}^{-12}$	0.034	0.604	$1.25\times {10}^{-5}$
${\mathrm{NDVI}}_{\mathrm{UAS}}$ (method 2)	0.971	0.393	0.164	0.833
$p$ value	$1.23\times {10}^{-12}$	0.132	0.491	$6.19\times {10}^{-5}$
${\mathrm{NDVI}}_{\mathrm{UAS}}$ (method 3)	0.955	0.181	0.135	0.844
$p$ value	$6.50\times {10}^{-11}$	0.503	0.570	$3.92\times {10}^{-5}$

^aThe manned aerial flight conducted 2 days after the June 23, 2015, UAS flight.

^bThe tractor-mounted phenotyping system measured NDVI the same day as the July 29, 2015, UAS flight.

The tractor-mounted phenotyping system measured NDVI the same day as the July 29, 2015, UAS flight. The NDVI values derived from the phenotyping system correlated strongly in the late planting ($p<1\times {10}^{-4}$, Table 2), but no significant correlation was found in the early planting. The linear regression model showed a positive relationship and comparable values, except that the range of the UAS NDVI values was wider than those of the phenotyping system NDVI ($\text{slope}=0.67$, $R^2=0.71$, Fig. 5). Unlike the manned flight, the phenotyping system did not correlate well with any of the soil signal filtering methods for the early planting. The phenotyping system only calculates NDVI from a few points per plot along the centerline of the row, but the points are necessarily centered on plants spaced along the row. Because fewer points are used to calculate mean NDVI, an outlying point can greatly shift the mean, compared with a platform gathering up to 1200 points. Table 2 provides complete Pearson’s correlation coefficients as well as $p$ values for NDVI comparison between tractor-mounted, manned aerial, and unmanned aerial surveys, where ${\mathrm{NDVI}}_{\mathrm{UAS}}$, ${\mathrm{NDVI}}_{\mathrm{MND}}$, and ${\mathrm{NDVI}}_{\mathrm{GND}}$ represent the UAS-derived NDVI based on the soil signal filtering methods [Fig. 4(a)], manned aircraft-derived NDVI, and phenotyping system measured NDVI, respectively.

3.3.

UAS-Derived Measures and Field Measures Comparison

The late planting showed the highest variability in plant stress due to aphid damage and had the highest aphid density for the season, cumulating in a significant relationship between NDVI and maximum aphid density during the July 29, 2015, late season flight (Fig. 6). For this flight, NDVI values ranged from 0.49 to 0.66 in the late planting and as aphid density increased, NDVI values decreased ($r=-0.63$, $p<0.01$). Maximum aphid density also correlated negatively with mean crop height in the late season flight (Fig. 6), but the strength of correlation was less than that of NDVI ($r=-0.485$, $p=0.0571$). The NDVI values from the July 29, 2015, flight in the late planting had a very low positive correlation with yield ($p=0.0720$, Fig. 6). Mean crop height showed a similar trend with a weaker correlation to yield relative to NDVI (Fig. 6). The strength of the relationship between UAS-derived crop height and UAS-derived NDVI was strongly dependent on the stage of growth, with the weakest correlation observed for the end-of-season July flight. Plant response was generally low and did not appear to correlate with any UAS-derived measures of plant health. Pearson’s correlation coefficients of UAS-derived metrics and field measurements for both plantings in all flights are given in Table 3.

Fig. 6

Linear regression charts comparing yield and aphid density to UAS-derived NDVI and crop height in the late planting for the July 29, 2015, flight. Shading represents the 95% confidence interval of the regression.

Table 3

Pearson’s correlation coefficients for UAS-derived metrics (height and NDVI) in early and late plantings.

	June 10, 2015: late	June 10, 2015: early	June 23, 2015: late	June 23, 2015: early	July 29, 2015: late	July 29, 2015: early
NDVI range	0.58 to 0.69	0.53 to 0.81	0.74 to 0.81	0.70 to 0.80	0.49 to 0.66	0.60 to 0.67
NDVI/yield	$-0.123$	0.800***	$-0.290$	0.68**	0.462	0.010
Height/yield	0.020	0.763***	0.136	0.672*	0.412	0.388
NDVI/height	0.891***	0.970***	0.482	0.759**	0.429	0.037
NDVI/aphid density	0.138	0.379	0.351	0.215	$-0.631$*	$-0.289$
NDVI/plant response	N/A	N/A	$-0.094$	0.439	$-0.216$	$-0.280$

Note: Significant probabilities of each correlation are shown with an asterisk: * p<0.01, ** p<0.001, *** p<0.0001. No asterisk indicates no significant correlation at p>0.01 level.

4.

Discussion

The late planting had the largest aphid populations and produced the most variability in plant stress. As shown in Table 3, the maximum aphid density metric and its relationship to UAS-derived NDVI were most relevant during the end-of-season flight (July 29, 2015), exhibiting a negative correlation relative to other flight dates (for both late and early planting crops), suggesting that aphid activity has to be extensive before it is reflected in NDVI values. As observed in Fig. 6, $R^2$ values for UAS-derived NDVI in relation to aphid stress and yield are low but are informative (all regressions were significant) and likely representative of the variability in the system. The inherent variability in the system was apparent in the aphid/yield relationship ($R^2=0.35$) used to calculate thresholds for tolerable aphid activity based on the field data.³⁰ The UAS-derived NDVI/yield relationship ($R^2=0.21$, Fig. 6) was likely hampered by the same variability. Given the apparent within-plot variability of NDVI and crop height measurements (see Fig. 3), future field observations should include hyperspatial (within-plot) georeferenced data to match the resolution of the UAS-derived metrics.

UAS-derived NDVI was the more informative variable, based on strength of linear relationship, compared with UAS-derived crop height when exploring their use as surrogate measures for aphid density and yield (see Fig. 6). This disparity in performance between the two metrics was more substantial for aphid density than yield. When comparing the correlation between UAS-derived NDVI and crop height, the strength of the relationship was dependent on the stage of growth (see Table 3). The weak height/NDVI correlation was only seen at the end of the season in the late planting. This is because the leaves of plants are mostly dead or dying, altering the NIR reflectance and resulting in low NDVI, regardless of plant height.

Based on the correlations shown in Table 3, UAS-derived NDVI was not found to be a good indicator of plant response for evaluation of sooty mold coverage caused by aphid infestation. This is likely because sooty mold mainly affects the lower leaves of the plants, which are not as visible to the UAS, at least at the altitudes flown here. The worst plant response levels reflect a 41% to 50% level of sooty mold coverage, so it is possible that more sooty mold coverage would affect canopy leaves and therefore be reflected in mean NDVI values. Only three plots were rated this high on plant response scale over the season, and these had mean NDVI values similar to plots with an 11% to 40% level. Plots evaluated at a 0% to 10% level of plant response generally had higher mean NDVI values, but this may have been because of the insect damage impacting the physiology and therefore the spectral response of the plant rather than a direct impact of the sooty mold on visible spectral response.

By comparing NDVI measurements by different platforms, it was found that NDVI was strongly correlated between UAS measurements and other platforms, depending on plant growth stage and soil signal filtering methods. This dependence is observed in Table 2. Despite differences in resolution, flying height, and NDVI extraction methods, manned and unmanned aerial surveys were interchangeable in terms of evaluating plant health based on the sensors utilized. The modified sprayer tractor experiment was more difficult to compare with the UAS survey. While this platform using tractors is readily available to growers, the number of points collected and positioning of the sensors may, in some cases, lead to incorrect assumptions of crop health for the entire plot due to outlying points affecting mean values. Furthermore, differences in the footprint of the GreenSeeker ($5\mathrm{cm}\times 60\mathrm{cm}$) relative to the UAS camera’s per-pixel GSD and differences in viewing angle may also play some role in the reduced correlations observed (see Table 2). An appropriate soil signal filter and noise removal method for the GreenSeeker may improve results in sparser vegetation. However, the difficulty in comparing both sensors may stem more from the differences in the red and NIR bandwidths used by each sensor. GreenSeeker centers around 660 and 770 nm for red and NIR, respectively, whereas the Canon S110 NIR equipped on the UAS centers around 625 and 850 nm, respectively, with a broader bandwidth. Another important factor, which may be the most important, is that the GreenSeeker is an active sensor that generates its own light source. This sensor has been thoroughly tested by the scientific community and is relatively unaffected by light conditions.^31,32 In contrast, the UAS remote sensing approach is more impacted by light conditions due to scene to scene variation in ambient lighting, viewing angle relative to the terrain, and altitude (see Sec. 2.4.2).

Soil signal filtering for NDVI is only necessary when the canopy is sparse [see Fig. 4(a)], and even then a polygon method of filtering ground between rows may be sufficient. NIR reflectance is typically much lower in soil than in vegetation, even unhealthy vegetation, and filtering out low NIR reflectance may be more effective than filtering out low NDVI values. However, determining a threshold for removal will, in general, be more variable than selecting a low NDVI threshold. Low NIR removal may be a simpler way of filtering soil than an SAVI because it does not require a soil line or an estimated correcting factor. One concern when filtering low NIR reflectance or NDVI values is the possibility of filtering out mold that may be covering the leaves of the canopy. For this study, it does not appear that sooty mold was dense enough to impact NDVI measurements, but higher plant response levels (70% to 100%) may require special care to keep these low NDVI values from being excluded. Among the soil signal filtering methods, the difference between NDVI values was rarely dramatic enough to recommend one method over the other.

Filtering soil and lower canopy signal for estimating crop height proved to be more necessary than soil signal filtering for NDVI [see Fig. 4(b)]. “Sparky” crops such as corn and sorghum tend to produce lower mean crop heights if the ground signal is not filtered.¹⁴ Lower leaves on sorghum plants are easily visible within gaps in the canopy, so filtering out these lower leaves provides a more accurate estimation of canopy height and therefore a better indication of crop health. Selecting a height threshold should be dependent on field observations and plant growth stage. This threshold could increase as the season goes on. Alternative approaches employed for estimating crop heights from UAS-derived point cloud data include percentile heights (e.g., 95%) or a mean height based on a centerline width that follows the apex of the crop.^14,33–37 These can operate on either the UAS-SfM point cloud or generated crop surface model. In comparison to our method, a measure such as a 95-percentile height may provide a more plot representative average flag leaf crop height or apex crop height. However, that does not necessarily mean that such a metric would be more indicative of aphid stress or yield as examined here. Our method uses a tuned height threshold to segment lower canopy and then applies a mean value extracted from cells within fitted polygons to each plot row; as such, it is comparable to a percentile height metric. Because we are averaging our hyperspatial UAS-derived metrics across a plot to correlate with field observations, we would not expect to see a major difference in the correlation patterns observed using an alternative percentile height metric. For example, we expect those plots with a higher average crop height based on our method to have, in general, a higher average 95-percentile height.

Determining a best metric for crop height in relation to ground truth data was beyond the scope of this study as our focus here was on assessing UAS-derived NDVI and mean plant height as indicators of aphid stress. The approach for measurement of sorghum height implemented in the field consists of picking three plants within a two-row plot, measuring the height from ground to flag leaf or apex using a height rod stick, and reporting an average for the three plants. The plant locations are not georeferenced, and the field observer is supposed to pick plants that are a representative average for the two center rows of a plot. This is a highly tedious and subjective process. Given the subjectivity and sparsity of this approach, we believe the UAS-SfM observations provide a much more detailed and accurate representation of crop height in the study area. This claim is supported by other studies in the literature that show the potential of UAS-based photogrammetric approaches for crop growth monitoring and phenotyping.^33–36 Furthermore, UAS provides orders of magnitude increase in data collection efficiency at the field-scale relative to manual ground observation approaches. For example, the UAS-SfM approach can provide “plant level” crop height measurements across entire fields at daily intervals (see Fig. 3). This detailed information could be used to track crop evolution and potential degradation in growth rates as a function of aphid stress. This information could be applied at the plant, plot, and field level to track spreading and other factors.

UAS-based SfM photogrammetry to derive 3-D point cloud data represents an alternative to airborne lidar. Although a comparison of airborne lidar and UAS-SfM is beyond the scope of discussion, it is important to emphasize some key differences among the methods. Lidar is a pulsed ranging technique whereas SfM is photogrammetric and relies on accurate image-to-image key point correspondence and collinearity to reconstruct the 3-D scene. As such, it generates what is called a first (or single) return textured point cloud whereas modern linear mode airborne lidar systems provide multiple return detection capability. This multireturn capability has enabled lidar to be widely applied to forestry because it allows for canopy and below canopy measurement. However, the potential benefits of multireturn lidar data over relatively short vegetation such as sorghum depend on the range resolution of the lidar system, which is a factor of the laser pulse width and receiver dead time, among others. Lidar systems with very short pulse widths and small beam divergence are needed for penetrating small gaps in shorter crops to derive useful multiple return information of the crop canopy. Furthermore, SfM methods are image-based and therefore susceptible to false parallax induced from moving vegetation between overlapping images (such as from wind) and to poor feature correspondence in areas where the scene is highly uniform (such as some agricultural fields with dense crops). This can sometimes result in sparse or noisy point cloud data over vegetation dependent on weather conditions and vegetation structure.³⁸ This can be negated, as was done here, by flying during low wind conditions and filtering point cloud noise. A major advantage of SfM is the potential of acquiring hyperspatial resolution point cloud data. Compared with traditional airborne lidar conducted at high altitudes, sampling densities can go from submeter to subcentimeter for UAS-SfM using low flying heights and high-resolution cameras. Alternatively, small UAS-borne lidar is becoming more prevalent and can enable sampling densities comparable to UAS-SfM dependent on flying height and scan characteristics.

5.

Conclusions

Results from this study show that a small UAS equipped with a modified consumer-grade camera for NIR capture can be utilized to derive informative NDVI and crop height metrics using a standard SfM processing workflow for purposes of monitoring aphid stress on sorghum grain development. Aphid density was experimentally manipulated while other factors that may affect plant stress were minimized when possible (e.g., use of fertilizer and herbicides per regional standards) or were likely homogeneous across the small plots (e.g., plant water stress associated with rainfall pattern). Early in the growing season, aphid density, in general, positively correlated to UAS-derived NDVI due to limited impact on plant stress detectable by the sensor, although a significantly strong linear relationship was not constituted. As aphid populations increased and the season progressed, results showed that plant stress as indicated by lower NDVI values (e.g., 0.49 to 0.66 in the late planting) was detected later in the season correlating to the increased stress and reduced yield. More specifically, the aphid density of late planting reached a high negative correlation to NDVI ($r=-0.631$) on July 29, 2015, and a lower negative correlation in the early planting ($r=-0.289$) (see Table 3 and Fig. 6). Therefore, the relationship supports an interpretation that lower NDVI values are indicative of higher aphid densities. It further supports the conclusion that the effectiveness of the UAS approach presented here was limited to later in the season when aphid activity was more extensive.

Similar patterns were observed between increased stress and lower average UAS-derived crop height, but UAS-derived NDVI provided a stronger relationship (see Fig. 6). This pattern also held when comparing yield relationships. Although mean crop height correlated with NDVI during midseason flights, results from this study suggest that it is not a replaceable surrogate for NDVI. The two are likely better used in complement. Results further suggest that mean NDVI is more sensitive and thereby a better indicator at the plot level to growing levels of aphid pressure compared with a mean crop height metric.

The variability of the relationships between UAS-derived metrics (crop height and NDVI) and field measurements (aphid density, yield, and plant response) explored in this study substantiate that precision agriculture for pest monitoring using UAS multispectral imaging combined with SfM processing provides great potential as well as complications. Multiple UAS-based measurements or combining UAS-based measurements with selected ground observations may help reduce this variability and better differentiate stresses.³⁹ Results do, however, suggest that automated classification algorithms can be developed using simple measures of NDVI and crop height derived from UAS-SfM mapping approaches to discriminate with reasonable accuracy zones of crop damage or stunted growth at late season stress levels apparent in this data set. More research is warranted to determine if more moderate levels of aphid infestation are capable of being detected by NDVI or crop height earlier in the season. Similarly, examination of other vegetation indices beyond NDVI for aphid stress detection could reveal added value to the UAS approach presented here as could employing more advanced pattern recognition methods.

Comparison of NDVI measures from the manned aerial and tractor platforms based on the sensors examined showed strong correlation to UAS-derived NDVI, depending on plant growth stage and soil signal filtering methods (see Table 3 and Fig. 5). NDVI extracted from the manned and unmanned aerial surveys proved more interchangeable for purposes of evaluating plant health compared with the tractor-mounted NDVI approach. Practical considerations, such as cost, flying time, and data granularity, are more likely to determine which platform is used for a given situation. The benefit of small UAS is its capability to efficiently acquire NDVI measurements at much higher spatial resolutions and temporal frequencies at field scale.

Finally, this work explored the relation between NDVI and crop heights averaged across plots to enable comparison with field measurement data. Future experimental design should look at field measurement support to enable assessment of UAS-derived NDVI and other metrics at the individual plant level for aphid stress detection to take full advantage of the hyperspatial resolution of the data. Similarly, UAS studies should be designed to examine the spatial-temporal distribution of aphid stress as it evolves at the field scale.