One aspiration of precision agriculture is to minimize costs while maximizing crop yield by allowing farmers to identify problem areas in the field and deploy mitigation tactics.1 Remote sensing increases the efficiency of this process by gathering data associated with crop health quickly and automating processes for crop health visual display and evaluation.2 Satellite imagery is not collected with the frequency required for precision agriculture, and atmospheric effects or cloud cover can have dramatic effects on the quality of data produced. Ground-based (tractor-mounted) sensors achieve high resolution, but operational and data efficiency measures may be much reduced compared with other approaches. High-resolution imagery is necessary for more precise data, and unmanned aircraft systems (UAS) are able to fly much lower and can achieve higher temporal and spatial resolutions than satellite imagery or imagery captured using manned aircraft.1,3
Over the past decade and into the present, an increasing amount of research has been done on the use of remote sensing with UAS and other platforms for detecting measures of crop health. Typically, the calculation of vegetation indices, such as normalized difference vegetation index (NDVI), is used to estimate the amount of photosynthetically active radiation absorbed by plants, which has been found to be a good indicator of overall health.4 The potential of remote sensing with UAS to predict crop yield and estimate biomass using vegetation indices has been well explored in the scientific literature. UAS-derived NDVI predicted grain yield in winter wheat at a high level of accuracy.5 Similarly, UAS-based monitoring of a sunflower crop found significant linear regressions between NDVI and grain yield as well as biomass and nitrogen content.6 Research has also discovered good correlation between NDVI and yield or biomass in rice7,8 and grain sorghum.9
Using NDVI to estimate crop health can be complicated by the effects of ground that is visible through gaps in the canopy. Early in a crop’s growth stage, the canopy can be very sparse and mean NDVI values may be deceptively low due to the amount of soil reflectance included in the NDVI calculation. Vegetation indices can be tuned to reduce soil background effects. One study compared multiple soil-adjusted indices with NDVI in an area with low vegetation cover.10 The simplest index is the soil-adjusted vegetation index (SAVI), which uses a correcting factor, , an estimated value between 0 and 1 based on the vegetation density.11 More complex indices such as the transformed SAVI utilize the slope and intercept of an equation (also known as the soil line) fitted through a plot of red versus near-infrared (NIR) reflectance for various bare soil conditions. In the study, these indices were not found to outperform soil-unadjusted vegetation indices at vegetation cover below 30%. Furthermore, the requirement of establishing a soil line for applying algorithms for soil adjustment can limit its use for the commercial agricultural applications.
The use of photogrammetric height determination for crops by UAS is not as heavily researched as the use of vegetation indices, but some studies have been done using three-dimensional (3-D) point clouds to obtain an estimate of crop height and correlate it with yield and biomass. These point clouds are generated through a photogrammetric technique called structure-from-motion, or SfM.12 Research introduced by Ref. 13 using terrestrial laser scanning suggests a reasonably accurate method for deriving plant height from the point cloud data is a difference approach that calculates crop height by taking the difference between a digital surface model (DSM) of the crop canopy generated from the point cloud and a digital elevation model (DEM) of the bare-ground surface generated from a secondary data source or from ground points filtered out of the point cloud. Studies have also applied this approach to derive crop height from UAS-SfM derived point cloud data.1415.–16 The combination of vegetation indices and UAS-based canopy surface models have been shown to estimate biomass in barley and corn more accurately than vegetation indices alone.15,16 The results from these studies suggest that crop height metrics can be used as a complement to NDVI and potentially as a surrogate for assessing crop health under certain conditions.
Small UAS are also becoming increasingly affordable and accessible to the modern farmer with new regulations permitting more flexible use in different countries, and preprogrammed flight plans enable nonpilots to easily conduct flights for mapping purposes.1 Powerful software can automatically stitch raw imagery into orthomosaics and generate point clouds via SfM with geoferencing based on a ground control point (GCP) network and position information from onboard GPS stored in each image’s exchangeable image file (EXIF) format data. Consumer-grade cameras can be modified with a filter to convert a visible channel to an NIR channel, forgoing the need for more expensive four-band color-infrared cameras. A study17 comparing field measurements from more advanced multispectral sensors to consumer-grade cameras found that both types were equivalent under ambient light conditions but that unstable illumination and angular variation in reflectance in sunny conditions could impact performance for consumer-grade UAS-mounted cameras. A high percentage of image endlap and sidelap can help reduce this effect. Reference 17 recommends using at least 75% endlap and 60% sidelap. The algorithms for brightness correction, including a bidirectional reflectance distribution function (BRDF), are also recommended for creating reflectance maps on which to base NDVI calculation.17
Sorghum, Sorghum bicolor (L.) Moench, is grown throughout much of the world in small to large acreage production systems. The grain is used for animal feed, human consumption, and other purposes. Other sorghum varieties are grown for animal forage.18 The crop is also increasingly grown for the production of ethanol. In South Texas, where the study here was conducted, the sugarcane aphid, Melanaphis sacchari (Zehntner) (Hemiptera: Aphididae), has begun to infest sorghum crops.19 The insect directly disrupts sorghum physiology and photosynthesis and also produces large amounts of honeydew that serves as a growth media for black mold, which also disrupts the plant’s photosynthesis.20 Through this process, insect infestation may disrupt the spectral response of the plant, potentially allowing remote sensors to detect this form of plant stress.21 Insect damage has previously been identifiable via multispectral and hyperspectral reflectance on other types of crops.22 For example, wheat aphid damage has been identified using benchtop-based spectral measurement of individual leaves.23 Aerial detection of the insect infestation is a more complicated process. One study on UAS-based detection of potato beetle infestation found no correlation between plot-scale vegetation indices, but it was able to detect insect damage using object-based feature extraction.24 A significant negative correlation has been found between plant injury rating due to aphids and NDVI in grain sorghum fields using an airborne multispectral imaging system.25 Unfortunately, aerial detection of plant stress may require substantial, large-scale plant stress after the window of deploying a mitigation measure has passed. Insect detection at the plant level may be capable of detecting damage before it is too severe to correct, but it may also be inefficient and computationally intense. There is the additional problem of differentiating between insect disruption of photosynthesis and other stressors that have a similar impact such as nutrient deficiencies.
The only study of which the authors are aware that used UAS remote sensing to detect insect stress in sorghum investigated impacts of white grubs.26 The method used an unsupervised clustering approach to segment the field into three zones (decimated, transition, and healthy) based on the RGB pixel values. The method showed potential but was developed for severely damaged and readily identifiable stress. Although such a method could be feasible for detection of severe levels of aphid stress on sorghum, at that point it would be too severe for mitigation. Furthermore, the study did not assess the utility of NIR vegetation indices or 3-D crop structure metrics that can be extracted from the UAS acquired imagery; most notably, the study did not examine UAS for detecting variable levels of aphid stress with its unique challenges.
The purpose of this study is to investigate the utility of a small UAS equipped with a modified consumer-grade camera for NIR capture to derive NDVI and crop height measures using SfM photogrammetry for assessment of aphid stress on sorghum grain development. The study area is a small plot field experiment subjected to highly variable aphid pressure. Pregrowth, midseason, and late-season flights were conducted; NDVI and crop height metrics were extracted from acquired imagery, and statistical analysis was performed to assess correlation to aphid presence and yield estimation. NDVI derived from these flights was also compared with NDVI data collected from a manned aircraft flight conducted midseason as well as NDVI collected late in the season from a modified ground crop sprayer tractor-mounted with multispectral sensors. Additionally, the filtering methods for soil signal and lower canopy (height only) were utilized and compared among one another to provide more informative UAS estimates of mean per-plot NDVI and canopy height values, respectively.
The study was conducted at research fields at the Texas A&M AgriLife Research and Extension Center at Corpus Christi, Texas, USA, during the 2015 growing season. The research complex consists of several hundred acres where a variety of crop research experiments are conducted. For this project, the specific study area was an acre field where sorghum was planted at varying times and sugarcane aphid population densities were manipulated with insecticide (Fig. 1). The study consisted of two plantings of a grain sorghum hybrid known to be susceptible to the aphid (DKS 53-67, DeKalb/Monsanto, St. Louis, Missouri). Each planting was planted in its own section, containing 20 plots. Each plot had four rows of sorghum planted 0.96 m apart from one another. The plots were 40 feet (12.19 m) long and 12 feet (3.66 m) wide. The early planting was seeded on April 2, 2015, and the late planting was seeded on May 4, 2015.
Field Measurement Surveys
Each week, beginning May 21, 2015, field data were collected from the two center rows of each four-row plot, including estimated aphid density, plant response to aphid damage, and growth stage. To estimate aphid density, 20 leaves in each plot were sampled. When counting aphids, actual counts were only done for leaves with 10 or fewer insects. All estimates above this level were done in ranges with midpoints within these ranges used as the estimated count: 11 to 25 aphids (18 midpoint), 26 to 50 aphids (38), 51 to 100 aphids (75), 101 to 500 aphids (300), 501 to 1000 aphids (750), and greater than 1000 aphids per leaf (1500 was used as the midpoint based on field observations). Plant response was measured on a scale from 1 to 10, with each number representing a percentage of plants in a plot covered in sooty mold with a rating of 1 indicating 0% to 10% sooty mold coverage, and increasing at regular intervals to a rating of 10 indicating 90% to 100% sooty mold coverage. The sorghum growth stage was noted at the time of each survey: preflowering vegetative stage, denoted by the number of leaves on the plant (e.g., V7 indicates preflowering plants with seven leaves); approaching and through flowering period (boot stage through flowering); and a reproductive stage, during which the grain head begins to develop (milk) and matures (soft to hard dough), until the plant is ready to harvest.18
At the end of the season, the two center rows of crops of each four-row plot were harvested and yield data were estimated for each plot. Yield was estimated in kilograms per hectare (kg/ha) and adjusted to a standard of 14% moisture.18 Aphid infestation and iron chlorosis were the two main sources of crop damage present in the study area during the summer of 2015. Several aphid density measures were computed at the end of the season. The most relevant seasonal measure to this study was the per-plot maximum aphid density, computed as the highest mean number of aphids per leaf observed over the season. Unusually heavy rainfall during late spring likely had some impact on aphid population by causing a late arrival of the aphids and therefore lower aphid populations, especially in the plots planted earlier in the season.
A small, fixed-wing UAS called the eBee (senseFly, Cheseaux-sur-Lausanne, Switzerland) equipped with a three-band consumer-grade camera modified for NIR was utilized over the fields at the research center. The UAS has fully autonomous flight that can be programmed to acquire imagery at a desired flying height, percent image endlap, and percent image sidelap. The system can fly for a maximum time of on a fully charged battery, dependent on wind conditions and payload weight. For NIR capture, a 12-megapixel Powershot S110 (Canon USA, Melville, New York) was used with a sensor size of and pixel array of , resulting in a pixel pitch size of . The nominal focal length was set at 5.2 mm for flight operations, which equates to a 3.6-cm ground sample distance (GSD) at 100 m flying height above ground, where .
The camera is capable of storing images in RAW and JPEG formats and is modified to capture imagery in three bands of the electromagnetic spectrum: green (500 to 575 nm), red (575 to 650 nm), and NIR (800 to 900 nm). A filter was applied to the blue sensor on the original camera to pick up wavelengths in the NIR, and the camera is programmed to autocapture imagery triggered by the onboard navigation and processor control system of the UAS platform.
Four UAS flights were conducted over the summer to capture preseason conditions, midseason conditions, and end-season conditions. Flights targeted, on average, 75% lateral and 80% endlap in image overlap. The first flight took place on May 1, 2015, just after planting and before the early planted sorghum had begun to emerge. This flight was conducted at a lower altitude using an RGB camera for the sole purpose of deriving a high-resolution baseline DEM of the pregrowth, bare-ground surface to estimate crop heights detailed further below. Two midseason growth flights using the NIR camera occurred on June 10 and June 23, 2015. The final flight was flown on July 29, 2015, just before harvest (Table 1). The flights covered a total area of roughly 40 hectares per flight; however, only the imagery acquired over the test field and surrounding area was further analyzed for this study. Programmed flying heights for each NIR survey were adjusted dependent on wind conditions to maximize flight coverage and stability given UAS endurance limitations. Adjustments in flying height above ground level resulted in a maximum difference in GSD of approximately 5 mm between the June 10 and July 29, 2015, flights (Table 1). This is considered a negligible difference in GSD and did not warrant reprocessing to a uniform GSD for the needs of this study.
UAS flight information including point cloud density and spacing. Flying height is above ground level.
|Flight 1||Flight 2||Flight 3||Flight 4|
|Date||May 1, 2015||June 10, 2015||June 23, 2015||July 29, 2015|
|Flying height (m)||62||96||91||82|
|Camera model and bands||Canon IXUS 127 HS (R-G-B)||Canon PowerShot S110 (R-G-NIR)||Canon PowerShot S110 (R-G-NIR)||Canon PowerShot S110 (R-G-NIR)|
|Image array (pixel)|
|Early planting growth stage||Bare ground||Stage V8||Flag/boot stage||Full bloom, leaves dead|
|Late planting growth stage||Bare ground||Stage V6/V7||Stage V7/V8||Full bloom, leaves dead|
|Point density ()||220.59||97.64||79.74||60.03|
|Point spacing (m)||0.07||0.10||0.11||0.13|
Georeferencing of the acquired imagery is critical. Without proper ground control, the raw positional accuracy of the acquired imagery stemming solely from geotagging by the UAS platform’s onboard single-channel nondifferential GPS is roughly 1 to 5 m. To ensure high geodetic positional accuracy, six GCPs were placed at the four corners and two midpoints on either side of the survey area. Georeferencing was accurate within 1 to 5 cm horizontally and vertically. All flights were processed with the local State Plane Coordinate System, in this case, NAD83 Texas South meters, as the horizontal output coordinate system, and NAVD88 converted from ellipsoid heights using Geoid12b as the vertical output coordinate system.
Image Processing and Metric Extraction
Over 250 raw images were captured in each of the four flights covering the entire mapped area including the study site. These were transferred to a hard drive for postprocessing. Following each flight, the raw image data set was processed using SfM photogrammetry. Traditional photogrammetry requires metric cameras precisely calibrated. However, metric cameras are expensive and not conducive for widespread use of small UAS for surveying and mapping applications. In contrast, SfM photogrammetry exploits information from multiple overlapping images to extract 3-D object information and negate the need for precise camera calibration. There are many SfM-based commercial and open-source software solutions for processing of UAS collected imagery for mapping purposes. In this work, Pix4Dmapper Pro (Pix4D SA, 1015 Lausanne, Switzerland) was utilized to process the imagery. The basic SfM processing workflow is summarized as follows:27
1. Find corresponding features (key points) from all overlapping images using a feature-matching algorithm, such as a variant of the scale invariant feature transform (SIFT). SIFT is a well-known computer vision algorithm that allows for feature detection regardless of scale, camera rotations, camera perspectives, and changes in illumination.28
2. Key points as well as approximate values of the image geoposition provided by the UAS autopilot (onboard GPS) are input into a bundle block adjustment to reconstruct the exact position and orientation of the camera for each image. This process also simultaneously calibrates the camera for lens distortion and other intrinsic parameters. Input of GCPs can be done here and results readjusted.
3. Based on this reconstruction, the matching points are verified and their 3-D coordinates calculated. Densification of the sparse point cloud is then performed to increase the point density. The resultant set of 3-D points is used to generate a triangulated irregular network (TIN) and obtain a DSM.
4. The DSM is then used to project every image pixel and to calculate a geometrically corrected (orthorectified) image mosaic with uniform scale.
Standard geospatial data outputs from the UAS-SfM processing workflow are a 3-D point cloud, DSM, and orthomosaic. Here, these outputs were used to generate two main products for each flight: reflectance maps from which to extract NDVI and crop height maps. GIS analysis was then used to extract mean NDVI and crop height values from this data by averaging the values within polygons created for each data row in each plot. More details on these products are provided below. Figure 2 summarizes the image processing workflow.
When computing a band ratio index such as NDVI, the interest is in the variation of reflectance of the terrain across the different bands. However, the pixel value in the raw imagery and derived orthomosaics are influenced by both the variation in illumination and sensor response. Regarding illumination, the spectral signature of available incoming light can vary within a scene due to atmospheric conditions, sun-view geometry, and shadow of the plants, among other factors. Additionally, the total amount of irradiance can vary due to changing conditions such as moving clouds, time of day, or seasonal variation (e.g., winter versus summer). This effect can be estimated and corrected using radiometric calibration targets placed within the scene with known albedo properties for the wavelength bands of interest captured by the specific sensor or camera. Adjusting for variability in the amount of irradiance can also be performed using an upward looking sensor mounted on the UAS or on the ground. Radiometric calibration is important when the objective is to obtain absolute or stable NDVI measurements for comparison across surveys conducted at different times, which are likely subjected to different spectral illumination conditions. It is generally not as important for NDVI computation within a given flight due to the normalization of the formula and assuming stable illumination conditions; however, variability in local illumination during a flight can potentially impact NDVI in addition to surface reflectance. In regard to sensor effects on pixel brightness, such factors include sensor sensitivity to different wavelengths, ISO/aperture/shutter speed settings, optical vignetting effects (variability in amount of received light on the sensor due to lens and sensor geometry with edges having less brightness), gamma corrections, spectral overlap, and other factors.
It is important to mention that the above factors do not fully account for the reflectance function of a target as viewed by a sensor. For a complete characterization, one would need to look at the BRDF for a given illumination and viewing geometry. In practice, this is not feasible, particularly for small UAS surveys with numerous images, and it is not necessary. Simplified corrections for brightness variability due to illumination and sensor effects provide useful results for deriving vegetation indices.17 Furthermore, high image overlap from the UAS flight enables weighted pixel averaging to help compensate for differences in viewing geometry.
For this work, reflectance maps were created in Pix4D to correct sensor effects on pixel brightness from image to image and generate a more spatially stabilized NDVI value for a given flight. Raw imagery was captured in 16-bit TIFF format and represented linear encoded values that were not smoothed. To generate a reflectance map, the processing software uses parameters present in the EXIF data of the images to correct for some of the factors mentioned above that may vary from image to image including ISO sensitivity, aperture setting, shutter speed, and vignetting. Although radiometric calibration may have reduced a small degree of variability due to the spectral signature of illumination during the flight, it was determined to not be necessary for the purposes of this study. The objective here is to compare relative NDVI values over the study plots for a given flight extracted directly from the sensor using a standard UAS-SfM processing workflow common to most applications. Furthermore, each UAS flight targeted similar times of day with consistent illumination conditions to minimize variability within and across flights.
Three separate reflectance maps were created for each band captured by the sensor—red, green, and NIR. An NDVI map was then generated using a raster calculation based on the well-known band ratio formula:4 . NDVI maps were generated for each flight except the bare-ground flight on May 1, 2015.
Three sets of NDVI values were obtained for each flight. The first method used no soil signal filtering; the second method filtered out low NDVI values that might correspond to ground (soil) pixels; and the third method utilized a raster calculation to remove NIR band values below a percentage of the maximum NIR reflectance value before calculating NDVI. For the second method, NDVI values below 0.50 were excluded from the per-plot averages. This threshold will exclude bare ground but may exclude some unhealthy plants as well. The third method excludes pixels with low NIR reflectance values that fall below a certain percentage (threshold) of the maximum NIR reflectance. These maximum values are evaluated as being the peak values found in the entire field reflectance map, which varies with each flight. Finding this threshold requires manual examination by iteratively increasing the percentage cutoff value while assessing how many “mostly soil” pixels are removed and “mostly vegetation” pixels are retained. Differences in ambient conditions (e.g., sunlight and atmosphere), imaging geometry (e.g., camera perspective and overlap), and reflectance of the soil and vegetation influence the choice of an effective threshold for each flight. For the June 10 and July 29, 2015, flights, NIR values below 25% of maximum NIR reflectance were excluded. For the June 23, 2015, flight, the threshold was 50% of maximum NIR reflectance. Figure 3 shows NDVI maps for all three NIR camera flights. NDVI values were extracted from the NDVI maps and scaled to the plot level metrics using pairs of row polygons as explained in Sec. 2.5.
Crop height extraction
Each pixel within the study area had an overlap of at least five images and flights targeted low wind conditions to reduce parallax from moving vegetation. This allowed for robust and dense generation of 3-D point cloud data over the sorghum plots (see Table 1). A noise filtering algorithm was applied after point clouds were generated to reduce spurious points. This algorithm finds points that are isolated or have three or fewer other points within a grid of cells. Points that have three or fewer other points in their surrounding grid (with the respective point in the center cell) are removed. Cell size was set at 25 cm to compensate for average point density and to ensure a reasonable buffer above which we do not expect isolated points.
Points within each SfM photogrammetric point cloud were triangulated using a Delaunay triangulation to create a seamless TIN. The TIN was then rasterized into a DSM using linear spatial interpolation based on the TIN’s triangular facets. Because the average point spacing across flights is around 0.1 m (see Table 1), a step size of 0.1 m was used to obtain the highest usable resolution and ensure a consistent resolution from flight to flight for crop height extraction. Recall that the May 1, 2015, flight was conducted during pregrowth with the ground exposed. This flight was used to generate a bare-earth DEM for computing crop height. Raster calculations were used to differentiate the mid- and late-season flights from the DEM derived from the May 1, 2015, flight to provide an estimate of crop height (see Fig. 3).
The resulting crop surface models were further processed in three methods, similar to the NDVI extraction methods. The first method was not filtered for soil or lower canopy. The second method filtered out values under 0.5 m, and the third method filtered values under 0.25 m. Filtering out low values ensures that only the upper leaves of the canopy are included in the per-plot mean height values. To filter the low values, pixels below a certain threshold were set to null. These null values were not included in the per-plot averages. Crop height values were extracted from the crop height maps and scaled to the plot level metrics using pairs of row polygons as explained below.
Each of the 40 plots was recreated in a GIS shapefile after the second UAS flight. To further minimize error sources from soil between the rows, a polygon was created for each row, centered on the plants. Only two rows were created per plot to generate NDVI and crop height statistics for the two rows where field data were collected during the growing season. All row polygons were uniform in size: 10 m in length and 1.8 m in width (see Fig. 3).
Each pixel in a raster contains a value for that area in the field. Each pair of data row polygons contains 1200 pixels. Each pixel corresponds to an area in the field of about . Zonal statistics were compiled for each plot to include the range, mean, standard deviation, minimum and maximum NDVI, and crop height values in the area within the data row polygons. Means and standard deviations were collected for each measure and filtering method.
Surveys with Other Platforms
On June 25, 2015, a manned aerial survey was flown over the sorghum crops and mean NDVI was extracted on a per-plot basis. Imagery was obtained using a DuncanTech (Auburn, California) MS3100 color-infrared digital camera mounted in a camera hole on the fuselage of a Cessna 172 aircraft. This flight was conducted at an altitude of about 900 m and the GSD of the imagery was about 21 cm. Mean NDVI was calculated for each plot across all four rows, based on a 500-pixel sampling () in the center of each plot.
On July 29, 2015, a modified ground crop sprayer tractor (Spider, Lee Co. Inc., Idalou, Texas) was driven through the sorghum crops. A set of two multispectral sensors (GreenSeeker, Trimble Navigation Limited, Sunnyvale, California) was mounted to a support bar (boom) attached to the front end of the platform. Sensors were spaced at 0.96 m (i.e., one sensor per row) and captured between 6 and 11 NDVI readings per plot across all four rows. The footprint of the sensor was , with a viewing angle of (32 deg) and operating height of 0.8 to 1.2 m. The average NDVI values were calculated for each plot. Platform ground speed during data collection averaged . Spatial location and ground speed of the platform were recorded by a GPS receiver (AgGPS 162, Trimble Navigation Limited, Sunnyvale, California) throughout the data collection process. A hydraulic cylinder was installed vertically on the center of the tractor front boom to allow for changes in sensor height to account for crop growth.
Field data, including maximum aphid density, plant response (degree of damage to the plants caused by aphid infestation), and end-of-season yield were compared with UAS-derived NDVI and crop height using Pearson’s correlation analysis of all pairings of field data and UAS-derived data. UAS-derived per-plot NDVI values were also compared with NDVI values extracted from the manned flight and the tractor-mounted phenotyping system. For significant correlations, the form of the relationship was explored using univariate linear regression.29
To compare the soil signal filtering methods for NDVI and crop height extraction, a variance-to-mean ratio (VMR) was computed for each plot using each filtering method. The VMR, also known as the index of dispersion, is computed by dividing the square of the variance by the mean. These ratios were averaged across the late planting to find a mean ratio for each flight and each filtering method. Statistical analysis was performed using the statistical analysis system software suite.28
Comparison between Different Soil Signal Filtering Methods
VMRs in NDVI were compared among the three flights for the late planting [Fig. 4(a)]. The late planting had the most visible ground between the plants and filtering for low values had the most impact on the mean NDVI for this planting. The most effective removal in most cases was using filtering of low NIR values with filtering at 25% of max NIR reflectance for the June 10 and July 29, 2015, flights and 50% for the June 23, 2015, flight. Removing values with low NIR reflectance proved to be about as effective as removing low NDVI values. While the final flight shows a lower VMR for method 2 with filtering at an NDVI value of 0.5, there remained an unacceptable amount of unhealthy vegetation removed with this method. A lower NDVI threshold may remedy this problem.
Soil signal filtering was most beneficial in situations where the most soil was visible. Without filtering out soil, a plot’s NDVI will appear very low despite the plants being apparently healthy. This is evident on the June 10, 2015, flight. At this point, the plants were young and the canopy was sparse and fairly uniform among all plots in the late planting. As observed in Fig. 4(a), the VMR for the June 10, 2015, flight is very similar for both the NIR and NDVI filtering methods and much lower than VMR without filtering. This result suggests that these methods are interchangeable for plots with sparse canopies. During the denser canopy of the June 23, 2015, flight, NIR filtering still showed the lowest VMR. Therefore, for the purposes of comparing NDVI to field data, we chose filtered NIR reflectance to extract NDVI.
VMRs were also compared among three methods in measuring mean crop height for the three flights in the late planting [Fig. 4(b)]. The first method used no soil signal filtering. The second filtered out values below 0.5 m in the DSM, and the third filtered values below 0.25 m. The estimated canopy height (based on field observations) ranged from about 0.6 to 1.0 m.
The third method of filtering values below 0.25 m effectively removed soil, but it included lower leaves that were visible through gaps in the canopy. The most effective filtering method was the second method of filtering values below 0.5 m. This removes not only soil but also the lower leaves of the plants to provide a more accurate estimation of mean canopy height. Method 2 had the lowest VMRs, and the difference was most noticeable in the flights over sparse canopies. Therefore, method 2 was used to estimate canopy height for examining correlations between crop height and field data.
Comparison between Tractor-Mounted, Manned Aerial, and Unmanned Aerial Surveys
Imagery from the manned flight was not corrected for brightness; therefore, NDVI values appear low compared with those gathered by the UAS. The normalization of the NDVI calculation, however, ensures that relative NDVI measurements are comparable across platforms. The manned aerial flight was conducted 2 days after the June 23, 2015, UAS flight. NDVI per-plot values extracted from the manned flight data correlated very strongly with the UAS-derived NDVI in the early planting (, Table 2). A positive linear relationship was detected with higher NDVI values obtained from the UAS NDVI extraction that included soil signal filtering (, , Fig. 5). In the late planting, however, no significant correlation was detected. This is likely due to differing methods of NDVI extraction. While the polygon method employed with the UAS-derived NDVI extraction kept soil between the rows from being included in mean NDVI, the manned flight utilized a method that included soil, causing large variations between per-plot mean values. This explanation is further substantiated by comparing the unfiltered UAS-derived NDVI to the manned aerial survey, which showed an improved positive correlation (, , Table 2).
Pearson’s correlation coefficients as well as p values for NDVI comparison among tractor-mounted, manned aerial, and unmanned aerial surveys. Method 2, low NDVI filtered; method 3, low NIR filtered.
The manned aerial flight conducted 2 days after the June 23, 2015, UAS flight.
The tractor-mounted phenotyping system measured NDVI the same day as the July 29, 2015, UAS flight.
The tractor-mounted phenotyping system measured NDVI the same day as the July 29, 2015, UAS flight. The NDVI values derived from the phenotyping system correlated strongly in the late planting (, Table 2), but no significant correlation was found in the early planting. The linear regression model showed a positive relationship and comparable values, except that the range of the UAS NDVI values was wider than those of the phenotyping system NDVI (, , Fig. 5). Unlike the manned flight, the phenotyping system did not correlate well with any of the soil signal filtering methods for the early planting. The phenotyping system only calculates NDVI from a few points per plot along the centerline of the row, but the points are necessarily centered on plants spaced along the row. Because fewer points are used to calculate mean NDVI, an outlying point can greatly shift the mean, compared with a platform gathering up to 1200 points. Table 2 provides complete Pearson’s correlation coefficients as well as values for NDVI comparison between tractor-mounted, manned aerial, and unmanned aerial surveys, where , , and represent the UAS-derived NDVI based on the soil signal filtering methods [Fig. 4(a)], manned aircraft-derived NDVI, and phenotyping system measured NDVI, respectively.
UAS-Derived Measures and Field Measures Comparison
The late planting showed the highest variability in plant stress due to aphid damage and had the highest aphid density for the season, cumulating in a significant relationship between NDVI and maximum aphid density during the July 29, 2015, late season flight (Fig. 6). For this flight, NDVI values ranged from 0.49 to 0.66 in the late planting and as aphid density increased, NDVI values decreased (, ). Maximum aphid density also correlated negatively with mean crop height in the late season flight (Fig. 6), but the strength of correlation was less than that of NDVI (, ). The NDVI values from the July 29, 2015, flight in the late planting had a very low positive correlation with yield (, Fig. 6). Mean crop height showed a similar trend with a weaker correlation to yield relative to NDVI (Fig. 6). The strength of the relationship between UAS-derived crop height and UAS-derived NDVI was strongly dependent on the stage of growth, with the weakest correlation observed for the end-of-season July flight. Plant response was generally low and did not appear to correlate with any UAS-derived measures of plant health. Pearson’s correlation coefficients of UAS-derived metrics and field measurements for both plantings in all flights are given in Table 3.
Pearson’s correlation coefficients for UAS-derived metrics (height and NDVI) in early and late plantings.
|June 10, 2015: late||June 10, 2015: early||June 23, 2015: late||June 23, 2015: early||July 29, 2015: late||July 29, 2015: early|
|NDVI range||0.58 to 0.69||0.53 to 0.81||0.74 to 0.81||0.70 to 0.80||0.49 to 0.66||0.60 to 0.67|
Note: Significant probabilities of each correlation are shown with an asterisk: * p<0.01, ** p<0.001, *** p<0.0001. No asterisk indicates no significant correlation at p>0.01 level.
The late planting had the largest aphid populations and produced the most variability in plant stress. As shown in Table 3, the maximum aphid density metric and its relationship to UAS-derived NDVI were most relevant during the end-of-season flight (July 29, 2015), exhibiting a negative correlation relative to other flight dates (for both late and early planting crops), suggesting that aphid activity has to be extensive before it is reflected in NDVI values. As observed in Fig. 6, values for UAS-derived NDVI in relation to aphid stress and yield are low but are informative (all regressions were significant) and likely representative of the variability in the system. The inherent variability in the system was apparent in the aphid/yield relationship () used to calculate thresholds for tolerable aphid activity based on the field data.30 The UAS-derived NDVI/yield relationship (, Fig. 6) was likely hampered by the same variability. Given the apparent within-plot variability of NDVI and crop height measurements (see Fig. 3), future field observations should include hyperspatial (within-plot) georeferenced data to match the resolution of the UAS-derived metrics.
UAS-derived NDVI was the more informative variable, based on strength of linear relationship, compared with UAS-derived crop height when exploring their use as surrogate measures for aphid density and yield (see Fig. 6). This disparity in performance between the two metrics was more substantial for aphid density than yield. When comparing the correlation between UAS-derived NDVI and crop height, the strength of the relationship was dependent on the stage of growth (see Table 3). The weak height/NDVI correlation was only seen at the end of the season in the late planting. This is because the leaves of plants are mostly dead or dying, altering the NIR reflectance and resulting in low NDVI, regardless of plant height.
Based on the correlations shown in Table 3, UAS-derived NDVI was not found to be a good indicator of plant response for evaluation of sooty mold coverage caused by aphid infestation. This is likely because sooty mold mainly affects the lower leaves of the plants, which are not as visible to the UAS, at least at the altitudes flown here. The worst plant response levels reflect a 41% to 50% level of sooty mold coverage, so it is possible that more sooty mold coverage would affect canopy leaves and therefore be reflected in mean NDVI values. Only three plots were rated this high on plant response scale over the season, and these had mean NDVI values similar to plots with an 11% to 40% level. Plots evaluated at a 0% to 10% level of plant response generally had higher mean NDVI values, but this may have been because of the insect damage impacting the physiology and therefore the spectral response of the plant rather than a direct impact of the sooty mold on visible spectral response.
By comparing NDVI measurements by different platforms, it was found that NDVI was strongly correlated between UAS measurements and other platforms, depending on plant growth stage and soil signal filtering methods. This dependence is observed in Table 2. Despite differences in resolution, flying height, and NDVI extraction methods, manned and unmanned aerial surveys were interchangeable in terms of evaluating plant health based on the sensors utilized. The modified sprayer tractor experiment was more difficult to compare with the UAS survey. While this platform using tractors is readily available to growers, the number of points collected and positioning of the sensors may, in some cases, lead to incorrect assumptions of crop health for the entire plot due to outlying points affecting mean values. Furthermore, differences in the footprint of the GreenSeeker () relative to the UAS camera’s per-pixel GSD and differences in viewing angle may also play some role in the reduced correlations observed (see Table 2). An appropriate soil signal filter and noise removal method for the GreenSeeker may improve results in sparser vegetation. However, the difficulty in comparing both sensors may stem more from the differences in the red and NIR bandwidths used by each sensor. GreenSeeker centers around 660 and 770 nm for red and NIR, respectively, whereas the Canon S110 NIR equipped on the UAS centers around 625 and 850 nm, respectively, with a broader bandwidth. Another important factor, which may be the most important, is that the GreenSeeker is an active sensor that generates its own light source. This sensor has been thoroughly tested by the scientific community and is relatively unaffected by light conditions.31,32 In contrast, the UAS remote sensing approach is more impacted by light conditions due to scene to scene variation in ambient lighting, viewing angle relative to the terrain, and altitude (see Sec. 2.4.2).
Soil signal filtering for NDVI is only necessary when the canopy is sparse [see Fig. 4(a)], and even then a polygon method of filtering ground between rows may be sufficient. NIR reflectance is typically much lower in soil than in vegetation, even unhealthy vegetation, and filtering out low NIR reflectance may be more effective than filtering out low NDVI values. However, determining a threshold for removal will, in general, be more variable than selecting a low NDVI threshold. Low NIR removal may be a simpler way of filtering soil than an SAVI because it does not require a soil line or an estimated correcting factor. One concern when filtering low NIR reflectance or NDVI values is the possibility of filtering out mold that may be covering the leaves of the canopy. For this study, it does not appear that sooty mold was dense enough to impact NDVI measurements, but higher plant response levels (70% to 100%) may require special care to keep these low NDVI values from being excluded. Among the soil signal filtering methods, the difference between NDVI values was rarely dramatic enough to recommend one method over the other.
Filtering soil and lower canopy signal for estimating crop height proved to be more necessary than soil signal filtering for NDVI [see Fig. 4(b)]. “Sparky” crops such as corn and sorghum tend to produce lower mean crop heights if the ground signal is not filtered.14 Lower leaves on sorghum plants are easily visible within gaps in the canopy, so filtering out these lower leaves provides a more accurate estimation of canopy height and therefore a better indication of crop health. Selecting a height threshold should be dependent on field observations and plant growth stage. This threshold could increase as the season goes on. Alternative approaches employed for estimating crop heights from UAS-derived point cloud data include percentile heights (e.g., 95%) or a mean height based on a centerline width that follows the apex of the crop.14,3334.35.36.–37 These can operate on either the UAS-SfM point cloud or generated crop surface model. In comparison to our method, a measure such as a 95-percentile height may provide a more plot representative average flag leaf crop height or apex crop height. However, that does not necessarily mean that such a metric would be more indicative of aphid stress or yield as examined here. Our method uses a tuned height threshold to segment lower canopy and then applies a mean value extracted from cells within fitted polygons to each plot row; as such, it is comparable to a percentile height metric. Because we are averaging our hyperspatial UAS-derived metrics across a plot to correlate with field observations, we would not expect to see a major difference in the correlation patterns observed using an alternative percentile height metric. For example, we expect those plots with a higher average crop height based on our method to have, in general, a higher average 95-percentile height.
Determining a best metric for crop height in relation to ground truth data was beyond the scope of this study as our focus here was on assessing UAS-derived NDVI and mean plant height as indicators of aphid stress. The approach for measurement of sorghum height implemented in the field consists of picking three plants within a two-row plot, measuring the height from ground to flag leaf or apex using a height rod stick, and reporting an average for the three plants. The plant locations are not georeferenced, and the field observer is supposed to pick plants that are a representative average for the two center rows of a plot. This is a highly tedious and subjective process. Given the subjectivity and sparsity of this approach, we believe the UAS-SfM observations provide a much more detailed and accurate representation of crop height in the study area. This claim is supported by other studies in the literature that show the potential of UAS-based photogrammetric approaches for crop growth monitoring and phenotyping.3334.35.–36 Furthermore, UAS provides orders of magnitude increase in data collection efficiency at the field-scale relative to manual ground observation approaches. For example, the UAS-SfM approach can provide “plant level” crop height measurements across entire fields at daily intervals (see Fig. 3). This detailed information could be used to track crop evolution and potential degradation in growth rates as a function of aphid stress. This information could be applied at the plant, plot, and field level to track spreading and other factors.
UAS-based SfM photogrammetry to derive 3-D point cloud data represents an alternative to airborne lidar. Although a comparison of airborne lidar and UAS-SfM is beyond the scope of discussion, it is important to emphasize some key differences among the methods. Lidar is a pulsed ranging technique whereas SfM is photogrammetric and relies on accurate image-to-image key point correspondence and collinearity to reconstruct the 3-D scene. As such, it generates what is called a first (or single) return textured point cloud whereas modern linear mode airborne lidar systems provide multiple return detection capability. This multireturn capability has enabled lidar to be widely applied to forestry because it allows for canopy and below canopy measurement. However, the potential benefits of multireturn lidar data over relatively short vegetation such as sorghum depend on the range resolution of the lidar system, which is a factor of the laser pulse width and receiver dead time, among others. Lidar systems with very short pulse widths and small beam divergence are needed for penetrating small gaps in shorter crops to derive useful multiple return information of the crop canopy. Furthermore, SfM methods are image-based and therefore susceptible to false parallax induced from moving vegetation between overlapping images (such as from wind) and to poor feature correspondence in areas where the scene is highly uniform (such as some agricultural fields with dense crops). This can sometimes result in sparse or noisy point cloud data over vegetation dependent on weather conditions and vegetation structure.38 This can be negated, as was done here, by flying during low wind conditions and filtering point cloud noise. A major advantage of SfM is the potential of acquiring hyperspatial resolution point cloud data. Compared with traditional airborne lidar conducted at high altitudes, sampling densities can go from submeter to subcentimeter for UAS-SfM using low flying heights and high-resolution cameras. Alternatively, small UAS-borne lidar is becoming more prevalent and can enable sampling densities comparable to UAS-SfM dependent on flying height and scan characteristics.
Results from this study show that a small UAS equipped with a modified consumer-grade camera for NIR capture can be utilized to derive informative NDVI and crop height metrics using a standard SfM processing workflow for purposes of monitoring aphid stress on sorghum grain development. Aphid density was experimentally manipulated while other factors that may affect plant stress were minimized when possible (e.g., use of fertilizer and herbicides per regional standards) or were likely homogeneous across the small plots (e.g., plant water stress associated with rainfall pattern). Early in the growing season, aphid density, in general, positively correlated to UAS-derived NDVI due to limited impact on plant stress detectable by the sensor, although a significantly strong linear relationship was not constituted. As aphid populations increased and the season progressed, results showed that plant stress as indicated by lower NDVI values (e.g., 0.49 to 0.66 in the late planting) was detected later in the season correlating to the increased stress and reduced yield. More specifically, the aphid density of late planting reached a high negative correlation to NDVI () on July 29, 2015, and a lower negative correlation in the early planting () (see Table 3 and Fig. 6). Therefore, the relationship supports an interpretation that lower NDVI values are indicative of higher aphid densities. It further supports the conclusion that the effectiveness of the UAS approach presented here was limited to later in the season when aphid activity was more extensive.
Similar patterns were observed between increased stress and lower average UAS-derived crop height, but UAS-derived NDVI provided a stronger relationship (see Fig. 6). This pattern also held when comparing yield relationships. Although mean crop height correlated with NDVI during midseason flights, results from this study suggest that it is not a replaceable surrogate for NDVI. The two are likely better used in complement. Results further suggest that mean NDVI is more sensitive and thereby a better indicator at the plot level to growing levels of aphid pressure compared with a mean crop height metric.
The variability of the relationships between UAS-derived metrics (crop height and NDVI) and field measurements (aphid density, yield, and plant response) explored in this study substantiate that precision agriculture for pest monitoring using UAS multispectral imaging combined with SfM processing provides great potential as well as complications. Multiple UAS-based measurements or combining UAS-based measurements with selected ground observations may help reduce this variability and better differentiate stresses.39 Results do, however, suggest that automated classification algorithms can be developed using simple measures of NDVI and crop height derived from UAS-SfM mapping approaches to discriminate with reasonable accuracy zones of crop damage or stunted growth at late season stress levels apparent in this data set. More research is warranted to determine if more moderate levels of aphid infestation are capable of being detected by NDVI or crop height earlier in the season. Similarly, examination of other vegetation indices beyond NDVI for aphid stress detection could reveal added value to the UAS approach presented here as could employing more advanced pattern recognition methods.
Comparison of NDVI measures from the manned aerial and tractor platforms based on the sensors examined showed strong correlation to UAS-derived NDVI, depending on plant growth stage and soil signal filtering methods (see Table 3 and Fig. 5). NDVI extracted from the manned and unmanned aerial surveys proved more interchangeable for purposes of evaluating plant health compared with the tractor-mounted NDVI approach. Practical considerations, such as cost, flying time, and data granularity, are more likely to determine which platform is used for a given situation. The benefit of small UAS is its capability to efficiently acquire NDVI measurements at much higher spatial resolutions and temporal frequencies at field scale.
Finally, this work explored the relation between NDVI and crop heights averaged across plots to enable comparison with field measurement data. Future experimental design should look at field measurement support to enable assessment of UAS-derived NDVI and other metrics at the individual plant level for aphid stress detection to take full advantage of the hyperspatial resolution of the data. Similarly, UAS studies should be designed to examine the spatial-temporal distribution of aphid stress as it evolves at the field scale.
The authors thank Keith Fuhrmann, Leo Deleon, Travis Ahrens, Luke Pruter, James Glover, and Darwin Anderson from Texas A&M AgriLife Research and Extension Center at Corpus Christi for planting, plot maintenance, and data collection. Tim Johnson provided assistance in managing the manned aerial survey activity and data. The authors also thank Jacob Berryhill of Texas A&M University-Corpus Christi for assistance on UAS data collections.
Carly Stanton obtained her bachelor’s degree in geographic information science from Texas A&M University-Corpus Christi (TAMU-CC) in 2016. Currently, she is an IT analyst in the GIS department at CSX Technology working on positive train control and locomotive simulators.
Michael J. Starek is an assistant professor in the School of Engineering and Computing Sciences at TAMU-CC. He holds a PhD in civil engineering from the University of Florida and was formerly a National Research Council postdoctoral fellow of the US Army Research Office. His research focuses on the applications of geodetic imaging and emergent remote sensing techniques, including UAS and Lidar, for measurement and study of natural and built systems.
Norman Elliott is a supervisory research biologist at the USDA-ARS, Wheat, Peanut, and Other Field Crops Research Unit, Stillwater, Oklahoma, USA. He conducts research on and develops decision support tools for integrated pest management of insect pests of field crops, focusing on area-wide approaches for managing insects using host plant resistance, biological control, cultural and chemical control, and airborne remote sensing technology for pest infestation monitoring.
Michael Brewer is a field crops entomologist and associate professor of entomology at Texas AgriLife Research stationed in Corpus Christi. His research interests are risk assessment and decision support for managing insect pests of cotton and sorghum and understanding the spatial ecology of plant/insect/natural enemy interactions that affects plant health risk and that forms the basis for area-wide management.
Murilo M. Maeda is an assistant research scientist at Texas A&M AgriLife Research and Extension Center in Corpus Christi, Texas, USA. He received his MSc degree in 2012 and his PhD in 2015 in agronomy from Texas A&M University. His position focuses on the management of a cropping systems and remote sensing program for agricultural research applications and crop precision management.
Tianxing Chu is a postdoctoral research associate at Texas A&M University-Corpus Christi in Corpus Christi, Texas, USA. He received his PhD in photogrammetry and remote sensing from Peking University, China. His current research interests include applying remote sensing and GIS techniques in the study of precision agriculture, UAS-based remote sensing techniques for sustainable agriculture, and precise vegetation monitoring as well as ubiquitous navigation and positioning.