Spatio-temporal evaluation of plant height in corn via unmanned aerial systems

Abstract. Detailed spatial and temporal data on plant growth are critical to guide crop management. Conventional methods to determine field plant traits are intensive, time-consuming, expensive, and limited to small areas. The objective of this study was to examine the integration of data collected via unmanned aerial systems (UAS) at critical corn (Zea mays L.) developmental stages for plant height and its relation to plant biomass. The main steps followed in this research were (1) workflow development for an ultrahigh resolution crop surface model (CSM) with the goal of determining plant height (CSM-estimated plant height) using data gathered from the UAS missions; (2) validation of CSM-estimated plant height with ground-truthing plant height (measured plant height); and (3) final estimation of plant biomass via integration of CSM-estimated plant height with ground-truthing stem diameter data. Results indicated a correlation between CSM-estimated plant height and ground-truthing plant height data at two weeks prior to flowering and at flowering stage, but high predictability at the later growth stage. Log–log analysis on the temporal data confirmed that these relationships are stable, presenting equal slopes for both crop stages evaluated. Concluding, data collected from low-altitude and with a low-cost sensor could be useful in estimating plant height.

1 Introduction stage, 15 with approximately half of biomass and N accumulated relative to maturity. 16,17 Therefore, early-season estimation of yield potential in cereals can be generated when the plant attained its maximum height (at flowering) or right before this point (1 or 2 weeks before flowering). Specifically for corn (Zea mays L.), plant height is needed for biomass estimation via stem volume calculation (measured via the cylindrical formula based on plant height and stem diameter both determined at comparable phenological stages). Previous researchers documented a high degree of correlation between ground-truthing based stem volume calculation and plant biomass at flowering in corn. [18][19][20][21][22][23][24] Application and process involved in plant height measurement conducted using UAS platforms were discussed by few researchers. [25][26][27] The process involves (i) collecting aerial data imagery from a camera mounted onboard in UAS, (ii) generating ultrahigh resolution crop surface models (CSMs), and (iii) determining plant height from the CSM, 28 herein, defined as CSM-estimated plant height. However, studies validating CSM-estimated plant height via ground-truthing measurements to predict field crop yields are scarce in the scientific literature.
Early-or even mid-season crop production forecasts assist producers to make informed decisions regarding crop and nutrient management, yield estimation, marketing, storage, and transportation. [29][30][31][32] Various models have been used to make such predictions but current applications of most of these models are only for large-scale (regional-or state-level) production systems. As crop management progresses from large-scale uniform management to site-specific using precision agriculture technologies, evaluation of within-field variation and more accurate yield forecasts should be pursued. Following this rationale, plant height relates not only to plant growth during the vegetative stages, but this plant trait can also be used to improve the relationship between active optical sensor readings and yield estimates. 31,32 Therefore, accurate and rapid plant height prediction could facilitate and improve yield forecast in corn. The overall objective of this study was to examine the relationship between plant height data collected from UAS at critical developmental stages and the final biomass estimation of corn hybrids of different maturity groups under different N management and planting densities.  The nitrogen experiment (NE) was implemented in 9.1 m × 10.6 m plots, with five N fertilization levels using urea ammonium-nitrate ranging from 0 to 200 kg N ha −1 , in 50 kg N ha −1 intervals. The plant density experiment (PE) and plant density gap experiment (PGE) were conducted in 6.1 m × 10.6 m plots, whereas the hybrid experiment (HE) was planted in 6.

Platform, Sensor, and Ground-Truthing
An UAS platform (S800, DJI, Shenzhen, China) was used to collect aerial imagery. This platform includes the Wookong-M onboard autopilot system and GPS v2 unit (S800, DJI, Shenzhen, China). Flight missions and parameter settings were assigned using UgCS ground station software. 33 The platform sensor included in each flight was Alpha ILCE A5100 RGB Sony (Tokyo, Japan), mounted with a Sony SELP1650 PZ 16-50 mm lens (sensor resolution is 6000 × 4000 pixels). Aperture and exposure time were adjusted manually prior to each flight mission considering the ground speed of the UAS and light conditions at the time of flights. In both flights, camera setting was performed using manual exposure control; shutter speed was set to 1∕4000 s, aperture to f5, and ISO to 640 and 16 mm focal length configuration. Two UAS missions were performed (17 and 29 July). Highly visible yellow and black (1 m × 1 m) crosscentered plastic ground targets were used as ground control points (GCPs). In this project, 14 yellow and black cross-centered plastic (1 m × 1 m) GCPs were used as main sources for imagery geolocation. The GCPs were distributed on the borders and internal alleys of the experiments following 34 the recommendations. The average distance between GCPs was 42 m in both missions.
Two critical corn growth stages were identified as target candidates for UAS missions: (1) the late vegetative herein termed as preflowering and (2) onset of reproductive or flowering stage. 35 These UAS mission timings were relevant because of the importance of the aforementioned corn growth stages to determine if plant height estimates can populate crop yield forecasting models. The goal of this step was to overlay CSMs from UAS with ground-truthing data then check goodness-of-fit of CSMs to capture spatio-temporal change of plant height at both stages. The ground-based data collection was divided into destructive biomass sampling and nondestructive in situ plant height measurements. GCPs and plant samples were georeferenced by implementing a Global Navigation Satellite System-Real-Time Kinematic survey for spatial and temporal monitoring. The data layer containing the geolocated plant positions was overlaid with the orthomosaic and CSMs using ArcMap (ArcGIS v10.3, Environmental System Research Institute Inc.). 36 Absolute plant height, field ground-truthing, was measured via a centimeter resolution wooden ruler. Field sampling procedures define absolute plant height as the vertical distance between the base of stem and the top region of the plant where leaves reach maximum height without any external intervention (n ¼ 331 plants measured 2 weeks before flowering and n ¼ 331 plants determined at flowering). Stem diameter (n ¼ 331 measured) was determined at the base of the plant following the procedure described by Ciampitti et al. 20 The field measurement performed 2 weeks before flowering was separated by 5 days from the UAS mission; thus plant height was adjusted to the date of the UAS mission using the observed plant height change rate computed between flowering and 2-weeks prior. This adjustment did not modify the proportion of variation accounted for the aerial imagery but significantly reduced the bias in the final observed plant height values, with lower plant height values for the ground-truth data (adjusted by 5 days within the period of height growth). For biomass determination, each individual plant was cut at the stem base and fresh weight was collected in situ. Both stem diameter and plant biomass were measured only at flowering time.

Data Processing Workflow: Crop Surface Model, Orthomosaic Generation, and Plant Height
The UAS missions were conducted at 65-m altitude to achieve a ground sampling distance, expressed as the distance between the centers of two consecutive pixels measured on the ground, of 0.015 m. An overlapping and side lapping of 80% was employed in accordance with Photoscan manual recommendations for successful CSM reconstructions [ Fig. 2(a)]. 37 Ground speed setting of the UAS was 7 ms −1 obtaining one image per 1.8 s to achieve the expected overlapping on the track of the UAS. A total of 265 images were collected per mission. The GCPs and UAS imagery data set were integrated and processed for true color [red, green, and blue (RGB)] orthomosaic and CSMs for plant height. A workflow for CSMs reconstruction was implemented using Photoscan [ Fig. 2(a)]. Processing steps included: feature matching, solving camera intrinsic, and extrinsic orientation parameters, reconstructing of the dense point cloud (DPC), and texture mapping. Parameter setting for imagery alignment presented the following characteristics: low accuracy and referenced pair preselection, tie and key points limited to 0 and 40000. The Photoscan imagery aligment algorithm detects points in the source images, which are stable under changing viewpoints and lighting conditions. 37 Then, Photoscan software generates a descriptor for each point based on its local neighborhood. These descriptors are used to detect correspondences across the images. 37 Later the software estimates the camera intrinsic and extrinsic orientation parameters using the internal bundle-adjustment algorithm to approximate accurate camera locations. 37 The distance between all GCPs was comparable and located along the image data set to minimize horizontal and vertical geometrical error. The DPC was reconstructed by Photoscan by implementing the height-field algorithm based on pairwise depth map computation. Moreover, the quality value for the DPC recontruction was set to medium for optimizing the computation time and data set size following Photoscan manual recomendations. The DPC reconstruction achieved 2550 and 2765 points∕m 2 , respectively, for each mission timing. A spatial interpolation procedure, via inverse distance weighting (IDW), was applied to the DPC to generate the CSM. Orthomosaic and CSM native Photoscan spatial resolutions were 1.0 and 2.0 cm∕pixel for data sets from both missions. The absolute plant height estimation was solved as the difference between the CSM and a digital terrain model (DTM) of bare ground surface [ Fig. 2(b)]. The DTM was reconstructed from the flowering RGB orthomosaic (captured 2-weeks prior) and CSM data sets. 26 The first step includes the ground class segmentation in the RGB orthomosaic [ Fig. 2(b)]. A support vector machine (SVM) classification was implemented in ENVI 38 to solve the ground class segmentation [ Fig. 2(b)]. The training data set included 4000 pixels and iterated in "vegetation," "bare soil," and "shadow" classes. In the iteration phase, a linear discriminant was explored with unsatisfactory results (over all accuracy ¼ 0.55). Thus, a nonlinear classification approach was implemented on the decision surface hyperplane and a radial kernel function was utilized for discrimination between classes. The gamma in kernel function was set to 0.25, the inverse of the number of computed attributes, 39,40 and the penalty parameter was set to 95. 40 The overall accuracy of the nonlinear SVM classification on the three classes was 0.79. The "bare soil" raster was exported from the CSM with bare soil areas into ArcMap and the DTM solved by using the overlapping CSM vertical and horizontal determined from the bare soil over the bare soil class data from segmented ground class regions. Ground class regions utilized in the IDW interpolation included the borders and alleys of the trials, considering an average distance of 12 m between adjacent alleys.
The absolute plant height estimated data were obtained by a map algebra subtraction between the CSM and the DTM over 0.08-m cylinder radius length assigned to each plant center location [ Fig. 2(b)]. Estimated plant height was assigned to upper mean quintile CSM pixels in the cylinder area assigned to each plant.

Plant Height Validation and its Relationship with Stem Volume and Biomass
Plant height data extracted from UAS imagery analysis and collected from field measurements (ground-truth data) were linearly regressed using the GraphPad Prism software. 41 The proportion of variation accounted by the fitted model at each developmental stage was evaluated. In addition, a linear relationship between plant biomass and stem volume calculation was examined; whereas an exponential model was fitted for the plant biomass and plant height obtained via CSM. Both fits were performed using the GraphPad Prism software. For plant height validation, model fit was calculated by determination of the root mean square error (RMSE, measurement of estimated versus observed values). Outlier detection was executed via the robust standard deviation of the residuals. 42 An allometric evaluation was performed for plant height data extracted from UAS imagery and within-field measurements. Thus, reduced major axis was performed with the Standardised Major Axis Estimation and Testing Routines (SMATR) contributed package 43 to R development software. 44 For the different phenological timings, slopes were tested to compare independent fit versus a shared fit for this parameter (if slopes are equal or not). Parameters were log 10 transformed (Y ¼ αX β → log Y ¼ log α þ β log X) prior to the analysis 45 and normal distribution of residuals was verified.

Crop Surface Model and Orthomosaic Generation
Early process of CSM construction is presented in Fig. 3. The UAS images taken at different sections were stitched together using GCPs as a reference (Fig. 3) in PhotoScan software. The importance and the number of GCPs necessary to ensure accuracy of UAS image construction have been previously discussed by other researchers. [46][47][48] In terms of geometric quality, the accumulated horizontal and vertical error was 0.7 cm∕pixel in orthomosaic and CSM from 2-weeks prior to flowering, and 0.5 cm∕pixel for the flowering raster products. Furthermore, a woody table (dimensions ¼ 0.8 m length × 0.4 m wide × 0.6 m height) was used for nonvegetation geometric evaluation. A total of six local GCPs were implemented along the top of the table to evaluate the vertical displacement between the original GCPs and the same one located in the CSM reconstruction. The vertical error in this case was 0.6 cm∕pixel.

Plant Height from Unmanned Aerial Systems Versus Ground-Truth Plant Trait
A strong positive correlation was obtained between CSM-estimated plant height and groundtruth data collected when corn plants were at flowering stage (R 2 ¼ 0.79, RMSE ¼ 0.09 m, n ¼ 331, and mean ¼ 1.84 m) [ Fig. 4(b)]. The correlation between CSM and ground-truth data measured two weeks prior to flowering (R 2 ¼ 0.63, RMSE ¼ 0.11 m, n ¼ 331, and mean ¼ 1.05 m) was relatively weaker (lower R 2 ) and with a slightly higher RMSE [ Fig. 4(a)]. The RMSE to mean plant height ratio prior to flowering was 14%, close to threefold higher compared to the ratio estimated at flowering time (5%). For the preflowering measurement, the lower proportion of the variation accounted for the CSM-estimated plant height was primarily due to lack of uniform development within the corn canopy and plants emerging at different timing due to soil-weather factors (e.g., saturated soil areas, low residue with less temperature). At flowering, maximum plant height was attained, 49 corn canopy become more uniform with less heterogeneity (lower RMSE to mean plant height ratio) and better prediction power (higher R 2 ). A correlation obtained between measured and CSM-estimated plant height is consistent with previous findings for corn, 25,26,50 barley (Hordeum vulgare L.), and rice (Oryza sativa L.). 27,51 A significant correlation between plant height measurements at flowering stage support the conclusion drawn by Geipel et al. 50 that imagery taken at end of stem elongation is better correlated with ground-truth data. Few researchers have studied the corn growth stage that UAS imagery should be taken to improve plant height estimation. 50,52,53 Other studies not using UAS imagery to evaluate the relationship between actual plant height and remotely sensed plant height 52 also concluded that late vegetative stage sensor-based plant height measurements correlated with actual plant height. Additionally, plant height measurements at late stage of corn were found to correlate with grain yield. 54 It is worth noting that at both corn stages plant height was underestimated, similar to the findings presented by Grenzdörffer 26 and Shi et al. 53 For understanding the stability of the plant height estimation, two evaluations were executed. The first one was done by comparing the linear regression slopes (for equality) of the estimated-and observed-plant height relationship [Figs. 4(a) and 4(b)] between the two growth stages evaluated to understand the stability of the estimation between dates and across plant height class (log-log transformation analysis) [ Fig. 4(c)]. Results showed similar slopes across classes for both mission timings. A second evaluation was performed to understand the absolute and relative magnitude of the plant height estimation for both dates. Ground-truth plant height data were divided into three equal classes for each crop stage. Prior flowering ground-truth mean plant height data classes were low 1.22 m, medium 1.52 m, and high 1.71 m. For the 2 weeks before flowering timing, within the low plant height group, 22% of the data in this class were underestimated by the CSM-estimated plant height trait; while for the high plant height group, this analysis resulted in 24% of the plant height observations being underestimated. At flowering, only 10% of the data on plant height across all classes (low 1.77 m, medium 2.12 m, and high 2.24 m) were underestimated by the CSM-estimated plant height trait. Synthesizing, this analysis allowed us to conclude that there was a better prediction of plant height due to a lower underestimation at flowering, which was also related to lower plant heterogeneity within the corn canopy.

Unmanned Aerial Systems Based Plant Height Relation with Biomass
Since ground-truth plant height was better estimated at flowering, the biomass data collected at the same growth stage were utilized to better understand the relationship between plant height and plant biomass. Plant biomass and CSM-estimated plant height exhibited a statistically significant correlation at flowering [ Fig. 5(a)]. However for the plant biomass trait, the proportion of the variation accounted by the CSM-estimated parameter alone was low (R 2 ¼ 0.31, n ¼ 332, and P < 0.05). Examination of Fig. 5(a) shows substantial variation present in the data, and possibly nonlinear behavior at greater plant height values. Plant biomass estimation substantially improved (R 2 ¼ 0.79, n ¼ 332, and P < 0.05) when the stem diameter (determined at equal growth stage and for the same plants as the plant height trait) was considered as a part of the stem volume calculation [ Fig. 5(b)]. Allometric equations were previously utilized in corn to predict biomass with different levels of success depending on the variation of the data (genotype by environment by management interaction) and the timing of the sampling. 15,55,56 More accurate biomass estimation performed via utilization of allometric models could be utilized as a tool to forecast corn yields. Last, improvements of biomass prediction for corn after flowering stage were documented when the apical ear shoot diameter (maximum diameter of the ear organ) was included in the stem volume calculation. 23 Thus, improvement in corn biomass prediction will be of a great challenge for the remote sensing discipline because the reproductive organs (ears) are placed at varying positions within the corn canopy. In a simplified approach, a combination of various data layers collected from multiple sensors [e.g., plant height, stand counts, normalized difference vegetation index (NDVI) 57 ] in a spatio-temporal fashion might allow to adjust in real-time corn yield estimations based not only on plant size but also considering plant nutrient status and the complex interaction with the environment.
For corn crop, a correlation between ground-truth plant height measured at late vegetative or early reproductive and plant biomass has been previously documented. 12,15 For the current study, plant height was adequately predicted for both corn growth stages: 2-weeks before (with more variation detected) and at flowering. Similarly, a significant positive relationship between corn plant height measured late-vegetative using sensors, mounted on satellite or run manually, with biomass or yield was reported. 17,58 Not many UAS based results are available for corn, 50,53 but our results on the relationship between plant height measured by UAS platform with biomass is in agreement with previous research reported for other field crops. 27,51 Last, the relationship documented in this study for corn crops between stem diameter and plant biomass is in line with findings previously presented by Mourtzinis et al. 15 A nondestructive way of measuring stem diameter from images mounted on UAS and other ground-truth sensors remains as a critical research gap for improving plant biomass prediction and the potential for yield forecasting purposes. From a remote sensing standpoint, different vegetation indices and multi/hyperspectral sensors can be investigated to improve plant biomass prediction and yield forecast procedure.

Conclusions
Spatial-temporal correlation between CSM-estimated versus ground-truthing plant height trait suggested that the CSM integration could assist in biomass estimation. Both dates evidence plant height underestimation but with higher departure for this trait for the pre-flowering stage. Imagery overlapping and plant height heterogeneity become critical factors in the plant height estimation process. At flowering stage, plant biomass and yield prediction could still be used for late management practices, such as nutrient fertilization and fungicide/insecticide protection. Nonetheless, accurate corn yield prediction at early growth stage (before flowering) remains a topic needing additional research. The evidence suggests that both plant traits such as stem diameter and/or nutrient content estimation should be targeted to increasing reliability of forecasting yield procedures. Future research should also look into the integration of UAS and spectral remote sensing data into ultrahigh spatial-resolution analysis for crop growth modeling.