2.1.

Study Site

A field trial was established at the Texas A&M AgriLife Research and Extension Center (27° 46.948′ N, 97° 33.605′ W) at Corpus Christi, Texas, during the summer of 2015. The field was 85-m long and 54-m wide. Seeds of 35 cotton varieties were planted on April 1, 2015, in north-to-south oriented rows at a rate of 13 seeds/m. Each plot consisted of two rows that were 10.7-m long, spaced at 0.96 m. Each variety was replicated four times as indicated by different colors in Fig. 1. In the first replication, the variety numbers were arranged in ascending order, while in the latter replications, plots were arranged in a randomized complete block design. The border plots with dark blue color in Fig. 1 represented filler rows and were excluded from subsequent processing and analysis. Table 1 shows the cotton varieties and types planted in the study field. Cotton emergence was observed on April 6, 2015, and the cotton was harvested on August 17, 2015. Four ground control points (GCPs) were set up at the corners of the test field and used as geodetic benchmarks for image georeferencing purpose. The coordinates of the GCPs were measured using the Altus APS-3 receiver (Altus Positioning Systems, Torrance, California) in support of the TxDOT virtual reference station network, which enables instant access to centimeter-level positioning accuracy under the WGS-84 frame.

Fig. 1

Geographical illustration of the cotton study site.

Table 1

Cotton varieties and the types planted in the study field.

2.2.

Ground Data Collection

Routine management practices such as fertility, disease prevention, and weed and insect control followed the guidelines provided by the Texas A&M AgriLife Extension Service for the region. To determine population and plant density, stand counts were conducted on 0.0004 hectare on April 28, 2015, for each plot. All plots were harvested using a custom two-row cotton spindle picker model 9900 (Deere & Company, Moline, Illinois) on August 17, 2015. This equipment was modified for small-plot research and allowed yield to be established on a per plot basis.

2.3.

Unmanned Aircraft System Platform and Imagery Data Collection

The UAS platform used for this study was a Phantom 2 Vision+ multirotor copter (DJI, Shenzhen, Guangdong, China). Its integrated fisheye-lens camera provided 14 megapixel images with RGB channels.³⁰ Its gimbal enabled stabilized nadir observation to the field during the flight experiments. Similar to the airborne photogrammetric technique, the UAS platform requires taking images with a high degree of spatial overlap for a favorable processing outcome.²⁷ To select an appropriate flight strategy, the optimal image resolution, camera response time per shot, as well as the battery life were also taken into consideration. After balancing all these factors, it was determined that the UAS flew at an average height of 15 m with an average horizontal speed of $1\mathrm{m}/\mathrm{s}$, and the camera captured a shot once the UAS traveled 1 m horizontally.

Raw images were collected throughout the whole cotton growing and development phases from early April to late July 2015 for a total of 16 datasets. Specifically, during the first week after emergence, data were collected on a per-day basis for the purpose of monitoring the germination process. The routine flight experiment was conducted every week or every other week afterward, depending on the local weather conditions.

The flight time for each mission was $\sim 20\min$ to cover the whole test field, and the nadir-view images were taken to enable $\sim 70\%$ along-track and 60% across-track overlap between the images. All flights were conducted around 12:00 PM local time to ensure the homogeneity of light intensity. To minimize human intervention and maintain consistency between different flight attempts, the camera settings (e.g., white balance, ISO rating, sharpness, and exposure parameters) were kept on automatic or default mode.

2.4.

Growth Information Generation

At this stage, the goal is to extract cotton ph and cc estimates for each flight. Figure 2 shows the overall processing sketch using RGB images obtained from our lightweight UAS platform. Our study case first utilized the Pix4Dmapper Pro software (Pix4D SA, 1015 Lausanne, Switzerland) to generate initial mapping products. The software uses scale-invariant feature transform,³¹ or similar descriptor algorithm, to find key points and match a large set of images. It geometrically describes the projection between two corresponding points in a pair of images representing the same 3-D object. The GCPs were then loaded to the process to create georeferenced products. After importing the matched key points into the SfM algorithm, two-dimensional (2-D), and 3-D products (georeferenced orthomosaic images and 3-D point clouds) were generated. For generating 3-D models, SfM is an important stage to establish the correspondence between images and the reconstruction of 3-D objects to be studied. It requires multiple overlapped images as input to extract the 3-D point cloud given 2-D image common features. Detailed descriptions of the SfM algorithm and workflow can be found in Chapter 10 of Ref. 32. Similar to the Pix4Dmapper that was used in this research, the frequently implemented photogrammetry techniques can also be accessed in other equivalent commercially available software packages and open source solutions such as Agisoft PhotoScan Professional,³³ Correlator3D UAV,³⁴ insight3d,³⁵ and VisualSFM.³⁶

Fig. 2

Diagram of the cotton growth estimation using RGB images obtained from the lightweight UAS platform.

The orthomosaic images generated had an average file size of 410 megabytes. Figure 3 shows an example of the point cloud data, based on the flight experiment carried out on June 26, 2015, focusing on a small part of the whole test field. The legend shows negative height because the height was georeferenced to the WGS-84 reference ellipsoid. A 0.9-m height difference is observable from the bare soil to the peak (highest point) of the cotton plants. In our study case, over 18 million points were generated in a point cloud file for the whole field for each flight, making the file size usually larger than 650 megabytes.

Fig. 3

An example of the point cloud data, based on the flight experiment carried out on June 26, 2015, focusing on a small part of the whole test field.

Cotton ph was estimated using the Quick Terrain Modeler 8.0.5 (Applied Imagery, Chevy Chase, Maryland) after loading the point clouds. In a first step, a bare soil-based digital surface model (DSM) (terrain elevation), ${\mathrm{DSM}}_0$, is supposed to be created by the point cloud dataset collected before cotton emergence. In this study, however, the first flight experiment was completed on April 7, 2015, the second day after cotton emergence was observed. In spite of that, its DSM was still considered the ${\mathrm{DSM}}_0$ due to the tiny and ignorable size of cotton hypocotyls observed by the UAS camera. The ${\mathrm{DSM}}_0$ was then subtracted from each subsequent digital surface model ${\mathrm{DSM}}_i$ after emergence, producing the cotton surface profile ${\mathrm{CSP}}_i$ at the $i$‘th day after emergence (DAE). To allow an estimate of the uncertainty of height fields, the bare soil areas of the DSMs after emergence were compared against the ${\mathrm{DSM}}_0$. In general, the DSM uncertainty at the bare soil areas was $\sim 5\mathrm{cm}$ among all flight experiments in this study. It was mainly introduced by routine tractor plows, meteorology changes as well as image processing errors.

In a specific CSP model, the ph information of each variety was determined by averaging the height samples around the centerline of the cotton profile. More specifically in Fig. 4, the two rows in the cyan frame represent a variety plot, and the red dots depict the samples to be used for calculating the ph of this variety. The red dots were cloud points around the centerline of the cotton profile with 0.10-m width at each row. The height was finally calculated by summing all the samples of these two rows and dividing by the sample number.

Fig. 4

Point cloud illustration of ph calculation for each cotton variety plot.

For assessing cc, the georeferenced orthomosaic images were loaded into ArcMap 10.3.1 platform (Esri, Redlands, California) for classification. The study area was classified into two categories (i.e., cotton canopy and bare soil). After manually training a few samples on each category, the software performed interactive supervised classification and cc was calculated as the proportion of the ground area covered by the vertical projection of crown perimeters, as described in Ref. 37.

3.1.

Plant Height

The UAS flight experiments ranged from April 7 to July 23, 2015, which covered the cotton growing and development cycle from emergence to early harvest stages. Due to the low flight altitude in this study case, the DSMs generated in Sec. 2.4 yielded $7.3\mathrm{mm}/\text{pixel}$ horizontal resolution. The ph and cc were estimated from April 12, 2015, (7 DAE) and the datasets collected previously were excluded because of the small size of the cotton seedlings during the early germination stage.

Cotton development follows a sigmoid curve pattern characterized by a slow growth rate during emergence, followed by a geometric increase in growth during leaf production, blooming and boll development phases, and slowing down and declining during maturation.³⁸ Figure 5 illustrates the sigmoid regression curve of cotton ph against DAE based on the visible UAS imagery with $\mathrm{SSE}=\text{sum of squared errors}$, $R^2=\text{coefficient of determination}$, adjusted $R^2=\text{adjusted coefficient of determination}$, and $\mathrm{RMSE}=\text{root-mean-square errors}$. Each green dot represents the estimated ph on a specific data collection day by averaging the height statistics of all 35 varieties in 4 replications.

Fig. 5

Sigmoid regression of cotton ph against DAE based on UAS imagery.

The cotton grew slowly before May 1, 2015, (26 DAE) and then exponentially until June 19, 2015 (75 DAE). The maximum crop height was observed on July 2, 2015 (88 DAE). The increased variation in crop height after 40 DAE is likely caused by the phenotypic differences between the varieties.

3.2.

Canopy Cover

In addition to the ph estimation, cc is also closely related to crop growth and development. To assess the cc, the orthomosaic images, which were created after applying the SfM algorithm, were loaded into the ArcMap engine. The camera settings were kept on automatic or default mode, and the image brightness values, for both bare soil and canopy, varied from day to day. Therefore, each orthomosaic image generated an individual set of training samples, representing either canopy or soil, and the interactive supervised classification strategy, specifically maximum likelihood classification using sample set,³⁹ was performed to form the classified raster image. After making a raster-to-polygon conversion, cc was obtained by summing the area of all polygons over the investigated area. It is worth noting that the presence of weeds in the field may result in overestimation of cc. To eliminate the interference of the weeds, particularly during the early growing stages, polygons in between two adjacent rows were removed and excluded from the cc estimation. This process was performed by discriminating the 2-D coordinates of the centroid of the polygons, and any polygons that appeared off the cotton rows were filtered out.

The sigmoid curve in Fig. 6 depicts the cc regression against days after cotton emergence, confirming similar growth pattern as the ph. The cc calculated was based on the whole test field, rather than focusing on a specific area.

Fig. 6

Sigmoid regression of cc against DAE based on UAS imagery.

3.3.

Relationship Between Plant Height and Canopy Cover

In this study, the ground measurements of cotton ph were not available, which disables a direct validation of the height model estimated from UAS imagery. Alternatively, an indirect method was conducted in this section to justify the UAS-based ph information.

As shown in Ref. 40, there is a linear relationship between ph and cc, which satisfies

Comparison between observed cc derived from the orthomosaic image classification and estimated cc calculated using Eq. (1) is shown in Fig. 7. Results show a strong linear relationship between these two sets and demonstrate that the estimated and observed ccs coincide well with a coefficient of determination $R^2$ of 0.990. This, in turn, proves the reliability of crop height estimated from UAS imagery. As for the validity of the observed cc, it will be addressed in a separate paper with an average accuracy of 88% achieved.

Fig. 7

Comparison between the observed cc derived from orthomosaic classification and estimated cc calculated using Eq. (1). The 1:1 relationship is represented by the solid line, and the dashed line depicts the linear regression between actual observed cc and estimated equivalent.

4.1.

Yield Modeling and Analysis

In this paper, an investigation was conducted to model and assess the relationship of the cotton yield with the ph and/or cc on June 26, 2015, the approximate date of transition between rapid growth and crop maturation.

Three regression types (linear, quadratic, and exponential) were established for modeling the cotton yield (Table 2). The independent variables were ph and/or cc on June 26, 2015. The yield models were calibrated using all 35 cotton varieties (samples) of replications 2, 3, and 4 as shown in Fig. 1. Therefore, a total of 105 samples were used for model calibration Eqs. (2–10). In each sample, the plot yield, height, and cc information were calculated independently. On the other hand, the remaining 35 samples from replication 1 were used for model validation.

Table 2

Regression models established for modeling cotton yield against the ph and/or cc in terms of linear, quadratic, and exponential fits.

A comparison of the estimated yield against the observed yield for the 35 samples from replication 1 was made to evaluate the regression models. Correlations vary according to the models applied and Figs. 8Fig. 9–10 present the validation results estimating cotton yield using the models established in Table 2. Specifically, the estimated yields of subfigures (a–c) in Fig. 8 are derived from fitting Eqs. (2–4) in Table 2. Similarly, the estimated yields of subfigures in Figs. 9 and 10 are derived from Eqs. (5–7) and Eqs. (8–10), respectively. The blue asterisks in each subfigure depict 35 validation samples (varieties) in the first cotton replication. The dash lines illustrate the linear relationship between estimated and observed yield, while the solid lines indicate a 1:1 relationship. $\theta$ angle denotes the included angle between the dash line and solid line in units of degrees. A smaller $\theta$ angle indicates a smaller separation to the 1:1 line which represents the most ideal estimation-observation relationship.

Fig. 8

Validation estimating cotton yield using the model Eqs. (2–4) established in Table 2. Estimated yields of subfigures (a), (b), and (c) are derived from fitting Eqs. (2), (3), and (4), respectively.

Fig. 9

Validation estimating cotton yield using the model Eqs. (5–7) established in Table 2. Estimated yields of subfigures (a), (b), and (c) are derived from fitting Eqs. (5), (6), and (7), respectively.

Fig. 10

Validation estimating cotton yield using the model Eqs. (8–10) established in Table 2. Estimated yields of subfigures (a), (b), and (c) are derived from fitting Eqs. (8), (9), and (10), respectively.

Generally, the yield estimated by both ph and cc produces better linear results than those obtained by using only ph or cc, regardless of the fit type. In other words, Figs. 8(c), 9(c), and 10(c) outperform other solutions by using fitting Eqs. (4), (7), and (10), respectively, in terms of the SSE, $R^2$, RMSE, and $\theta$ angle. From the perspective of the fit type, all the models combining both ph and cc, i.e., Figs. 8(c), 9(c), and 10(c), generate similar correlation results as depicted in the corresponding subfigures. Likewise, the ph-only models, i.e., Figs. 8(a), 9(a), and 10(a), are not sensitive to the applied fit types according to the in-figure evaluation statistics, yet generate the worst correlations. The performance of the models using only cc was found to be between the ph-only models and the ph/cc combination models. For example, Fig. 8(b) performs worse than Fig. 8(c), while better than Fig. 8(a). The same phenomenon also appears in exponential models. However, as an exception, Fig. 9(b) shows a slight improvement on the correlation statistics than Fig. 9(c) by estimating the yield using only cc and generates the least dispersed correlation results among quadratic Eqs. (5–7).

Therefore, according to the result analysis, cotton yield is found correlated to both ph and cc by establishing linear, quadratic, and exponential models as stated in Table 2. These models generally agree with the expected behavior of larger yield at larger ph and cc. Different from previous studies on physiological progression of cotton growth and yield, these results provide a promising opportunity of predicting the yield of cotton when the rapid growth phase is about to finish before maturation and boll opening by considering the 2-D and 3-D growth statistics. Moving forward, upon these results, physiological rules might be acquired by agricultural producers to quantitatively manage and schedule plant care activities (such as irrigation, fertilization, and pest control) that, in turn, benefit healthy and robust growth of the cotton plants. To better understand the variability that may arise in estimating cotton growth and yield pattern to ensure the most efficient use of the concerns addressed above, producers or researchers are advised to arrange long-term observations under a multiyear scope. It is also worth noting that the estimation-observation validation lines, as shown in Figs. 8(c), 9(c), and 10(c), are still $\sim 24\mathrm{deg}$ off the 1:1 relationship, which makes most estimated yield lower than the observation. This phenomenon deserves further examination to enhance follow-up studies.

In this specific case, the independent variables used for estimating the cotton yield, i.e., the ph and cc, are all based on the image dataset collected on June 26, 2015, the end of the rapid increase in growth. The models established by using other dates may produce different results considering the nature of the cotton growth, and this possibility will be further investigated in a future study. Moreover, both the model training and validation phases used cotton samples of various varieties. This means the models proposed in this study ignore the differences of the growth characteristics of these varieties, thus a more delicate work needs to be carried out if aiming at estimating the yield of a specific variety.

4.2.

LiDAR-Derived Vegetation Metrics for Unmanned Aircraft System-Structure-From-Motion Point Cloud Data

UAS-based SfM photogrammetry to derive 3-D point cloud data of crop structure, as performed here, represents an alternative to LiDAR. Although a comparison of airborne LiDAR versus UAS-SfM for crop monitoring is beyond the scope of discussion, it is important to emphasize some key differences between the methods to assess the potential of LiDAR-based metrics adapted to SfM point cloud data for progressing future results in this work. SfM relies on image-to-image pixel correspondence and collinearity to reconstruct the 3-D scene. As such, it generates what is called a first (or single) return point cloud, whereas modern discrete-return (or full-waveform) airborne LiDAR systems provide multireturn detection capability.⁴¹ This multireturn capability has rendered LiDAR being widely applied to forestry because it enables canopy and below canopy measurement. As mentioned in Sec. 1, however, the potential benefits of multireturn LiDAR data over short vegetation such as cotton is debatable due to limitations in the range resolution of many LiDAR systems. Another key difference in point cloud phenomenology between the two methods over vegetation stems from LiDAR being a pulsed ranging technique, whereas SfM is photogrammetric and susceptible to false parallax induced from moving vegetation between overlapping images (such as from wind). This can sometimes result in noisier point cloud data over vegetation dependent on weather conditions and vegetation structure.⁴² However, SfM computed from hyperspatial resolution imagery collected from a low flying UAS, as done here, can provide upward of two orders of magnitude increase in point density relative to traditional airborne LiDAR collected at higher altitudes above ground. This high point density enables noise to be easily smoothed and provides a high-definition point cloud for reconstructing vegetation structure.

In this work, SfM point cloud data were used to extract a single biophysical metric, mean ph, for which to assess cotton growth rates and predict yield performance. Results showed that mean ph values enabled high accuracy cc estimation validating SfM data integrity. Given the hyperspatial point density of the UAS-SfM approach, this opens the door for the application of a wide array of LiDAR vegetation metrics adapted to SfM point clouds for improving crop phenotyping and yield prediction. The literature provides numerous examples of LiDAR-based metrics for estimating biophysical attributes of vegetation, primarily from the forestry community.^23,43–45 Multireturn LiDAR metrics often employ descriptive structure statistics calculated from height normalized LiDAR point clouds. These include selected height measures such as percentile of height, mean height, or max height; variability of height measures such as coefficient of variation; selected canopy return density measures; and selected cc measures such as cc above mean height.⁴⁴ Full-waveform LiDAR metrics describe the radiometric and geometric attributes of the return waveforms over canopy.⁴⁶ Waveform shape fitting methods are used to extract metrics such as height of median energy, standard deviation of pulse width, waveform distance, number of peaks, roughness of outermost canopy, and so on.⁴⁴ With SfM being photogrammetric, full-waveform digitization is not plausible; however, several studies, such as Refs. 46 and 47, have shown the potential of using small-footprint, discrete-return LiDAR data to generate pseudowaveforms for reconstructing waveform composition over forested terrain. Such methods are well suited for adaption to the very high point density of the UAS-SfM point cloud for biophysical parameter estimation at the plant and plot level.

The aforementioned LiDAR metrics open an array of possibilities for enhancing UAS-SfM approaches to monitor crop performance such as cotton. Furthermore, by expanding the range of the input feature set, advanced machine learning approaches can be applied to progress the predictive performance for yield and growth estimation. Such exploration is currently being investigated in extension to the results presented here.