Manned aircraft versus small unmanned aerial system—forestry remote sensing comparison utilizing lidar and structure-from-motion for forest carbon modeling and disturbance detection

Abstract. Sustainable forest management relies on the acquisition of timely (change detection) and accurate structural information of forest landscapes. Light detection and ranging (lidar) remote sensing platforms enable rapid, three-dimensional (3-D), structural data collection with a high spatial resolution. This study explores a functional carbon model applied to a dense, closed deciduous forest. Data are collected by manned airborne systems and unmanned aerial system, producing both lidar and structure-from-motion (SfM) 3-D mapping. A hybrid approach combining cost-effective SfM-generated data with lidar-derived digital elevation models also is explored, since the SfM fails to produce adequate terrain returns. Carbon modeling results are comparable to those achieved by the initial developers (r2  =  0.64 versus r2  =  0.72), despite the challenging uneven-aged forest environment. Vertical profiles, mapped utilizing a volumetric point density from the manned airborne lidar, are leveraged to train a binary classifier for disturbance detection. Producer’s accuracy, user’s accuracy, and Kappa statistic for disturbance detection are 94.1%, 92.2%, and 89.8%, respectively, showing a high likelihood of detecting disturbances (harvesting). The results bode well for the use of unmanned aerial system (UAS) systems, and either lidar or SfM, to assess forest stocking. Although disturbance detection is successful, further study is required to validate the use of UAS, and especially SfM, for this task.


Introduction
Forests traditionally have supplied societal resources, e.g., timber, recreation, etc., while also sequestering carbon, providing a wildlife habitat, and aiding in climate regulation. 1,2 Managing these forests in a sustainable manner requires timely and accurate/precise measurement methods, for change detection and quantification needs, respectively. 3 It is in this context that remote sensing utilizing laser scanning systems, also known as lidar (light detection and ranging), has come to the fore. Lidar systems have reached a maturity level to where they may be considered a standard data source for structural measurements of forests, [4][5][6] where these measurements refer to the spatial arrangement in a three-dimensional (3-D) space, as well as the distribution and abundance of vegetation structural elements in a scene. 7 Lidar systems generate 3-D point clouds, depicting a scene structure by emitting a laser pulse and measuring the time delay of the return pulse to directly measure a distance from the sensor. 8 If the initial pulse location and direction of the pulse is known, a 3-D point cloud of measurements can be generated to describe the scene that is being assessed. [9][10][11] The structural measurements made by lidar systems can be used for the measurement of a variety of forest characteristics, including stems-per-hectare, volume-per-hectare, canopy statistics, biodiversity information, biomass estimates, and even carbon estimates. [11][12][13][14][15] These carbon estimates are of particular importance, especially in the context of sustainable forest principles identified by the Montreal process and REDD+ initiatives. 16 Both efforts stress the need for temporally recurring measurements of forests to monitor carbon, biomass, and biodiversity changes. 17 The need for understanding forest structural characteristics and changes is increasing as the role of forests is better understood in climate mitigation strategies and natural climate solutions. 18 In fact, even detailed forest structural complexity has been shown to directly correlate with nitrogen, light use efficiencies, and aboveground net primary production. 19 It is with this in mind that lidar has been shown to be an effective measurement tool for forest structure, biomass, and carbon allocation. [9][10][11] The multireturn per pulse nature of lidar systems has allowed us to better the capture of structural information of heterogeneous and older forest growth that earlier, image-based methods failed to characterize. 20 However, a well-known hindrance to using airborne laser scanning (ALS) versus a traditional ground crew, or direct measurement method, is cost. 21,22 ALS collections on a fixed-wing aircraft typically cost tens of thousands of US dollars more than a standard ground crew-based collection for a typical forest extent. 23,24 A growing alternative to the commercial fixed-wing aircraft approach is the small, unmanned aerial system (sUAS), which is defined by the Federal Aviation Association as an unmanned aircraft weighing less than 55 lbs (25 kg) at the time of takeoff. 25 There has been an increase in the use of sUAS platforms for remote sensing of forests in recent years, with Jaakkola et al. producing one of the earliest sUAS lidar systems for forest management in 2010. 3,6,21,22,[26][27][28] This increase in sUAS use can be attributed to a few advancements: the increased payload and flight time capability seen with the modern sUAS, a reduction in sensor mass of cameras, lidar systems, and GPS/ inertial measurement unit (IMU), and an increase in GPS/IMU accuracy in placement of the airframe and characterization of the airframe's attitude during flight (roll, pitch, and yaw). All of these factors allow for larger areas to be imaged, while enabling high-quality data products and a low cost of operation. 26 In the earliest examples, the flexibility of utilizing a sUAS allowed for the generation of highdensity point clouds (100 to 1500 pts∕m 2 ), leading to accurate structural measurement not typically seen with lower density measurements (height bias = −1.6 cm). 21 These data were used to validate the application of sUAS data for biomass change and defoliation, resulting in a fit of r 2 ¼ 0.92 and demonstrating the utility of sUAS lidar system for forest characterization. Even earlier examples of sUAS lidar systems, generating relatively low point densities of 8 pts∕m 2 , demonstrated adequate accuracy levels in a tree location with an uncertainty of 0.53 m, which is on the order of ground crew accuracies. 22 The performance at high-and low-density measurement capability, flexibility of flight duration/scheduling that the sUAS provides, and a demonstrated ability to achieve point accuracies within specified requirements of governmental surveying and mapping standards, have enabled sUAS application for forest structure assessment and associated change detection. 29,30 Wallace et al. 6 found that with only 50 pts∕m 2 , compared to the higher density values of 100 to 1500 pts∕m 2 , a sUAS platform is able to reliable repeat forestry measurements over 10 flights of the same scene (mean height error of individual crowns = 0.35 m), noting the applicability of sUAS data products to be used for plot-level studies, as well as a tree-level measurement. This tree-level measurement capability is highlighted in a later study, which achieved a 98% detection rate of individual crowns by utilizing a high-density point cloud generated from the same sUAS lidar system. 3 With this level of capability to measure fine structural details, at the tree-and plot-level, we therefore expect the sUAS system to perform well in our plot-level study. However, a question remains whether lidar or high-density, imagederived point clouds would be best suited for forest assessment purposes.
These two common approaches to generating 3-D point cloud products with a sUAS, i.e., structure-from-motion (SfM; lower cost solution) and lidar (improved characterization of layering, but more expensive), therefore, require attention. SfM has been more prevalent in sUAS studies due to the reduced cost to procure adequate cameras and recent increase in computing power, which enabled the processing of large image quantities required for typical forest extents. 28,31 SfM products can be derived using a common RGB camera, by acquiring overlapping imagery from multiple viewpoints. Computer vision algorithms then detect matching points in overlapping images and use standard photogrammetric methods for height extraction of each stereo/overlapping point 31,32 sUAS systems are well-suited to SfM missions, especially rotor type sUAS air frames, as they are capable of capturing hundreds to thousands of images of the target scene with a large overlap, 360-deg coverage, and minimal motion blur. A disadvantage of the SfM methods is that in order to gather a height measurement of a target, it must be in the unobstructed view of the camera from multiple perspectives. A dense, closedcanopy forest presents a significant challenge to this type of measurement, since it typically is difficult to detect enough ground points for height normalization, resulting in a need for an externally sourced digital elevation model (DEM). 21,22,[32][33][34] One of the traditional methods of generating models for the estimation of plot-level forest metrics is to use regression analysis on a group of selected statistics, generated from the lidar/ SfM point clouds themselves or digital surface models, such as raster canopy height models (CHM), to predict ground-measured (field truth) values for the imaged plots. 35 Once these models have been validated, they can be applied to entire forests, assuming the whole forest has been imaged. 36 This type of analysis generally performs well on the study sites for which the models were developed; 37-39 however, many models lack common predictors and do not perform well when applied to other types of forests. 35 Zhao et al. 35,40 proposed a functional model that derived biomass predictions from canopy height distributions (CHD), extracted from the CHM of each study plot. This approach showed an ability to maintain performance across scale changes and over a temporal period of 4 years. During this study, based on synthetic reference data, it was found that the model performed well (r 2 ¼ 0.82 − 0.94; RMSE ¼ 14.6 − 33.7 Mg∕ha) across varying plot sizes (0.01 to 1 ha). A follow-up study explored the application of this model to real reference data, while simultaneously investigating the temporal utility of the model for biomass predictions at future dates in the same region. 40 The training and testing data were sourced from a coniferous plantation in Scotland, comprising smooth terrain and dominated by Sitka spruce (Picea sitchensis), with additional contributions from European larch (Larix decidua), Norway spruce (Picea abies), and lodgepole pine (Pinus contorta). In this context, the model performed well in predicting initial data (r 2 ¼ 0.72; RMSE ¼ 21.5 Mg∕ha) and predicting biomass at later dates for the same region (r 2 ¼ 0.90; RMSE ¼ 15.7 Mg∕ha). The ability to train the model on one set of dates from a previous data collection and predict biomass on the same region at a later date has a direct applicability to the carbon-monitoring portion of sustainable forest management principles.
The primary objectives of this article therefore relate to our ability to accurately and precisely assess forest structure (carbon) and its changes (carbon monitoring), while comparing a low-cost, SfM approach to a more detailed, but costly lidar-based method. Specifically, we will (i) compare carbon models, derived from sUAS versus manned airborne lidar/RGB cameras, for a dense, heterogeneous deciduous forest located in the southeastern United States (USA) and (ii) assess the efficacy of a structural detection algorithm, based on sUAS versus manned airborne systems, to differentiate carbon changes across the landscape. The latter objective will evaluate disturbed (partially harvested) and intact forest plots, all toward ensuring compliance with sustainable forest management principles. Critically, both ALS and SfM data products were produced for both platforms as an additional comparison. The assessment was performed by using a functional model, described by Zhao et al., 35,40 to generate a carbon model of each plot measured by both platforms. In support of the second objective, a detector is implemented as a binary classifier using descriptions of the vertical distributions of each plot from voxels obtained by the ALS systems. The detection of partially harvested sites is of use to larger forest product organizations that interface with smaller landowners, which are abundant in the southeastern USA, to monitor compliance with sustainable forest management principles.

Study Area
derived from carbon inventories completed by foresters on the ground. These reference data were measured at specific plot locations within the forest at each site. The carbon inventory process details and associated ground reference data will be described in Sec. 2.2.
The Clinch Mountain study area covers ∼1646 ha and is rugged terrain composed of steep ridges and narrow valleys. Temperate deciduous species dominate the forests here, comprising mostly eastern USA broadleaf species. Reference data existed for 804 stems at this study site. The vegetation species included were predominantly deciduous types, comprising mostly the following species: maple species (Acer negundo, A. rubrum, and A. saccharum), yellow poplar (Liriodendron tulipifera), and oak species (Quercus alba, Q. coccinea, Q. muehlenbergii, Q. prinus, Q. rubra, Q. stellata, and Q. velutina) comprising 23.63%, 22.76%, and 18.66% (65.05% total) of the vegetation present, respectively. There were only three (0.37%) examples of eastern red cedar (Juniperus virginiana), the lone coniferous species, at this study site. This area served as an ideal location to highlight the use of ALS systems for gathering forestry information rapidly in difficult-to-traverse terrain. A map of this location can be seen in Fig. 1.
The Om Sanctuary was far smaller than the Clinch Mountain study area, covering only 17 ha. Similarly, reference data for a far smaller number of trees were available as well (149 trees). A map of the Om Sanctuary can be seen in Fig. 2. This location comprised a more diverse species mixture: yellow poplar (L. tulipifera), sourwood (Oxydendrum arboreum), chestnut oak  (Q. prinus), and red maple (A. rubrum) comprising 26.17%, 15.44%, 11.41%, and 11.41% (64.43% total) of the vegetation present, respectively. A notable difference between the species population at the Om Sanctuary when compared with the Clinch Mountain site is the presence of conifers. Two species (Pinus strobus and P. rigidia) were present, totaling 11.41% of the vegetation. This is much larger than the 0.37% coniferous contribution at the other study site. This area served as a supplement to continue development of the sUAS models after data corruption (malfunctioning sensor) reduced the sample size of imaged plots in the Clinch Mountain location.
This location also proved as challenging as the Clinch Mountain site, since it was located on a steep slope, where the sUAS staging site, to maintain legal line-of-sight (LOS), was separated from the forest by a river. As with the previous site, these forests were also of the temperate deciduous type, being mostly composed of eastern USA broadleaf trees.

Reference Data Collection
Reference data were collected in the form of a carbon inventory by ground surveyors at both locations. This was completed in October 2016 at Clinch Mountain, and March 2017 at the Om Sanctuary. At the Clinch Mountain, the audit measured 151, 0.04 ha, fixed-radius plots (in slopefeet), spaced on a regular grid of 305 m across the mountain's face, for a total inventory of 1949 individual stems. The plots at the Om Sanctuary were not in a regular grid but did utilize the same fixed-radius circular plot size, each covering an area of 0.04 ha. The total inventory measured at the Om Sanctuary was 804 trees.
For each location, diameter at the breast height (DBH) of vegetation >5 0 0 (∼12.7 cm) and total height were measured, along with models produced of volume, above ground biomass (AGB), total carbon, and above ground live carbon (AGLC). Plot centers were located using a handheld global positioning system (GPS), with a positional accuracy of AE2 m. All measurements were collected using standard forest inventory operating procedures by forestry professionals. DBH was measured according to the United States Forest Service Forest Inventory and Analysis protocol, with a tape measure accuracy of AE0.254 cm (0.1 in.). Tree height was measured using laser hypsometers and clinometers to derive the tree height from known distances and angles from the base and top of each tree. Tree height accuracy was AE1 m. These measurement accuracies are important as they are inputs into the allometric equations used to generate the volume, AGB, total carbon, and AGLC values reported as reference data. All aspects for the forest carbon inventory was re-sampled and audited by an independent third party for accuracy and was determined to be within the stated limits. Furthermore, all calculations were conducted to the 2014 California Environmental Protection Agency Air Resources Board Compliance Offset Protocol, US Forest Projects. The inventory resulted in a standard error of 4.98 tonnes/ac giving a 90% confidence interval of þ∕ − 6.580%.

Data Collection
The manned ALS data collection was completed by a commercial vendor on July 20, 2017, over the Clinch Mountain study site only. A Vexcel color (RGB) camera system and a Leica ALS80 small-footprint discrete return lidar, mounted on a fixed wing aircraft, was flown at a constant altitude ∼2400 m above mean sea level, and 1478 to 1814 m above the ground level. This ALS has a scanning rate of up to 200 Hz and a pulse rate of 1 MHz, with intensity values for the first three returns of each pulse and up to five returns/pulse being recorded. Point densities achieved using this sensor's configuration ranged from 30.44 to 99.16 pts∕m 2 , with an average point density of 71.19 pts∕m 2 . The inconsistency between point densities was attributed to the inconsistent overlap of flight paths at the actual plot locations (plots at the center of the forest exhibited higher densities).
The sUAS data collections were completed on August 11 and 13, 2017, at the Clinch Mountain site and on August 12, 2017, at the Om Sanctuary. All flights were completed between 9h00 and 16h00 local time during (diffuse) overcast or clear sky conditions; there was not a noted performance degradation from the overcast conditions. The platform used for the sUAS was a Dà-Jiāng Innovations (DJI) Matrice M-600 Pro, with a custom payload rig capable of lifting a modular sensor platform; a lidar sensor and an RGB camera were flown for this study. AVelodyne VLP-16 small footprint discrete return lidar was utilized on the sUAS in conjunction with an Applanix APX-15 UAV as the GPS/IMU. The APX-15 employees real-time kinematic (RTK) correction to the GPS positioning data achieve position error bounds of 2 to 5 cm. This RTK correction was applied to the IMU data as well, achieving errors of 0.03 deg for roll and pitch, and 0.18 deg for yaw. System integration was performed by the UAS Research Lab in the Chester F. Carlson Center for Imaging Science at Rochester Institute of Technology. A depiction of the platform, as flown, can be seen in Fig. 3.
The flight altitudes above the plots varied from 60 to 90 m above the ground level, with a variability due to a lack of precision in the DEM used by the DJI flight software in the terrain following mode; in-flight variations were <3 m. The VLP-16 lidar has a pulse rate of 300 KHz and a scan rate of up to 20 Hz. The lidar system has a maximum range of 100 m, the maximum swath width, assuming −10deg to 10-deg sight lines like that of the ALS80, is 35 m. The VLP-16 is capable of 360-deg scanning, but it is not recommended to use scans at oblique angles for foliage mapping, as large scan angles may introduce error and occlusions. 29,41,42 The VLP-16 is a two return system (first and last return for every pulse), along with associated intensities. With this system, point densities achieved were 497.04 to 2393.93 pts∕m 2 , with a mean point density of 1504.2 pts∕m 2 . This variance in point density was attributed to the flight altitude, changes in flight overlap from plot-to-plot, and degradation in GPS timing signal, which resulted in returns being removed from the final point cloud. A summary of both lidar system specifications, as flown for both sites, can be found in Table 1.

Structure-from-Motion Point Cloud Generation
Contemporaneous imagery for the SfM portion of the study was collected by both platforms. The manned aircraft deployed a four-channel, visible and near infrared, Vexcel UltraCam Eagle with a capture scheme of 60% stereoforward overlap between exposures and 30% side overlap between flight lines. The delivered imagery measured 24267 × 21274 pixels, with a mean GSD of 9 cm at the ground level. The sUAS system deployed an Allied Vision Mako G-419 RGB camera with a capture scheme of 80% forward overlap between exposures and 80% side overlap between flight lines. The delivered imagery measured 2048 × 2048, with a mean GSD of 2.5 cm at the ground level. The sUAS utilized a checkerboard fight pattern and image sample rates of 1 Hz to generate a high overlap of stereoimagery for the generation of high-density lidar point clouds. The manned aircraft used a more traditional flight strip pattern of collection, optimized for the lidar sensor. A simple depiction of these two flight paths over a circular target can be seen in Fig. 4. Agisoft PhotoScan Professional v1.2.6 was used to implement the SfM algorithms for these two imagery data sets in order to generate 3-D point clouds from stereo imagery. 43 In depth details of the processing steps of PhotoScan can be found in the literature, [44][45][46] but briefly: PhotoScan initially detects key point feature matches between the images using a process based on the scale invariant feature transform; 44,45 3-D positions of the key points and camera positions are then estimated through an iterative bundle adjustment; and this information then is used to generate a dense point cloud from the imagery. 33 In our scenes, ground control points (GCP) were not available for georeferencing the generated point cloud coordinates, thus the direct georeferencing was completed using the GPS information from the APX-15 at the time of each image capture. This is the same method used by Wallace et al. and Turner et al. 33,45 An advantage of using this method for georeferencing was that the SfM closely matched the accuracy of the lidar point clouds, since both data sets drew their georeferencing processing from the same source. Throughout the processing workflow of PhotoScan, high accuracy settings were used for the alignment of photos, alignment of camera positions, and the generation of the dense point clouds. The final point clouds were exported to the American Society for Photogrammetry and Remote Sensing (ASPRS) ".las" file format for ingestion of lidar-specific software analysis.

Point Cloud Preprocessing
Each data set had to be processed through a ground classification algorithm for the generation of a DEM. These preprocessing steps were completed using the software suite LAStools (lasground_new function) with a step size of 6 m for the ground classification. 47 Point clouds were subset to 60-× 60-m grid cells, centered on the field plots. The large plot buffers avoid edge artifacts on the DEM produced, as the ground point classification routine typically contained false-positives along the edges of the point cloud. Circular buffers, matching the fixed radius of the reference plots of 37.5 ft. (11.43 m), were used for the final reduction of the point clouds (QGIS Version 2.18.0). 48 The clipping of the point clouds with the shape files were completed using the FUSION software suite (polyclipdata). 49 The height (DEM) normalization of point clouds was completed using LAStools (lasground_new/lasheight). All DEMs and products utilizing the DEMs, e.g., CHMs, were generated on the same 0.5 m raster grid and utilized the same parameters in the software workflow. The SfM for both data sets failed to generate enough ground returns to generate any sort of reliable terrain map, so a hybrid approach using the DEM generated from the corresponding lidar data sets was required for normalizing the SfM point clouds; this will be discussed later as a distinct disadvantage of the more cost-effective SfM approach. This is valid as both the lidar and SfM imagery were captured simultaneously and were georeferenced using the same GPS/IMU data.

Functional Carbon Model
A primary metric of concern for sustainable forest management is the measure of carbon stored in the forest and any changes to that sequestered carbon over time. It therefore is imperative to model these data, as it cannot be measured directly from the remote sensing data. It has been shown that the carbon and AGB relationship can be approximated as a linear function, whereby carbon is approximated as 50% of the biomass value. [50][51][52][53] This method is common as biomass and carbon can only be directly measured through destructive methods. 52 Zhao et al. introduced a linear, scale-and shape-invariant, temporally stable model for relating the CHD to biomass density. It is mathematically derived from the allometric relationships of canopy height and DBH, used by ground surveyors to generate biomass estimates from field data. 35,40 Our carbon model is derived from this same framework, since it is a linear model.
The model references a data product referred to the authors as the CHD, denoted as pðhÞ. This can be thought of as the percentage of ground, within the plot being measured, that is obscured by the top canopy at a given height. Effectively, the CHD is a histogram of the CHM height values measured at discrete height bins. Since the model is a function of a function itself, it is referred to as a functional model, i.e., the predictor pðhÞ is a function or curve and not a group of predictors. 40,54 The relationship between the CHD and biomass, or carbon density, can be described by Eq. (1), with the full derivation from allometric assumptions summarized by Zhao et al. 35 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 1 1 6 ; 3 5 1 C plot ¼ fðp i¼1;: : : n Þ ¼ R kðhÞpðhÞdh À ! discretized by Δh Δh P n i¼1 kðh i Þpðh i Þ; where kðhÞ or k i¼1;: : : n is a non-negative, nondecreasing function whose value at h ¼ 0, is zero. These constraints are derived from the assumption that no forest biomass or carbon is present if there is no canopy, and that forest biomass or carbon increases with increasing height values. 35 For our model, the discrete version of Eq. (1) is used, setting Δh ¼ 1 m; this generated n ¼ 40 height bins for our data, as the maximum tree height observed was 41.62 m. The fitting of the weighting function k i¼1;: : : n was accomplished using the MATLAB (ver. R2016b) function lsqlin and applying the constraints detailed previously. 55 A challenge with validating biomass and carbon models from other studies is that each data set covers a different distribution of biomass and carbon and potentially different forest types. Zolkos et al. addressed this issue in their review of 70 published AGB studies and found that expressing the residual standard error (RSE) of a model relative to the mean AGB or carbon value provides a satisfactory means of comparing studies between different regions with different forest CHM (or height) distributions. 56 The RSE is defined in Eq. (2). E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 2 ; 1 1 6 ; 1 5 8 where n represents the number of observations, x ip represents the predicted value from the model, and x if represents the field measured values of each observation. From this definition, RSE(%) is defined as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 3 ; 1 1 6 ; 7 3 5 RSEð%Þ ¼ RSE μ Carbon : In their review of these studies, it was noted that models assessed on temperate coniferous forest exhibited the lowest error rates [mean RSE(%) = 28.7%], when compared to temperatedeciduous forest [mean RSE(%) = 31%]. 56 In the study by Means et al., 24 it furthermore was found that heteroscedasticity existed in their models of basal area and volume, which also affect carbon and biomass models; they therefore modeled the natural logarithm of these dependent variables. 26 In addition to the RSE(%) metric, the RMSE and r 2 metrics are reported for each model and are defined in Eqs. (4) and (5), respectively, where n, x ip , and x if are defined as they were in Eq. (2). E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 4 ; 1 1 6 ; 6 0 1 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 5 ; 1 1 6 ; 5 5 0 All CHMs generated in this study used a 0.5-m raster grid for sampling and assigned the maximum height return as the value for each cell. Each modality produced dense enough point clouds that no interpolation was required for empty cells. A depiction of the DEM generation, CHM generation, and CHD generation from a point cloud product can be seen in Fig. 5.

Plot Disturbance Detection
An ability to accurately detect forest harvesting or disturbance would facilitate compliance with sustainable forest management principles. Detection of individual harvested trees has been shown in boreal forests with a success rate of 73.4%, utilizing simple differencing of two CHMs taken 2 years apart. 57 Individual tree detection in closed-canopy, heavily overlapping deciduous forests, such as the sites in this study, is difficult and generally inaccurate, which led to this analysis at the plot level and via single date assessment. This is effectively a binary classification problem, i.e., the forest is disturbed or not, based on ideally simple structural metrics. These structural metrics were derived from the normalized point cloud and from point density values at discrete heights throughout the canopy. The point density values are different from the CHD in that it considers all point values to derive structural information, rather than just the height values of the top canopy. The density values were calculated as the percentage of point returns that fell within each voxel, with each voxel being defined as a 0.5 m × 0.5 m × 0.5 m cubed volume. Density values are then aggregated in the x and y directions, to generate a vertical structure profile for the plot as a whole, i.e., the percentage of returns within a height bin. These vertical profiles are analogous to the return type seen in a waveform lidar system and have been used to classify forest change temporally. 35 Considering this, a few metrics were chosen as predication variables: • The height of the maximum energy, h max , an indicator of the height at which the dominant canopy resides and excludes ground returns; this value is expected to be lower in heavily disturbed or harvested plots. • Height of median energy (HOME), as defined by Drake et al., 52 being the height at which the 50th percentile of returns occurs, thereby including the effect of the understory. A key difference in the HOME metric in this study, when compared to the version that Drake et al. implemented, is that values below 1.5 m were ignored for our study to reduce the influence of ground returns on the HOME metric-we intended for the HOME metric to be indicative of forest canopy values. It is noted that the original definition of HOME can be influenced by the ground returns, and indirectly provide insight into gap percentages of plots. 53 • Gap percentage thus is calculated directly in this study, as the ratio of ground returns to total number of returns in a point cloud that contained only the first return value for each pulse. 58 For the binary classification operation, a linear support vector machine was trained using the fticsvm function in MATLAB (ver. R2016b). 55 To validate the classifier, two methods were chosen for comparison: holdout cross validation, where 50% of the observations were used for training and 50% used for validation, and leave-one-out cross validation, where all but one observation was used for training and the remaining observations used for testing. 59 One thousand trials were conducted, and the mean value of the producer's accuracy, user's accuracy, and the Kappa statistic were reported. These two methods of validation were chosen as there were only seven harvested plots in the 49-plot data set; thus, fold numbers were limited to seven for leave-one-out cross validation. It was determined that a comparison of the two validation methods would be the most informative of the performance of the classifier in differentiating between disturbed and undisturbed plots, given the small sample size of harvested plots. Although the sample size arguably is of concern for such a complex classifier, the cross-validation approach used here should at least provide a robust indication of performance.

Data Usage
Not all data products were used for both the carbon models and harvest detection, due to a hardware failure and occlusion issues that will be discussed in further details in Sec. 3. Table 2 provides a concise description of data source-analysis pairs, as a reference for readers.

Canopy Height Models Generation
It was imperative that the generated CHMs are accurate and precise measurements of the forest structure, since the functional carbon model is derived directly from the CHM. However, coordinate alignment of the point clouds was one challenge encountered with generating these models from different remote sensing platforms. We overcame this issue by processing point clouds in local coordinate systems and converting associated shape files used for clipping to match the local coordinates of each data source. This allowed for more accurate clipping of the point clouds for CHM generation, versus converting point clouds to a standard coordinate system, such as UTM 17N. Final alignments showed less than 1 m in alignment disagreement between the two data sources, i.e., manned versus sUAS platforms. This issue, however, does highlight the advantage of including GCPs in the target scene whenever possible, as there would be reference points to correct for these types of errors. A depiction of example CHMs for each data source can be seen in Fig. 6.
The SfM performance of the manned aircraft collection also was deemed unsatisfactory. Occlusion effects, due to the wide flight lines used to collect the lidar data, precluded many plots from generating full coverage of the study site, thus rendering it unusable for any type of modeling of the forest structure or carbon metrics. An example plot depicting the results of the occlusion affects is shown in Fig. 6(b). The sUAS SfM performance was visibly and quantitatively superior, when compared with the manned SfM, as seen in the CHM generated in Fig. 6(d). The sUAS SfM-generated point clouds covered the study area with a high density (1648.14 to 8826.46 pts∕m 2 , mean density of 3971.71 pts∕m 2 ), and had a mean CHM difference of 1.96 m, when compared to the CHM generated by the sUAS lidar across all measured plots. This improvement of performance over the manned SfM data was attributed to the checkboard flight path used to generate imagery from multiple view angles, thereby avoiding occlusion effects. However, neither SfM data set produced adequate terrain measurements and required normalization by the DEMs generated from the lidar point clouds, for their associated sensor platform.
The majority of the CHM error, when comparing the sUAS lidar and sUAS SfM, was attributed to CHM misalignment and the boundary "sharpness" with which the SfM methods were capable of generating 3-D data. This can be seen in Figs. 6(c) and 6(d); plot 113 contains large magnitude, high frequency changes in canopy height, and the lidar-generated CHM has a blurring of these edges when compared to the SfM-generated CHM. Additionally, with the exception of the manned SfM, the maximum height measured within each plot for each data set was within 1 m of the reference data. Recall that this is within the error bounds of the reference measurements, as described in Sec. 2.2. This indicates that the peaks of the tallest trees were captured by the lidar and the imagery, and that the lidar-generated DEMs accurately depicted the topography below the canopy. This metric was used since individual crown delineation was not in the scope of this study, nor were the individual crown locations cataloged during the ground reference collection.

Carbon Models
The manned lidar was the only data set able to cover enough sample plots to generate a truly independent model. The sUAS covered two different study sites, and the differences in species were enough to reduce performance to unusable levels [r 2 ¼ 0. ]. However, sample sizes were small (9 to 13 samples, depending on sensor) for each site due to loss of data from a hardware subsystem failure during flight (lidar and timing packet corruption occurred). The manned airborne system's lidar data performed well when modeling the logarithm of the carbon data [r 2 ¼ 0.64; RMSE ¼ 17.14 Mg C∕ha, RSEð%Þ ¼ 27.83%], falling just short of the performance of the model when applied to a coniferous forest by the original authors (r 2 ¼ 0.72; RMSE ¼ 10.75 Mg C∕ha). 40 These results corroborated results from other efforts, with an average of r 2 ¼ 0.78 for the studies reviewed by Brovkina et al., 60 noting these focused on more uniform coniferous forest types. This is a positive result, as it is common for biomass and carbon models to exhibit worse performance in deciduous forests, achieving lower R 2 values and larger RMSE values than for coniferous sites. 61 Additionally, the RSE(%) attained is below the average RSE(%) reported by Zolkos et al. 56 for models applied to this forest type. In this case, the sUAS produced better results; however, the smaller sample sizes arguably did result in reduced rigor in terms of sUAS performance.
An alternative method to further compare the manned versus unmanned performance, therefore, was developed. This method applied the sUAS-generated CHD functions to the manned aircraft-trained model at the four sites, which overlapped between the two data sources. The predictions for these four plots closely agreed between the two platforms, with r 2 ¼ 0.99; RMSE ¼ 1.77 Mg C/ha. This demonstrates that both systems are capable of producing similar predictions when imaging the same study sites, and that the sUAS, from a data quality and utility perspective, is a viable alternative to employing a manned system. The sUAS SfM data produced similar results to the sUAS lidar and underwent a similar testing procedure in terms of applying the SfM CHD to the trained, manned flight's carbon model. When investigating the four overlapping plots, similar performance again was observed when compared to those for the lidar comparison (r 2 ¼ 0.98; RMSE ¼ 2.13 Mg C∕ha). This indicated that a hybrid approach, i.e., lidar-derived DEMs and SfM-generated CHDs, is a viable option for this type of modeling. Table 3 provides a summary of carbon model performance on the various data sets utilized in this study.
These results suggest an opportunity for significant cost savings, since procuring and operating an RGB camera is significantly more affordable than lidar sensor packages. Considering that the underlying terrain typically does not change these areas, an approach could be to develop a base DEM from a lidar collection, then to use SfM at later dates for canopy structure measurements, even for temporal (change detection) analyses (Table 4).

Detecting Plot Disturbance
The sUAS lidar did not cover enough examples of disturbed forest to be used for training and testing of a classifier; thus, the manned lidar system was used exclusively for this purpose. However, we would recommend for future studies that sUAS data be used for a similar analysis in future studies. Voxel-based structural data have been used to discriminate between varying stand types over time; 40 we chose to explore the ability of such data to discriminate between potential harvested forest plots at a single point in time.
We explored the use of binary classification to label plots as disturbed or undisturbed. Three structural metrics were chosen for use as feature descriptors: height of maximum energy, h max , an indicator of dominant canopy height, HOME, and the gap percentage of each observation. The mean vertical profile of each class, disturbed and undisturbed, along with the mean value for HOME and h max for each class is shown in Fig. 7. Note the clear differences in the vertical profiles and the inverted relationship between HOME and h max in the disturbed plots, when compared to the disturbed plots.
A linear SVM was utilized as the binary classifier for this study. An example of one of the many generated decision boundaries in this study can be seen in Fig. 8. The SVM will generate a different decision boundary for each set of inputs, requiring multiple repeated trials to gain insight into the average performance of the classifier. In this study, the data set is clearly linearly separable, with many of the training sets yielding similarly oriented hyperplanes. These properties allowed for the development of a high performing classifier. As discussed before, the unbalanced nature of our data set, with seven samples for the disturbed class and 42 samples for the undisturbed class, necessitated two validation approaches: i. Leave-one-out cross validation-we use the leave-out phrase in reference to the disturbed training class, meaning that the data set as a whole is split into seven stratified folds, utilizing six folds for training and one fold for testing. This method yielded training sample sizes of six disturbed samples and 36 undisturbed samples, while the test fold contained the one remaining disturbed observation and six remaining undisturbed samples. This type of validation scheme resulted in highly accurate classification rates over 1000 iterations. The producer's, user's accuracy, and the Kappa statistic for the disturbed class with this validation method were 100%, 99.1%, and 99.2%, respectively. The classifier performed very well on average; however, there were large variances in the user's accuracy, with the Kappa statistic, indicating a lack in precision of the detector. The large variances were attributed to the small sample size of the disturbed class, with every false-positive and falsenegative resulting in a large change in measured accuracy. This is illustrated by the fact that the performance metrics for the undisturbed class were much less affected by these errors, as seen in Table 4. ii. A stratified-random holdout cross validation, where the sample distributions were split in half by class, disturbed or undisturbed. This resulted in 21 samples for the undisturbed plots in the training and test folds, while the disturbed plots alternated between three and four samples for the training and test folds. This method resulted in a reduction of performance across all metrics. The producer's, user's accuracies, and the Kappa statistic for this validation method were 94.1%, 92.2%, and 89.8%, respectively. These reductions in the performance were attributed to the reduced training instances to train the classifier; however, we feel that this validation method presents a more rigorous test environment, considering the unbalanced data set at hand. This method also suffered from the large variance issue that plagues the first method, due to the small sample size of the disturbed class.  We expanded the disturbance or nonsustainability detection to also include plots with moderate harvest conditions, 10 years prior to the collection date. These moderate disturbances proved to be difficult for the classifier to identify and were not clearly separable via the structural features we had selected. This can be seen in an example of the hyperplane generated by the detector while operating on the entire data set, depicted in Fig. 9, as well as noting the similarities in mean feature values between the undisturbed and moderately disturbed plots in Table 5. Any of the moderately disturbed observations are not separable utilizing these structural metrics, as they cluster within much of the undisturbed data points. This is due to the lack of aggressive harvesting, which in turn reduced the canopy variation, while also not introducing any large gaps in the canopy. It is worth noting that, in general, the gap feature did not provide as much insight into disturbance classification as HOME and h max . This is likely due to the mixed nature of the forest: when larger vegetation is removed, the understory remains and prevents the lidar first return from reaching the ground and registering as a gap location.
We concluded, based on these results, that (i) given the set of disturbance indicators, the disturbance detector/classifier was capable of reliably detecting heavily disturbed plots and could provide utility in flagging locations for inspection and (ii) when the moderately harvested plots were included in the algorithm, the detector performance dropped significantly. This implies that the simple structural features derived, while they have a history of providing insight to other predictive models, were not adequate for all harvest instances and severities. Exploration of more intricate structural features or nonlinear classification methods may improve results. However, in the context of sustainable forest management, the detector was successful in detecting those plots with the most severe loss, as can be seen by the decision boundary shown in Fig. 9. Such a severe disturbance arguably is representative of top-cutting or even aggressive timber removal practices, both of which could indicate a reduction in forest sustainability. Fig. 9 Example decision plane generated by the linear SVM when applied to full extent of disturbed observations, including both severe and moderate harvest situations. The large number of support vectors is an indication of a poorly conditioned data set with high rates of confusion. The gray plane is the decision boundary generated by the linear SVM, and the points circled in green are the support vectors.

Conclusions
This article had two primary objectives: (i) evaluate carbon modeling, derived from sUAS and manned airborne structural data products, utilizing data of a dense heterogeneous deciduous forest located in the southeastern portion of the United States (USA) and (ii) evaluate a disturbed, or harvested plot, support vector machine detection method to aid in mapping potentially harvested sites as means for ensuring compliance with sustainable forest management principles. Lidar and SfM data products were produced for both the sUAS and manned platforms over deciduous forests in the southeastern USA, including a unique lidar and unique SfM data set for each platform for use in the functional carbon model. The disturbance detector was implemented as a binary classifier using descriptions of the vertical distributions of each plot from voxels obtained by the ALS systems as features.
The comparison of data sources was completed by implementing a functional model described by Zhao et al. 35,40 to generate a carbon model of each plot measured by both platforms. Both lidar data sets produced robust, quality point clouds for DEM and CHM generation; however, both SfM data sets failed to produce enough ground returns for DEM generation, and the manned SfM data set was too sparse to produce an adequate CHM. A successful hybrid approach, using the lidar-generated DEM, therefore, was implemented to produce CHMs from the sUAS SfM data set.
The manned lidar produced an acceptable performing model, similar to other models developed for deciduous forests [r 2 ¼ 0.64; RMSE ¼ 17.14 Mg C∕ha RSEð%Þ ¼ 27.83%]. The data from the sUAS lidar and SfM methods were ingested into the trained model generated by the manned model. Both inputs produced carbon predictions that closely agreed with the manned lidar data, at r 2 ¼ 0.99; RMSE ¼ 1.77 Mg C∕ha r 2 ¼ 0.98; RMSE ¼ 2.13 Mg C∕ha, respectively. We concluded that this was an indication that the data products generated by each platform, manned and sUAS, were similar and that for small area usage the sUAS is a viable alternative to the manned system, as it also has been shown to be economically competitive. 26 More importantly, this showed that for models that rely on CHM, the more affordable method of using SfM methods to generate point clouds is a viable option, if a quality DEM is available for the study site.
The disturbance detector was envisioned as a tool to help land managers flag areas of interest, defined as areas that have been disturbed, whether by natural or man-made means; the latter could imply unsustainable practices. The manned lidar was used for this effort, as this data set contained up to five returns per pulse and covered the largest sample sizes of each class type. A linear SVM was used as the binary classifier. Detection of the heavily disturbed plots generated producer's accuracy, user's accuracy, and Kappa statistics of 100%, 99.1%, and 99.2%, respectively, for leave-one-out cross validation, and 94.1%, 92.2%, and 89.8%, respectively, for hold-out cross validation. When moderately disturbed plots were introduced in the classifier, detector performance fell drastically, since these plots exhibited similar features to undisturbed plots. It was concluded that features of higher complexity or nonlinearity need to be explored for detection of these types of forest and perhaps the investigation of a multiclass detector.
Overall, this article met both objectives, albeit with caveats. The sUAS data that did overlap with the manned lidar system performed well in the context of carbon modeling; however, a loss of 50% of the overlapping data (four of eight plots) due to data corruption highlight a strong need to develop system maturity. Additional challenges to sUAS collections were the FAA line of sight requirements. Navigating dense forests on steep mountainsides creates difficulty in maintaining a visual on the flight operations. Future considerations could include the construction of temporary or permanent towers to raise the vantage point or having multiple safety spotters available for assistance.
In the context of the second objective, the algorithm performed well in detecting heavily disturbed plots but struggled with moderately disturbed plots. Recommended future improvements would be to generate more intricate descriptions beyond the three structural indicators used in this study. Metrics that have height to living crown, or inspection of the width of the dominant canopy peaks within the vertical structure, could provide insight into the variation of canopy heights within the dominant lobe. This would relate to sustainable forest management practices, where clear cuts tend to be avoided.
the Council for Scientific and Industrial Research, South Africa. Imaging spectroscopy and structural (lidar) sensing of natural resources form the core of his efforts, which vary between vegetation structural and system state (physiology) assessment. He is currently a professor in the Chester F. Carlson Center for Imaging Science at the Rochester Institute of Technology, New York.
Darin Hale obtained his BS degree in 2003 and MS degree in 2010 in forestry from the University of Tennessee-Knoxville with concentrations in forest resources management and forest economics and policy, respectively. He is a certified forester with the Society of American Foresters and a professional member of the Forest Stewards Guild. He currently serves as the central Appalachia forest manager for The Nature Conservancy.