Applying object-based image analysis and knowledge-based classification to ADS-40 digital aerial photographs to facilitate complex forest land cover classification

Abstract. In general, considerable human and material resources are required for performing a forest inventory survey. Using remote sensing technologies to save forest inventory costs has thus become an important topic in forest inventory-related studies. Leica ADS-40 digital aerial photographs feature advantages such as high spatial resolution, high radiometric resolution, and a wealth of spectral information. As a result, they have been widely used to perform forest inventories. We classified ADS-40 digital aerial photographs according to the complex forest land cover types listed in the Fourth Forest Resource Survey in an effort to establish a classification method for categorizing ADS-40 digital aerial photographs. Subsequently, we classified the images using the knowledge-based classification method in combination with object-based analysis techniques, decision tree classification techniques, classification parameters such as object texture, shape, and spectral characteristics, a class-based classification method, and geographic information system mapping information. Finally, the results were compared with manually interpreted aerial photographs. Images were classified using a hierarchical classification method comprised of four classification levels (levels 1 to 4). The classification overall accuracy (OA) of levels 1 to 4 is within a range of 64.29% to 98.50%. The final result comparisons showed that the proposed classification method achieved an OA of 78.20% and a kappa coefficient of 0.7597. On the basis of the image classification results, classification errors occurred mostly in images of sunlit crowns because the image values for individual trees varied. Such a variance was caused by the crown structure and the incident angle of the sun. These errors lowered image classification accuracy and warrant further studies. This study corroborates the high feasibility for mapping complex forest land cover types using ADS-40 digital aerial photographs.


Introduction
Aerial photography technology has undergone sophisticated development in recent years.2][3][4][5][6] The forest land cover situation in Taiwan is remarkably complex, and mixed forest types are markedly prevalent.The artificial stereoscopic interpretation method is generally adopted in the use of aerial photographs to determine forest types in Taiwan; however, this method creates interpretation results that are often inconsistent.Image classification is another method that can be employed to determine forest types.This method generally examines differences in image pixel values in which the results are used for subsequent analyses.When high-resolution images are used for analyses, they provide more detailed spatial information.However, conventional pixel value-based classification methods create excessively large data volumes during calculations and ignore the implication of adjacent pixels, which results in errors. 7,8][9][10] In high-resolution images, because terrain features may consist of several pixels, the characteristics of objects forming the terrain features can be identified more easily through manual interpretation.2][13] OBIA primarily uses image segmentation techniques to divide highly homogenous adjacent pixels of an image into spatially correlated objects. 11Next, the unequally sized image objects are separated into different categories.2][13] In addition, shapes of image blocks obtained from the image segmentation process can be used to identify terrain features, and the interpretation process is similar to a manual-like interpretation process. 7All image blocks feature their own spectra and shapes. 14Similarly, knowledge-based systems can be used to interpret and classify the terrain features of image blocks.By adopting hierarchical logic and adding appropriate spatial characteristic-related information, classification result accuracy can be improved 1,12,13,15 and combined with some statistical models, such as classification and regression tree (CART) [16][17][18] and random forest. 13his study investigated the feasibility of using high-resolution digital images in forest land cover classification.The feasibility test procedure is as follows: first, we selected ADS-40 digital airborne multispectral photographs as the study materials.Next, we combined OBIA techniques with a knowledge-based classification method.Finally, we performed forest land cover classification using a stratified classification method.

Overview of the Research Area
The research site is located at the Da-Chia stream working cycle, Central Taiwan (longitude 120°4 3′ to 121°26′; latitude 24°03′ to 24°28′).This area is under the jurisdiction of the Dongshih Forest District Office (of the Forestry Bureau) and faces the Liwuhsi Business Area to the East, the Luanta Business Area to the southwest, the Puli Business Area to the south, and the Nanchuang, Taipingshan, Nan'ao, and Ta'anhsi Right Bank Business Areas to the north (Fig. 1).The forest land cover for this site primarily includes coniferous tree, broadleaf tree, bamboo forest, fruit tree, farmland, grassland, landslide area, wetland, roads, and bare grounds.

Images
This study used aerial photographs taken by the Aerial Survey Office on September 21, 2008, using a Leica ADS-40 airborne digital scanner and performed a series of orthorectificationrelated image processing to create orthoimages. 19The spatial resolution of the Leica ADS-40 ortho-images was 25 cm, providing detailed surface properties.The Leica ADS-40 airborne digital scanner was able to simultaneously acquire information from 11 wave bands.The panchromatic charge-coupled device (CCD) of the scanner contained three line scan cameras that captured images observed by the aircraft from a forward view angle of 27 deg, a nadir view angle of 2 deg, and a backward view angle of 14 deg.In addition, four multispectral line scan CCDs were installed at a nadir view angle of 0 deg and a backward view angle of 16 deg to provide red, green, and blue lights as well as capture near-infrared (NIR) light-related wave band information.The wavelength of each light type is listed as follows: panchromatic light, 465 to 680 nm; red light, 610 to 660 nm; green light, 535 to 585 nm; blue light, 430 to 490 nm, and NIR light, 835 to 885 nm.In addition, the images presented 14-bit radiometric resolution and excellent signal-to-noise ratios.The multispectral band was analyzed in this study.The images were stored in a 16-bit format but retained their original radiometric resolutions (multispectral band = 13 bits).

Ground Truth Data and Supporting Images
Ground truth data used by the Dajia River Business Area (i.e., the sample area) are mainly comprised of land cover type mapping (prepared by the Taiwan Aerial Survey Office) as well as national land cover mapping, natural vegetation mapping, and the Third Forest Resource Survey mapping (compiled by the National Land Surveying and Mapping Center).Such reference data were then combined with aerial photographs to determine terrain object and type, and the ground verification was carried out between September 16, 2011, and September 17, 2011 (Fig. 2).
In addition, the digital elevation model data for the area of Taiwan were collected by the Aerial Survey Office, Forestry Bureau.Analysis and digitization of the 3-D aerial photos resulted in a resolution of 20 m × 20 m and files with a scale of 1∕5000.

Image segmentation
This study performed an image segmentation scale experiment for image terrain boundaries.The segmentation method primarily used the discontinuity and heterogeneity observed in the spectral responses of original images to break the images into smaller blocks.These small blocks were then set as a basic unit in subsequent classifications for image reconstruction.To effectively extract complete terrain features, users were required to carefully assess the image segmentation situation and set the appropriate segmentation criteria.Likewise, the selection of a favorable segmentation scale facilitated the extraction of desired terrain features.This study performed image segmentation using the remote sensing software ERDAS IMAGINE ® image segmentation module and assessed the image segmentation situations using parameter adjustments.For example, the "edge detection threshold" function was used to determine whether the edge threshold existed between two pixels; the "minimal value difference" function was used to identify the difference between an object and its surrounding objects; and the variance factor was used to identify the extent of pixel value changes in images.The extent of pixel value changes in images was assessed to determine whether it fell within the threshold and, subsequently, ascertain whether the object block range was to be extended.The parameter settings influenced the number of object blocks segmented.To find the optimal image segmentation scale to objectify the images, the following segmentation parameters were used, as shown in Table 1.

Normalized difference vegetation index
E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 2 ; 3 3 6 ; -1 0 8 Haralick et al. 23 Contrast 200 and contrast 400 GLCM-dissimilarity Haralick et al. 23 Dissimilarity 200 and dissimilarity 400 GLCM-homogeneity ; t e m p : i n t r a l i n k -; e 0 0 6 ; 3 3 6 ; -2 9 0 Haralick et al. 23 Homogeneity 200 and homogeneity 400 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 7 ; 1 1 6 ; 5 1 2 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 8 ; 1 1 6 ; 4 6 5 where i and j are the categories being compared; C i is the covariance matrix of category i; μ i is the mean vector of category i; tr is the sum of the trace and diagonal lines of a matrix; and D T ij is the TD between categories i and j.

Classification rules
The classification method first selected terrains that possessed unique features and were highly distinguishable.Next, terrain type was divided into different subcategories in a hierarchical format, which effectively reduced terrain type-related confusions.Subsequently, logical classification rules were set by referring to expert interpretation knowledge, and a logical procedure was established by making logical inferences.This simplified the complex classification process and allowed intricate terrain features to be effectively classified.The stratified classification method was divided into four levels: level 1 was comprised of shaded and nonshaded areas; level 2 consisted of vegetation-covered and nonvegetation-covered areas; level 3 incorporated high and low biomass areas; and level 4 involved terrain featurerelated classifications.The stratified classification diagram is shown in Fig. 3.

Knowledge-based classification
This study employed a knowledge-based classification and constructed a stratified classification model using known classification knowledge.Next, the CART algorithm was used to assess training data of each terrain type from which classification rules were established.Concurrently, supporting mapping data were used to create the classification criteria.Classification methods for levels 1 to 4 were shown in Table 3. detection procedures were conducted using brightness.This study defined brightness as the mean of the R, G, B, and NIR wavebands [Eq.(1) in Table 2].
The difference between the shadow and the nonshadow areas was enhanced using the brightness of the image before dividing the image into the respective shadow and nonshadow areas.Because the brightness values for the shadow and nonshadow areas presented bimodal distribution, the brightness thresholds of these areas can be determined using the bimodal histogram splitting method.The minimum or trough value between two brightness peaks in the histogram can be used as the threshold value to differentiate between the shadow and the nonshadow areas.
Levels 2 to 4 classification.After level 1 classification, the image was divided into the respective shadow and nonshadow areas, and the nonshadow areas were used for the classification of levels 2 to 4. The classification of levels 2 to 4 were performed using CART algorithm.First, by selecting the training data of each terrain type, classification variables or wave band image-related data were extracted.Next, the CART algorithm proposed by Breiman et al. 24 was used to establish classification rules.The CART is a technique that can be used to make classifications and predictions in the field of data mining-related research.By analyzing original data, appropriate classification rules are established and used to make predictions for data with an unknown outcome; the process also involves the use of single input variable functions to construct dichotomous decision trees.Using cross validation, this study randomly distributed data to n groups.A total of n − 1 groups were then used to build a model in which the remaining group was selected to verify the model; the verification process continued until all groups had  been chosen as a model verifier.The maximum difference in risk was selected as the postpruning rule to create tree structures with minimum risk values.Next, changes in the misclassification rate were investigated for all tree structure sizes, and models were corrected to identify the optimal model size.The classification threshold values provided by decision trees can be used to set up classification rules.In addition, supporting data or classification conditions may be added according to the literature or expert experience to enhance image classification accuracy.

Classification accuracy assessment
Image classification accuracy assessment refers to the process in which the image results obtained from the image classification process are compared with current ground truth data to determine the classification accuracy.In this study, 633 sample location data were randomly selected and paired with aerial photographs for verification.The verification indicators that measured the accuracy of the classification results consisted of producer's accuracy, omission error, user's accuracy, commission error (CE), overall accuracy (OA), and the kappa index.

Image Segmentation
The selection of a favorable segmentation scale facilitates the extraction of terrain features.Images that underwent image segmentation using the segmentation scale 200 generated a higher number of image blocks, whereas those that underwent image segmentation using the segmentation scale 400 produced a lower number of image blocks because its parameter values were more tolerant of image pixel value changes (Table 4).
Figure 3 shows the segmented blocks of forest lands obtained using segmentation scales 200 and 400.According to the figure, the segmentation scale 200 produced more intricate images in which each canopy crown was segmented into multiple blocks.Conversely, the segmentation scale 400 produced more complete images in which each crown was segmented into one block; however, many crown clusters were not segmented.Figure 4 also compared the segmentation results of coniferous and broadleaf forests; the segmentation scale 200 oversegmented the broadleaf forests but favorably segmented the coniferous forests.Such a result may be caused by coniferous trees featuring smaller crowns, which enabled the smaller segmentation scale to display superior results.By contrast, the segmentation scale 400 produced more favorable broadleaf forest segmentation results but failed to segment a number of crowns in coniferous forests.
Concerning segmentation images of landslide areas and wetlands, the segmentation scale 400 produced more complete object blocks than the segmentation scale 200.However, the former showed slight boundary confusions at the boundary between the landslide areas and wet lands, whereas the latter showed no boundary confusions (Fig. 5).
Both segmentation scales 200 and 400 showed favorable terrain boundary detection results for roads, buildings, and farmlands in which the latter produced more complete blocks (Fig. 6).Regarding the segmentation results for farmlands, the segmentation scale 200 was able to accurately detect terrain boundary between vegetation and nonvegetation farmlands but oversegmented farmlands that grew the same crops.Conversely, the segmentation scale 400 showed more complete farmland segmentation blocks (Fig. 6).The completeness of the segmentation blocks improved the accuracy of the image classification results and prevented fragmented pixels of images from other image categories from affecting the classification accuracy. 7ble 4 Comparison between the number of segmented blocks.

Divergence Test
To understand the image divergence situation for each terrain category, this study performed a divergence test on the training data in which 26 wave bands of the multiscale images were assessed.The TD value was 2000 for all terrain categories, showing that all the categories shared an identical divergence value.

Classification Accuracy Assessment
In this study, images were classified using a hierarchical classification method comprised of four classification levels.The OA and the kappa coefficients for each level and classification are shown in Table 5, whereas the classification results for levels 1, 2, and 3 are shown in Fig. 7.
The classification results for vegetation-covered and nonvegetation-covered areas were combined in Fig. 8.A total of 633 inspection areas were selected for a classification accuracy assessment; the results showed an OA of 78.20% and an overall kappa coefficient of 0.7597.The confusion matrix is shown in Table 6.

Discussion
Among vegetation types, the classification results for farmlands, fruit trees, and grasslands were superior.Their omission rates were 1.67%, 4.92%, and 13.04%, respectively, whereas their CE rates were 9.23%, 6.45%, and 2.44%, respectively (Table 6).The use of support mapping and stratified classification allowed the aforementioned terrain types to achieve superior    25,26 Therefore, this study formulated classification rules by combining empirical knowledge and logic with support mapping using the knowledge-based system in which the classification range for farmlands was controlled.
Hsieh, Chen, and Chen: Applying object-based image analysis and knowledge-based classification. . .misclassified as broadleaf trees and bamboo forests; the misclassification of coniferous trees as bamboo forests was the most substantial.Similarly, broadleaf trees were misclassified as coniferous trees, bamboo forests, and fruit trees; the misclassification of broadleaf trees as bamboo forests was the most severe.However, misclassification of fruit trees was observed less, which was attributed to the inclusion of support mapping data.According to Fig. 9, misclassification of broadleaf trees generally occurred in blocks with bright crowns.
In a study on the use of digital aerial photographs to detect pests and diseases in trees of different classifications, Meddens et al. 24 showed that misclassifications were prone to occur in pixels containing bright crowns as a result of strong sunlight.Similar results were observed in this study in which both segmentation scales 200 and 400 segmented crowns with relatively brighter pixels into independent object blocks; misclassifications were also more likely to occur in such blocks, confirming the effect of bright crown blocks on classification accuracy.Figure 9 shows how the bright crown blocks of broadleaf trees were more prone to being classified as bamboo forests, which contributed to the high CE rates of bamboo forests and the high omission rates of broadleaf trees.
For nonvegetation-covered areas, wetlands showed the highest classification accuracy, followed by buildings, landslide areas, roads, and bare grounds.Misclassifications were effectively lowered for roads and buildings because of the inclusion of support mapping, which limited the distribution range for each terrain feature and lowered the CEs between the terrain features.
Wetlands were sometimes misclassified as landslide areas and vice versa.However, for other terrain features, misclassifications were less observed.Reference 27 performed image classification using an unsupervised classification method in which they found that the misclassification of water bodies as roads (and vice versa) was frequently observed.Conversely, in this study, the use of the knowledge-based classification in combination with classification rules established using support mapping effectively reduced the misclassification of wetlands as roads (and vice versa).Concerning wetlands that were misinterpreted as landslide areas, we compared the original images of such wetlands and found that image blocks with CEs were generally comprised of terrains with brighter image blocks.For example, CEs were mostly observed in wetlands containing a water body reflecting strong sunlight.Bare grounds displayed the lowest classification accuracy.The instances in which they were repeatedly misclassified as landslide areas or roads may be attributed to the three land types sharing similar spectral characteristics.
The aforementioned terrain classification results showed that by adopting knowledge-based classification rules and limiting the distribution areas for each terrain feature, the complexity of terrain classification can be lowered and classification accuracy can be elevated.In this study, we classified images using the knowledge-based classification method in combination with object-oriented analysis techniques, hierarchical classification rules, and decision tree classification techniques.Subsequently, classification parameters, such as object texture, shape, and spectral characteristics, were used to form a class-based classification method and used together with geographic information system mapping information.The results were then compared with manually interpreted aerial photographs.The result comparisons showed that the above classification method achieved an OA of 78.20% and a kappa coefficient of 0.7597.Such results indicate that using knowledge-based classification method, formulating classification rules using geographical and system data as support mapping, and including the concept of stratified classification, the likelihood of misclassification can be reduced and classification accuracy can be enhanced.On the basis of the image classification results, some classification errors occurred in images of sunlit crowns because the image values for individual trees varied.Such a variance was caused by the crown structure and the incident angle of the sun.These errors lowered image classification accuracy and warrant further studies.

1
Object-Based Image Analysis Performed During the Image Preprocessing Stage

Fig. 1
Fig. 1 Image and map of the research area.

Fig. 2
Fig. 2 Ground verification photos for each land cover type with coordinates (WGS 84: x , y ).
T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 3 3 6 ; -6 6 T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 3 ; 3
, and Chen: Applying object-based image analysis and knowledge-based classification. . .

Fig. 5
Fig. 5 Boundary detection results of landslide areas and wetlands produced using segmentation scales (a and b) 200 and (c and d) 400.

Fig. 4
Fig. 4 Segmentation results for a coniferous forest obtained using segmentation scales (a) 200 and (b) 400; segmentation results for a broadleaf forest obtained using segmentation scales (c) 200 and (d) 400.
classification accuracy.However, farmlands and grassland with herbaceous species showed poorer classification results when the general classification model was used because they featured similar spectral reflectance values.The use of geographical data as supporting data reduces classification errors, showing that the use of such data is a key to improving knowledge-based classification accuracy.

Fig. 9
Fig. 9 Misclassification of broadleaf trees because of crowns; (a) original image, (b) object image obtained using segmentation scale 200, (c) object image obtained using segmentation scale 400, and (d) image classification.

Table 1
Segmentation parameter settings for the images.

Table 2
Image-related data.

Table 2 (
Continued).Creating object imagesAfter ADS-40 digital aerial photographs were segmented using segmentation scales of varying sizes, the object image creation process began.The original four wave bands, vegetation index, gray-level cooccurrence matrix (GLCM), and block area (i.e., areas of blocks created by the respective segmentation scale) of the images were obtained to create the object images.In this study, classification image-related data from 26 wave bands were obtained, as shown in Hsieh, Chen, and Chen: Applying object-based image analysis and knowledge-based classification. . .Journal of Applied Remote Sensing 015001-6 Jan-Mar 2017 • Vol.11(1) Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Applied-Remote-Sensing on 17 Apr 2020 Terms of Use: https://www.spiedigitallibrary.org/terms-of-use 3.1.2

Table 2 .
3.2 Training Data AssessmentIn this study, aerial photographs were used to interpret and determine the training data, and land cover mapping was used to support the interpretation process.The nine land use classes which primarily include coniferous tree, broadleaf tree, bamboo forest, fruit tree, farmland, grassland, landslide area, wetland, roads, and bare grounds.The training data (50 training points for each class) were used to analyze each variable in this study.To elevate the image classification accuracy, the training data underwent assessment in which a transformed divergence (TD) test was performed on each type of training data.TD indicies are shown as follows:

Table 3
Classification methods for different classification level.

Table 5
Classification results of different level classifications.Chen, and Chen: Applying object-based image analysis and knowledge-based classification. . .

Table 6
Confusion matrix of the final classification results.