A landslide is a geological occurrence that can cause severe loss of life, agricultural production, and the environment. Landslide disasters occur almost every year, and response time is a crucial factor in the effectiveness of postdisaster relief in such situations. In order to adequately provide emergency relief, decision makers and rescue organizations need a global perspective on the situation onsite.1 With the development of satellite and airborne remote-sensing technologies, the outputs of very high-resolution (VHR) optical images and synthetic aperture radar (SAR) images have shown a marked improvement in quality.2 These images have been widely used in a variety of fields that use remote sensing, one of which is image processing for postdisaster emergency response.
The main objective of this paper is to devise a method for convenient and efficient building damage assessment based on an analysis of images of these structures in areas affected by landslides. According to our survey of the literature, research on building damage assessment has focused on this task against the backdrop of earthquakes. Chesnel et al.3 proposed a semiautomatic damage assessment method based on a pair of very high-spatial resolution images and some ancillary data. Their system assessed damage to buildings using the change detection method. Klonus et al.4 proposed a modified change detection method based on isotropic frequency filtering, spectral and texture analysis, and segmentation, which yielded superior accuracy to standard methods. Although conventional change detection methods always work well in assessing building damage, their anti-interference ability is poor. Their results are easily influenced by natural factors, such as the solar incident angle, seasonal variation, and the Earth’s surface humidity. Li et al.5 detected building damage in urban environments from multitemporal VHR image data using the one-class support vector machine. This method performed better at the object level than at the pixel level, with an overall accuracy of 82.33%. Gerke and Kerle6 proposed a two-step approach for building damage classification. This method yielded an overall classification accuracy of 70% for four classes and a building damage identification accuracy of 63%. Although high-classification accuracy can be obtained, it is premised on the airborne, oblique pictometry imagery available.
In addition to research that has used high-resolution optical images, other studies have been conducted based on SAR images to consider the impact of climatic conditions and occlusion on the optical images.78.9.10.–11 These methods have yielded satisfactory assessment results, but they cannot be generalized to other types of disasters, such as landslides. Thus, in this study, we design an automatic damage assessment framework for buildings in the aftermath of landslides according to the characteristics of the damage caused. The proposed framework first exploits object-based classification to detect landslides and combines the detection results and a building damage classification model (BDCM) to realize the entire assessment process. In particular, the main contributions of this paper are as follows:
1. An automatic assessment framework for building damage caused by landslide disaster based on VHR images is proposed. It can efficiently assess the degree of building damage, which can be helpful for subsequent disaster relief work.
2. An approach based on the modified object-based sparse representation classification (SRC) method and morphological processing is introduced for landslide detection.
3. A quantitative building damage classification scheme for landslide disaster is introduced. Based on this, a BDCM is proposed. The effectiveness of the model is also verified.
The remainder of this paper is organized as follows: In Sec. 2, we review related literature. Section 3 consists of a description of the morphological quantitative scheme for building damage caused by landslides. In Sec. 4, the proposed assessment framework is described in detail. Experiments to test the proposed framework are detailed in Sec. 5, and our conclusions are given in Sec. 6.
Landslide detection is clearly required before damage to a building can be assessed. In recent years, research on landslide detection has developed quite rapidly. Methods based on varying sensor data have been exploited to discriminate landslide-affected regions from the background and analyze them. Since optical images may carry many spectral characteristics and details of the target, they are widely used. Research on landslide detection based on optical images can be divided into two types: pixel-based methods and object-oriented methods. In opposition to the former, which mainly consider spectral and textural features,12,13 the latter class of methods enables the consideration of more complex and advanced features.1415.16.–17 To improve detection accuracy, some classical machine learning classification methods have been applied to remote-sensing image classification. These include support vector machines,18,19 artificial neural networks,20 and methods based on fuzzy theory.14,21 Studies show that these methods have achieved satisfactory classification effect based on VHR images.
Sometimes, weather in a disaster region is persistently foggy and cloudy following a hazardous occurrence. In this situation, SAR images are particularly useful due to their unique advantages of all-time and all-weather imaging.22 Li et al.23 found that landslide-affected areas show complicated scattering mechanisms because the landslide surface becomes relatively rough in the scale of the X-band. They distinguished landslide areas by using a supervised Wishart classifier and polarimetric scattering similarity parameters. This technique performs well in forested areas but is unsuitable for urban areas. Li et al.24 proposed another simple and effective approach for unsupervised classification of large-scale landslides. This involved first transforming PolSAR images from the RGB color space to the hue-saturation-intensity color space. A combination of -means clustering and hue-saturation-intensity imagery in different channels was then used stage by stage for automatic landslide extraction. This method is suited for large-scale landslide detection. Inspired by this approach, our framework involves the transformation of optical images from RGB to hue-saturation-value (HSV) space and the exploitation of a modified SRC method to detect landslide regions.
Since most natural signals can be satisfactorily represented by a sparse vector corresponding to the linear combination of a few dictionary atoms, sparse representation has been shown to be an extremely powerful tool in remote-sensing data processing. Chen et al.25,26 proposed a simultaneous joint sparsity model for hyperspectral image (HSI) target detection and classification. This exploits the fact that in HSI, neighboring pixels have similar spectral characteristics. Furthermore, they introduced a kernel sparse representation method27,28 that maps low-dimensional features to high-dimensional features and improves classification accuracy. Fang et al.29 proposed a multiscale adaptive sparse representation model for spectral–spatial HSI classification. It effectively exploits spatial information at multiple scales via an adaptive sparse strategy. Song et al.30 developed a classification strategy that applies extended multiattribute profiles to an SRC framework. This approach combines the advantages of sparse representation and rich structural information to provide efficient classification results for different multi/hyperspectral datasets. Furthermore, other sparse representation models have been proposed to improve classification accuracy, such as manifold-based sparse representation,31 regularized sparse representation,32 nonlocal weighted joint sparse representation,33 and sparse representation-based binary hypothesis.34
These sparsity-based classification methods have some common shortcomings that are mostly embodied in two aspects. On the one hand, they are almost at pixel level. Since the size of remote-sensing images is often very large, classification processing incurs high-time complexity. On the other hand, pixels in HSI are represented by -dimensional vectors, where is the number of spectral bands. If the number of HSI bands is small (such as three or four), pixel-based information is scarce, which leads to low-classification accuracy. To overcome these problems, we35 had proposed an object-based landslide and mudflow detection method that combined multichannel sparse representation and eight neighbor judgment methods. Using this, a large-sized remote-sensing image was divided into a number of uniform patches and each was vectorized as a processing object. This method may solve the above-mentioned defects, but much larger thresholds are needed in practice.
After further research, we have modified the SRC method to detect landslides. The implementation process is described in Sec. 4.
Building Damage Classification Scheme
By carefully observing the characteristics of landslide-affected buildings in VHR images, we found certain regularities in morphological changes in landslides that are different from those due to earthquakes. These regularities are as follows:
1. The affected buildings and landslides tend to have certain spatial relationships. The buildings tend to be located within or near areas hit by landslides. The distance between buildings and the landslide area appears to be inversely proportional to the degree of damage. That is, the shorter the distance, the more severe the degree of damage. The most severe degree of damage occurs when an entire building is buried under the landslide; buildings that remain intact in such cases are usually far from the landslide.
2. The affected buildings usually exhibit morphological changes. Some structures suffer obvious deformation due to the impact of mud. Some are surrounded or even invaded by a lot of rubble, and the texture of the roofs of certain buildings becomes very chaotic.
3. The degree of damage to a building is somewhat related to the depth of mud pouring into it. When a building is located along the edge of a landslide, mud usually pours into it, whereby the building is damaged but the texture of its roof remains unchanged. This is a specific but common form of damage in landslides.
Under the direction of experts at the Academy of Disaster Reduction and Emergency Management, operated by the Ministry of Civil Affairs and the Ministry of Education, we propose a quantitative building damage classification scheme for landslides according to morphological characteristics (Table 1). In this scheme, we classify affected buildings into four grades.
Building damage classification scheme applied to landslides.
|Damage grade||Grade name||Brief description|
|1||Intact||Intact, or negligible damage: far from landslide; no mud accumulation inside building|
|2||Slightly damaged||Slight structural damage; moderate nonstructural damage: close to or in contact with the landslide; shallow mud accumulation inside building|
|3||Heavily damaged||Severe structural damage: severe nonstructural damage; located along the edge of landslide; deep mud accumulation inside building, and apparent rubble around it|
|4||Buried||Complete collapse or washing away; most or all is buried underneath rubble; located within the landslide area|
We select two features to classify building damage according to the characteristics of two-dimensional remote-sensing images. They are (1) the spatial relationships between a building and the landslide area and (2) the roof texture. In order to calculate the spatial relationships, outlines of the building and the landslide are required. The former can be obtained either from relevant government departments or by hand annotation and the latter can be obtained by using the SRC-based approach.
Our proposed assessment framework is aimed at automatically classifying building damage when landslides occur. It consists of two parts—the SRC-based approach, which is used to detect landslides, and the classification model, which classifies affected buildings. The specific implementation process is shown in Fig. 1.
Sparse Representation Classification-Based Approach for Landslide Detection
According to the classification scheme, landslide detection, while imperative, is time-consuming. We studied the spectral characteristics of landslides and proposed an SRC-based approach for automatic landslide detection. It is composed of three procedures: dictionary construction, target detection, and morphological processing.
An overcompleted dictionary is crucial and necessary for the SRC method. Before the occurrence of a landslide, we collect a large number of typical sample patches ( size) from VHR remote-sensing images. These sample patches consist of landslide and background (nonlandslide) samples, each as a processing object. Since the landslide is mostly manifested in its spectral characteristics than in other features, and since the HSV color space is close to human color perception, we convert every patch from RGB color space to HSV.
In the HSV color space, hue is used to distinguish colors, saturation is the percentage of white light added to a pure color, and value refers to perceived light intensity.36 It is known that the method of calculating HSV spatial components might affect both recognition performance and computational efficiency. From practical experience, we quantify the HSV space component in unequal intervals. The quantization scheme is shown as
Since the sample patches are randomly selected, the constructed dictionary is redundant, and atom vectors are highly correlated. To solve this problem, we perform a preprocessing step called whitening, the goal of which is to make the dictionary less redundant. Thus, we construct an overcomplete dictionary where (i) the atoms are less correlated with one another and (ii) all atoms have the same variance.
With a test VHR image patch, a corresponding 256-dimensional HSV histogram vector is calculated in the same manner as that of an atom in the dictionary. According to the sparse representation theory, can be approximately represented by a sparse vector corresponding to the linear combination of all atoms, the mathematical representation of which is rendered asFig. 2. The landslide dictionary contains training samples and the background dictionary contains training samples. For a background patch, the sparse representation vector is shown in Fig. 2(a). We see that vector (red entries) corresponding to the landslide dictionary is a zero vector and vector (blue entries) corresponding to the background dictionary is a sparse vector. A whitened background sample histogram feature (black solid) and its approximation (red dashed) are shown in Fig. 2(b).
The sparse vector satisfies the equation , and the number of its nonzero entries should be as less as possible. So we get the loss function as26 is a preset upper bound on the sparsity level.37 In the absence of a priori information of the , the aforementioned problem is satisfactorily solved by the sparsity adaptive matching pursuit algorithm.38
The sparse vector is recovered by decomposing over the preconstructed dictionary . Thus, we may find the few atoms in best representing . The recovery process implicitly leads to a competition between the two subspaces ( and ). Once sparse vector is obtained, the residuals of recovery are calculated with the following equations:
According to the characteristics, a landslide should be a complete or continuous region, where each potential landslide patch should be an interconnection. Then in order to deal with broken or fragmented regions, some morphological processing operations, including the open operation, the close operation, and connected-component labeling, are applied to potential landslide patches. Some isolated patches should be eliminated if they are smaller than a given threshold and certain smaller holes should be filled. According to the average size of buildings and expert experience, our goal is to identify the landslide patch, whose area is larger than . Therefore, we set this threshold asFig. 3.
Building Damage Classification Model
In remote-sensing images, the outlines of the building and the landslide area embody polygons. We adopt the dimensionally extended nine-intersection model (DE-9IM) to analyze the spatial relationships of the two polygons.39 The DE-9IM model is a mathematical approach that defines a pair-wise spatial relationship between geometries of different types and dimensions. This model expresses spatial relationships among all types of geometries as pair-wise intersections of their interior, boundary, and exterior by considering the dimensions of the resulting intersections. It can be expressed as
There are many kinds of spatial relationship between two polygons. In this paper, we only need four. The corresponding spatial predicates are disjoint, touches, overlaps, and covers, and the matrix of the each predicate is as in Fig. 4.
Finally, based on the above input data and spatial relationship theories, we designed the BDCM on the basis of the previous work,40 which is shown in Fig. 5. In this model, we determine the building damage grade directly when the spatial relationship between the outlines of the building and the landslide areas are covered or overlapped. If the relationship is disjoint, the shortest Euclidean distance between two regions needs to be calculated. According to the experience of experts, 3 m is a suitable discrimination threshold. Therefore, we preset a shortest distance threshold to pixels ( is the resolution of the VHR image). When the shortest Euclidean distance is greater than , the degree of damage to the building is in the “intact” grade; otherwise, as in the “touch” relation, we need to calculate the texture of the roof of the building in the predisaster and the postdisaster images to classify the degree of damage as two grades according to textural differences. Since we find that texture is often not required, it is ignored in this paper.
In order to test the performance of the proposed framework, VHR images for the Wenchuan earthquake in China on May 12, 2008, were acquired and analyzed. This earthquake triggered massive landslides and mud-rock flow that caused a large number of casualties and massive damage to infrastructure. We used the QuickBird pre-event image acquired on June 26, 2005, and the QuickBird postevent image acquired on June 3, 2008. Both images were acquired in panchromatic and multispectral modes with spatial resolutions of 0.61 and 2.4 m, respectively. Following image fusion, we obtained the 0.61-m multispectral images used in our experiments.
The experiments were executed on a personal computer running on an Intel Core2 Quad CPU at 2.83 GHz with 4 GB of memory. Figure 6 shows images of Wenchuan County before and after the earthquake. One subset containing images of damaged buildings was marked as the experimental region and is shown bounded by a red box in Fig. 6.
Landslide Detection Experiments
Prior to damage grading, we cropped 4500 sample patches () consisting of 2000 landslide sample patches and 2500 background sample patches from several landslide remote-sensing images. For comparison, we calculated the HSV histogram features and the RGB histogram features for each sample patch. Finally, we concatenated the different color space histogram features and obtained two dictionaries.
The SRC method based on both HSV and RGB histogram features was applied to the experimental images of Wenchuan County, and the results were compared both visually and quantitatively by using receiver operating characteristics (ROC) curves. In our study, the identification of landslide patches and nonlandslide (background) patches is a binary classification problem, where outcomes are labeled either as positive (p) or negative (n). The ROC curve illustrates the performance of the binary classifier system as its discrimination threshold is varied. It is defined by a true positive rate (TPR) and a false positive rate (FPR), which are labeled as - and -axes, respectively. In our experiments, the class labels for all test patches were determined at each threshold. The TPR is the ratio of the number of hits (landslide patches determined as landslides) to the total number of true landslide patches, and the FPR was calculated by the number of false alarms (background patches determined as landslide) over the total number of background patches.
The pre- and postevent images of Wenchuan County used for the experiment are shown in Figs. 7(a) and 7(b), respectively. In order to evaluate our landslide detection method, the SRC experiments based on the above two dictionaries were performed and their experimental results were compared with the visual interpretation results. The visual interpretation results of the landslide areas from the National Disaster Reduction Center of China are shown in Fig. 7(c). Figure 7(d) shows the visual interpretation of landslide areas overlying the postevent image. The RGB histogram-based classification results are shown in Figs. 7(e) and 7(f), whereas Fig. 7(e) shows the binary classification results and Fig. 7(f) shows the final landslide detection areas overlying the postevent image. Similarly, the results of the HSV histogram-based classification are shown in Figs. 7(g) and 7(h). Both experiments employed morphological processing. To eliminate isolated potential landslide patches, the area threshold was set 540 pixels automatically, equivalent to . The corresponding ROC curves are shown in Fig. 8, which shows that the SRC method using the HSV histogram feature dictionary was more effective.
Experiments on Building Damage Classification Model
After obtaining landslide detection results through the SRC-based framework, we easily obtained the outlines of the landslide area. The outlines were entered into the BDCM. Meanwhile, the outline of the building obtained from the cooperation in advance was also entered into this model. In this experiment, the shortest distance threshold was set as 5 pixels, approximately equal to 3 m (). Finally, the model assigned a damage grade to each affected building (Fig. 9). The assessment results were then displayed, and all kinds of assessment reports could be generated depending on the user’s requirements.
In this case, the total number of buildings was 106, and the number of correct classifications was 92. The assessment results are depicted by a confuse matrix shown in Table 2. Here, the rows represent assessment results obtained using our method, and the columns show the reference data from the National Disaster Reduction Center of China. The results were satisfactory, with high producer accuracy and user accuracy. The overall accuracy was 86.79%, and the Kappa coefficient was 0.82.
The confuse matrix display of assessment results.
|Intact||Slightly||Heavily||Buried||User’s accuracy (%)|
|Producer’s accuracy||97.3||82.14||68.18||94.74||Overall accuracy (%) = 86.79|
In this paper, we developed an effective and reliable assessment framework to classify building damage caused by landslides. The framework can yield satisfactory results by taking advantage of VHR optical remote-sensing images. This framework not only exploits the advantages of remote-sensing data, but also reduces the participation of experts. Experiments showed that the framework can provide effective support for emergency disposal work and disaster management. Since the framework has been developed for VHR images, its performance will be affected by low-resolution images. Further modification and extended research will continue on this front.
This research was primarily supported by the National Natural Science Foundation of China (Grant No. 61501035), the Fundamental Research Funds for the Central Universities of China (No. 2014KJJCA15), the National Key Technology R&D Program of the 12th Five Year Plan of China (No. 2012BAK10B03), and the High-Resolution Special Scientific Research Project from the National Disaster Reduction Center of China, Ministry of Civil Affairs (No. 03-Y30B06-9001-13/15).
Bo Sun received his BSc degree in computer science from Beihang University, China, and his MSc and PhD degrees from Beijing Normal University, Beijing, China. He is currently a professor at the Department of Computer Science and Technology, Beijing Normal University. His research interests include pattern recognition, natural language processing, and information systems. He is a member of the Association for Computing Machinery and a senior member of the China Society of Image and Graphics.
Qihua Xu received his BSc degree from Southwest Normal University and his MSc degree in computer application technology from Renmin University of China. Since 2009, he has been with the School of Business, Northwest Normal University, Lanzhou, China. He is currently a PhD candidate majoring in computer application technology at Beijing Normal University. His research interests include machine learning and pattern recognition.
Jun He received her BSc degree in optical engineering and her PhD in physical electronics from Beijing Institute of Technology, Beijing, China, in 1998 and 2003, respectively. Since 2003, she has been with the College of Information Science and Technology, Beijing Normal University, Beijing, China. She was elected as a lecturer and an assistant professor in 2003 and 2010, respectively. She is currently teaching and engaged in research. Her research interests include image processing applications and pattern recognition.
Zhen Liu received his MSc degree in cartology and remote sensing and his PhD degree in remote sensing and geographic information systems from Beijing Normal University, Beijing, China, in 1998 and 2004 respectively. He was a postdoctoral scholar with UC Irvine. He is currently a full professor and the director of the Center of Information and Network Technology in Beijing Normal University. His research interests include feature extraction and change detection using remote-sensing images, spatio-temporal modeling and applications.
Ying Wang is an experienced researcher at Beijing Normal University, a consultant with China’s National Disaster Reduction Centre, Department of Disaster Relief and Ministry of Civil Affairs, China’s Ministry of Science and Technology. He has researched earthquake-induced hazard chains, comprehensive risk assessments, reconstruction, rural livelihoods, assessing rural housing vulnerabilities, emergency responses, postdisaster resettlement, recovery modeling, agricultural insurance, evacuation training, and brought scientific knowledge into communities to mitigate risk and maximize resilience in Yunnan and Sichuan.
Fengxiang Ge received his BSc and MSc degrees from Harbin Engineering University in 1996 and 1999, respectively, and his PhD degree from Tsinghua University in 2003. From 2003 to 2005, he was a research associate in the University of Hong Kong. In 2005, he joined Intel Corporation as a senior researcher and architect. In 2011, he joined Beijing Normal University as an associate professor. His research interests include signal processing and its applications.