Sea–land segmentation in synthetic aperture radar (SAR) images is an important step for several important tasks, such as coastline extraction, ship detection, and coastal monitoring.12.3.4.–5 However, sea–land segmentation in SAR images is not as a simple work as in photographic images. In the latter one, thresholding methods, for instance the Otsu's method6 and the local adaptive threshold method (LATM),7 are frequently used and achieve satisfactory segmentation results. The difficulties brought by SAR images mainly lie in two aspects: one is poor sea–land discrimination, which may lead to discontinuous boundaries, the other is the presence of the speckle effect.1,2 The first one is that the retuned SAR signal from the rough sea surface may exceed or be equal to that from the nearby land area. The second one, which is generated by scatters that are smaller than the SAR resolution elements, makes the segmentation task more complicated. As a consequence, sea–land segmentation in SAR images is difficult even for experienced interpreters.
Several methods that aim to solve the problems in SAR images have been proposed in the last decade.12.–3,5,8,9 Among them, in the era of the single SAR images, the commonly used sea–land segmentation methods can only exploit the magnitude information and cannot achieve ideal results. As a comparison, recent algorithms using polarimetric SAR (PolSAR) data are populated thanks to the successful launch of several PolSAR satellites. PolSAR offers a brand-new view for sea–land segmentation since it provides more features compared with single PolSAR.6,10 These multipolarization features that can represent different scattering mechanisms are extracted from scattering decomposition and eigenvalue analyses.11,12 The existing methods that utilize SAR polarization information mainly focus on simply using the multipolarization features by several typical image processing methods, such as gray-level thresholding,5,8 machine learning,9 and multipolarization feature-based segmentation.10,13 As a result, the limitations caused by ambiguous sea–land separation and the speckle effect are not fully tackled by current methods.
In addition, with the improvement of the spatial resolution of SAR sensors, more geomorphic information is presented in SAR images in detail, which makes the segmentation problem more challenging. For instance, the tidal creeks on the intertidal area may be considered as land while some elongate structures may be difficult to detect. In this case, correct description of the boundary details is critical to the segmentation accuracy.3
In this paper, we propose a graph cut (GC)-based approach integrating multipolarization features and edge information to address the sea–land segmentation problems for SAR images. GC is an efficient framework that can achieve satisfactory two-value segmentation by two steps: transferring an image to an undirected graph and finding a cut through the graph with minimal cumulative cost.3,10 GC can find a globally optimal segmentation for an image and generate a good balance of local terms at the same time. In GC, the impact of the ambiguous boundaries can be reduced by the local term, which incorporates neighborhood information and ensures a continuous shoreline.14 However, traditional GC belongs to the category of semisupervised segmentation methods that require users to specify foreground and background seeds for prior modeling.15 In this paper, improvements have been made on the basis of the GC framework in the view of the PolSAR data to achieve sea–land segmentation accurately with automatic seed selection. Three powerful multipolarization features are extracted from the quad-PolSAR data, i.e., the total power span, the polarimetric entropy , and the mean scatter angle . For each pixel in the PolSAR image, a vector is formed by the three features as its multipolarization descriptor. Automatic seed selection can hence be possible by locating the pixels in the plane based on the scattering mechanism of the sea and land. The selected seeds enable us to build the prior models for GC. Problems raised from the speckle effect are addressed by introducing edge constraint when modeling the boundary item in GC. The edge constraint is extracted by an edge map that is generated from the four-channel PolSAR image by the ratio of average (ROA) algorithm, which is significant for improving its speckle noise depressing ability.16 Our work significantly differs with previous works in three aspects:
1. Reliable and sufficient seeds are selected in a completely automatic way based on the plane and enable us to build the prior models for GC.
2. A multipolarization descriptor that fuses the total scatter power span, polarimetric entropy , and mean scatter angle is proposed to build the graph model in GC.
3. Edge map that is generated from the PolSAR images by the ROA algorithm is used as edge constraint for modeling the boundary term in GC. The edge constraint is helpful in both reducing the speckle effect and avoiding the under-segmentation for some thin and elongated structures.
In this section, we propose a sea–land segmentation method for PolSAR images. The proposed method can be divided into three parts. First, the decomposition11 is applied to extract multipolarization features. The latter is used both in seed selection and the forming of the multipolarization descriptor. Then, the edge map is generated from the four channels of quad-polarization images by the ROA algorithm. Finally, an edge-constrained GC was applied to separate sea and land. Figure 1 shows the flowchart of our method.
Multipolarization Descriptor and Automatic Seed Selection
PolSAR has attracted much attention since it can provide much more detailed information in comparison with single PolSAR in variant applications. In the sea–land segmentation case, we extract multipolarization features as descriptors to characterize the properties of the sea and the land. Based on the multipolarization descriptors, reliable seeds are selected to build the prior model for GC, and the undirected graph in GC is constructed subsequently.
Synthetic aperture radar polarimetry features
PolSAR data can offer a different view on describing the observed target surface by methods that exploit the combined information of the backscattering coefficients, among which the most famous one is the decomposition.11 The definition of the decomposition is based on the polarimetric coherency matrix12
Another common multipolarization parameter is the span, which is the total power of the polarimetric channels. Span is a combination of the information of the four channels and can offer more details than them.11 The definition of span is given asFig. 2, in which the polarimetric entropy is demonstrated in 2(a), the mean scatter angle is shown in 2(b), and 2(c) gives the image of the total power span.
Automatic seed selection by plane
We have extracted three powerful multipolarization features from the PolSAR data, i.e., the total scatter power span, polarimetric entropy , and mean scatter angle . Based on polarimetric entropy and mean scatter angle, Cloude and Pottier13 proposed a two-dimensional plane that can represent an all random scattering mechanism, given in Fig. 2(d). This plane is subdivided into eight basic zones to characterize different classes of land covers based on their scattering mechanisms.
In the sea–land segmentation case, the sea areas usually have both low entropy and scatter angle values for its smooth surface, whereas the values of the land parts are usually relatively high for its relative complicated texture distributions. As a consequence, seeds selection for the sea and the land will be possible by locating the pixels in the plane based on its entropy and scatter angle. Figure 2(d) shows the locations in the plane of some sample pixels. In Fig. 2(d), locations of the land area marked as red box in Fig. 2(c) are represented as red points, and the blue points indicate the locations of the sea area that are marked as blue box in Fig. 2(c). It is clearly seen that the sea area mainly fall into the category with entropy value less than 0.5 and value less than 42.5 deg, i.e., zone 8 as marked in Fig. 2(d).11 To this end, the seeds selection work for the sea can be simplified by threshold works based on the plane. In this paper, pixels that have values lower than 0.3 and values lower than 30 deg are selected as seeds of the sea area. The seeds of the land are selected from the pixels with entropy values higher than 0.4 and with values higher than 45 deg.
Ratio of Average-Based Edge Map Generation
The speckle appear in SAR images will lead the inaccuracy in sea–land segmentation work. Based on the multiplicative noise model, several studies have been developed to smooth the speckle noise. Among them, the ROA algorithm is often used in the present studies to address this problem in a variety of applications.
ROA is one of the most efficient edge detectors for SAR images, which belongs to the class of constant-false-alarm-rate operators and can be used to eliminate the speckle effect in SAR images by employing the ratio of neighboring pixels.
In ROA, a moving window is segmented to two subwindows P and Q along the assumed edge direction. Figure 3 shows the mask of ROA with four typical directions.13,17 For every direction, the ROA is expressed as
The resulting edge map can delineate the boundaries excluding the impact of speckle noise. Consequently, accurate sea–land segmentation will be possible by introducing the edge map into the GC framework.
Edge Constrained Graph Cut
We have proposed multipolarization descriptors to characterize the sea and the land by extracting features from the PolSAR data. In this end, seeds of the sea and land are selected automatically. Furthermore, edge map is generated by ROA depressing the speckle effect. Based on these, we accomplish the final segmentation work with our edge-constrained GC framework.
GC is an efficient two-class segmentation framework that is widely used in image segmentation works due to its compatibility and robustness. In GC, each pixel in an image is represented as a node in an undirected graph, and the image segmentation work is achieved by cutting through the graph with minimal cumulative cost, which is a typical energy minimization problem. The energy function for GC can be expressed as3 Based on the image and the edge set , an undirected graph can be built. In , each node is connected to its neighboring pixels by so-called -links and to the two terminal nodes by -links. The -links correspond to the global term and the -links correspond to the boundary term in the energy function .
The most crucial step of constructing the graph is the assignment of link weights. In this step, we form a multipolarization feature space by the three features in the multipolarization descriptor, i.e., the total power span, the polarimetric entropy , and the mean scatter angle . Prior models of the sea and land are built by the Gauss mixture models (GMMs) in the multipolarization feature space. In our experiments, the numbers of the components of the GMMs are 3 and 4 for the sea and land, respectively, because the land area usually has a relatively complicated distribution of texture. As a matter of fact, we find that the numbers of the components have little influence on the final segmentation results once they are selected in a reasonable range from 3 to 5.
Based on the GMMs, the -links of our graph model are defined as
When calculating the -links, we use the spatial distance between two multipolarization feature vectors and combine it with the edge maps that are generated from ROA, given by
Term is important in two aspects: one is smoothing the speckle noise around the boundaries and the other is detecting some thin and elongated structures.
In the experiment, the proposed method was evaluated and validated on the Radarsat-2 C-band SAR data. The processed images are selected from Radarsat-2 quad-polarimetric fine-resolution SAR images over Shanghai and San Francisco as described in Fig. 4. In Fig. 4, (a) is selected from data over Shanghai with size and others are selected from data over San Francisco with size , , and , respectively. The ground truth images are labeled by our interpreters carefully to evaluate the experimental results. Comparative experiments are carried out based on subimages that are selected from the processed images including scenes of complex distribution of texture and complicated structures.
Figure 5 shows the intermediate processing and segmentation results of the four selected images in Fig. 4. Figure 5(a) shows the span images the same as those in Fig. 4. Figure 5(b) shows the selected seeds for the land and sea that are displayed in red and blue, respectively. The seeds selection results can efficiently mark the sea and land. Edge maps that are generated from the ROA algorithm are shown in Fig. 5(c), in which the edges are delineated clearly. Figure 5(d) shows the final segmentation results, which can ensure continuous shorelines. In Fig. 5, the tidal creeks are detected accurately as described in the first row, and some slender structures are classified correctly as described in the third row. Figure 5 shows that the final results of our method can offer good visual matches compared with the ground truth that are given in Fig. 5(e).
Quantitative and Visual Comparison
We have demonstrated the effectiveness of the proposed method as applied to the Radarsat-2 PolSAR data. To be more complete, comparative experiments are carried out based on three baselines, i.e., GC15 without edge constraint and multipolarization descriptor, Hansch’s algorithm,10 and LATM.7 Among these three methods, GC is the basic framework of the proposed method, Hansch’s algorithm is an extension of the GC framework for PolSAR data, and LATM is a typical thresholding method. In the comparative experiments, the traditional GC and Hansch’s method use the same seeds used by our method to do the segmentation work. The building of an undirected graph is similar to the proposed method while the differences are that the multipolarization feature vector is replaced by the simple span image in building the GC model and the -link weights are calculated without the edge constraints. For the LATM, the inputs are the span images. In the experiments of the Hansch’s method, we use the same multipolarization feature vector as our method to build the -links and use the complex Wishart distribution to build the -links. In the experiments of traditional GC and Hansch’s method, the is set to 10 and the number of the components in the GMMs is set to 3 and 4 indicates the sea and land separately. We select subimages from the four testing images with complex shorelines and complicated structures. Visual illustrations of the segmentation results of different methods are presented in Fig. 6.
The results of the comparative experiment demonstrate that the results of our method are spatially consistent with few errors compared to the ground truth owing to the reliable seeds and speckle-reduced edge constraints. Specially, our method outperforms the comparative methods for the scenes that have thin and elongate structures, as presented in the first and third row in Fig. 6. Images in the second and fourth row in Fig. 6 demonstrate that our method performs well in the complex coastline case. The last row of Fig. 6 shows that detailed information can be extracted by our method accurately.
Quantitative analysis work is carried out by four measurement methods, i.e., recall of the land (ROL), precision of the land (POL), recall of the sea (ROS), and the precision of the sea (POS).3 We compute these measurements on the pixel level for the testing images in Fig. 6. In addition, we calculate the standard deviations (STD) of the four measurements for the testing images.
Table 1 presents the average ROL, POL, ROS, and POS that are calculated by the average of each image, and our method performs best on all the four measurements. Furthermore, the STDs, i.e., the STD of the testing images indicate that our method is the most stable one. Among the four measurements, the three comparative methods achieve relatively small ROL values owing to the ambiguous nature for sea–land separation in SAR images.1,2 In addition, POS values are small owing to the wind-roughed and speckled sea surface in SAR images, which may cause misclassification. All of these posed difficulties are caused by the presence of speckle and the returned signal from the roughed sea surface. By employing the ROA algorithm and multipolarization descriptor, our method addresses all these difficulties and achieves the best results.
Average ROL, POL, ROS, and POS and their STD obtained with different methods. The STD indicates the standard deviation and the bold values indicate the best.
|Method||ROL±STD (%)||POL±STD (%)||ROS±STD (%)||POS±STD (%)|
|Method by Hansch|
Effects of Introducing Edge Constraints
The incorporation of the ROA-based edge map in our GC framework is aimed at reducing the speckle noise and avoiding under segmentation for some slender structures. To further validate the effectiveness, we conduct comparative experiments with the canny edge directed GC that is proposed in Dongcai Cheng’s paper.3 In Cheng’s work, the edge constraint in GC is the edge map that is generated by canny detector rather than ROA. The comparative results are shown in Fig. 7, in which (a) indicates the span images, (b) represents the canny edges, (c) shows the results of canny edge directed GC, and the next two figures demonstrate the edge maps and segmentation results of the proposed method. Figure 7(f) shows the ground truths.
Experimental results show that our method is effective in some cases such as the thin structures, which may be easily corrupted by the speckle noise in SAR images, as presented in the first row of Fig. 7. The second row of Fig. 7 shows the results of a subimage with complicated distribution of texture and inadequate boundaries. The latter case may be caused by the signal returned from the water area that shares similar intensity compared with the nearby land as well as the speckle effect. In this case, our method outperforms the canny edge directed GC and achieves satisfactory results.
Influence of Parameters
Selection of in graph cut
The parameter balances the relative importance of the -links and -links in GC. When is small, the segmentation results are dominated by the -link weights, i.e., the global information and may lead to discontinuous boundaries. However, when the value of is too large, the image will be over-smoothed and cause under-segmentation around the boundaries. In both cases, the accuracy of the sea–land segmentation will be decreased.
To validate the influence of the value of , this paper sets different values of and compute the average ROL, ROS, POL, and POS on the testing images. Specially, the -measure is used as the balanced performance measurement,18 given byFig. 8; second, the value of is chosen in a finer range from 10 to 90 as represented in the right figure of Fig. 8. The combination of both figures in Fig. 8 validates that the accuracy reach the best when the value of is set to 10.
Influence of the number of components in the Gauss mixture models
In our method, the GMMs are used to fit the distribution of the selected seeds of the sea and land. To study the influence of the number of components in the GMMs, we set different pairs of (sea, land) components in a range from 2 to 5 and compute the average ROL, ROS, POL, and POS on the testing images. As the land area usually has a relatively complicated distribution of intensity, we set a larger number of the components in the GMMs for the land area. The results are represented in Table 2, from which it can be seen the numbers of the components have little influence on the final segmentation results once they are selected in a reasonable range from 3 to 5.
Influence of the number of the components in the GMMs.
|Number of components in the GMMs (sea, land)||ROL (%)||POL (%)||ROS (%)||POS (%)|
A GC-based approach is proposed in this paper to address the sea–land segmentation problems for PolSAR images. In our work, the ambiguous sea–land separation problem is addressed by exploiting the neighborhood information in GC. Specially, we employ the multipolarization features to select seeds for the sea and land automatically and build a graph model for GC. The accurate segmentation result is achieved by utilizing the ROA-based edge constraints, which can address problems associated with the speckle effect around the boundaries as well as the under-segmentation for some finer structures. We conduct experiments on two sets of Radarsat-2 PolSAR data gathered over Shanghai and San Francisco and show our algorithm outperforms three traditional methods with nontrivial margins, which demonstrates the effectiveness of our method.
Further improvements in accuracy for the purpose of geographic mapping and land use classification would require more powerful multipolarization feature and additional modifications in the GC framework.
Xiaoqiang She received his BS degree from the University of Science and Technology of China (USTC), Hefei, China, in 2011, and his MS degree from the Institute of Electronics, Chinese Academy of Sciences (IECAS), Beijing, China, in 2014. He is currently working toward his PhD at the Key Laboratory of Technology, Geo-spatial Information Processing and Application System, IECAS. His current research interest is PolSAR image processing.
Xiaolan Qiu received her BS degree in electronic engineering and information science from the University of Science and Technology of China, Hefei, China, in 2004 and her PhD in signal and information processing from the University of Chinese Academy of Science, Beijing, China, in 2009. She is currently a vice professor of the Institute of Electronics, Chinese Academy of Sciences, Beijing, China. Her research interests include SAR image processing and SAR signal processing.
Bin Lei received his BS degree in Tsinghua University, Beijing, China, in 2000 and his PhD in signal and information processing from the University of Chinese Academy of Science, Beijing, China, in 2014. He is currently a professor of the Institute of Electronics, Chinese Academy of Sciences, Beijing, China. His research interests include SAR image processing and SAR signal processing.