With the development of highway and rural roads, road networks have recently become more and more complex, and many new roads are constructed every day. In order to utilize and manage the roads’ network effectively, sketching the roads’ network based on aerial images is a significant task, but it is difficult to make sure that all aerial images are clear and of good quality. In addition to the influence of the changing weather on image acquisition, automatic road detection and identification can be greatly affected by complex infrastructures, farmland, and mountains.
Fischler did road detection with multisource aerial images 30 years ago. His detection could only be done on low-precision images but was not available for high resolution images. Later, many researchers provided the different detection methods in this topic.1 Wen and Wang2 studied an algorithm to automatically extract the main objects on roads from remote sensing images in 2000 where the roads are wide. One year later (2001), Cai and Wang3 proposed an algorithm for extracting main road networks from multiband remote sensing images. In 2004, Zhu et al.4 processed high resolution remote sensing images to get the roads and their details based on morphological segments,4 but not for thin roads. In 2007, Yang and Wang5 used the scene perception of remote sensing images and image classification to detect and analyze roads. In the same year (2007), Luo et al.6 suggested the method of combining spectrum and shape features to extract roads for clear images. Also, in 2007, Hu et al.7 did some related work based on aerial image characteristics; they analyzed road networks mainly by using the crossings and corners of the detected roads. Cem and Beril8 studied road network extraction from high resolution aerial images by using probabilistic and graph theoretical methods. In the past 30 years, this topic has become hotter, but there is no standard algorithm for detecting thin roads or identifying roads in vague aerial images under bad weather.
When the weather is bad, aerial images are blurred. One hard main task is to enhance aerial images of poor quality, because bad weather often causes aerial or remote sensing images to be blurred and reduces visibility. On the basis of the existing systems, the image enhancement is the foundation for solving this kind of problem. There are many methods to process complex and vague images, and one of them is presented by Eitan. He studied an algorithm by using hierarchy and an adaptive idea in segmenting visual scenes. This method can accurately segment butterflies from the background of similar textures and colors in grass, or tigers and leopards from the background of similar rock patterns, and so on.9 In 2007, Lawrence10 described a new image enhancement method which emphasized the importance of image enhancement. It stated that it was essential to enhance and process images according to image features. In 2007, Oakley and Hong11 published a paper about a method of contrast correction in color images, especially for fog images. Pu et al. also studied the detection of weak texture information by using a fractional differential.1213.–14 However, all of the above image enhancement algorithms were not for thin and vague road detection in aerial images.
Due to the low contrast of aerial images taken in bad weather or in complex situations, no matter whether they are among objects or between objects and the background, the thin or vague roads with a low contrast in gray scale (luminance) or a small color difference are easily ignored by a normal image processing method. Also, a slender road image is often mixed in with a variety of natural features and man-made facilities; sometimes, it is even difficult to identify these with human vision. In this case, a complex road network is hard to identify through the approach that is based on gray scale similarity.15 It is also difficult to detect ridge-like roads by using a conventional edge detection method and even when applying an improved smart Canny edge detection algorithm,16 some thin and small lines or curves cannot be completely detected in most cases. In general, roads in aerial images or remote sensing images are ridge-like linear objects, so developing a special ridge edge detection algorithm for the images is important. Wang17 studied a valley edge detecting algorithm for froth images. It is valuable for froth images because the valley edges in an image can be roughly detected by one threshold. The road edges in an aerial image cannot be detected by using only one threshold because of the effects from different objects. Hence, when Wang’s algorithm is used for thin or small road detection in an aerial image, it is hard to determine a correct gradient threshold: a low threshold will produce a lot of noise and a high threshold will lose useful information which influences road identification. Therefore, the studied method in this paper is first to enhance ridge edges (roads) based on a fractional differential, then detect thin or vague roads with a newly designed ridge edge detecting algorithm. Of course, for the whole method of detecting thin or vague roads, although ridge detecting is a significant step, more post functions are needed for the final road identification.
Ridge Edge Detection Algorithm for Thin and Vague Roads
In aerial images, there are complex crisscross road lines; sometimes they are clear, sometimes they are not clear due to sunlight, aircraft jitter, or complex ground scenes. In the unclear cases, the image processing technology is required to enhance images for detecting roads as accurately as possible. For a low precision image, a road maybe a ridge or valley line (curve) with a certain width, whereas for a high precision image, in addition to the above properties, the solid and dashed lines in a road are also ridge edges. In most aerial images, there are woods, roads and railways, grass and crops, residential buildings, rivers, and mountains, etc., which lead to the complexity of ground scenes. Hence the roads are difficult to detect by a conventional image processing method. In most cases, for such a complex image, the first- and the second-order differential operators cannot accurately detect all roads, and the operators often extract only the high contrast objects and partial roads as shown in Fig. 1 (gray scale image) and Fig. 2 (color image), where, though the roads are ridge edges, some nonroad objects are similar to the roads. These features are clearly observed on the gray level histogram of the image section profile, and this is why traditional edge tracing algorithms can only detect partial roads that have obvious edges. So, a new ridge edge detection algorithm is studied in this study according to the above characteristics.
In Fig. 1, the diagonal black line in the left picture is a section profile line, and from bottom to top its gray level histogram is shown in the right picture. In the left picture, the two vertical green lines correspond to the end points of the profile line, and in the histogram, from left to right, the five gray level peaks represent the five roads (see the arrows in the histogram corresponding to the arrows from bottom to up in the left picture).
When analyzing the above information, without connecting with the other objects with the same gray scales, the roads are ridge edges (such as the first and the fifth roads), which are easily detected. But a road is difficult to detect when the gray scale value of the ridge edges is too low, e.g., the fifth road. If a road is connected to or close to neighboring similar objects with the similar gray scales, the peaks of the histogram show the rectangles or closely linked multiple triangles, which creates difficulty for road detection.
The profile line in Fig. 2 (left picture) intersects (from bottom to top) with eight roads with ridge edges (see the arrows from left to right in the histogram corresponding to the arrows from bottom to up in the left picture). In fact, 24 obvious peaks are shown in the histogram, but it is difficult to judge the corresponding roads due to noise.
In order to detect these ridge edges (Figs. 1 and 2), this paper particularly designs a new image edge detection algorithm. It includes several image processing subalgorithms. The result of edge detection is a binary image without using any thresholding operation. The detecting rules are: (1) For each detecting point, in the four directions that are on horizontal, vertical, and the two diagonals, the algorithm checks if the point is a ridge edge point. If this is true, it is chosen as one ridge edge point candidate; (2) To avoid the impact of noise, the gray scale value of the candidate point must be higher than that of the weighted average value of neighborhoods in the same direction; (3) To finalize the detection result, a number of postprocessing functions are studied, such as deleting isolated points, removing extra lines and points through a thinning operation and a length thresholding function, and detecting the remaining candidate points with a template [since the short line is of less than 3-4(7/2) pixels] to remove the noise points or short lines/curves. The ridge edge detection procedure is presented as follows.
For each of the detecting points, the algorithm measures its magnitude gradient values in the four directions, respectively, and determines which direction is most likely to be a ridge edge point orientation. The detection kernel size can be when a road width is less than 6 pixels; the weight of each point in the kernel is inversely proportional to the distance from the detecting point. An example of a kernel is illustrated as follows (see Fig. 3). Assuming an image is , and the current detecting point is . Figure 3 is for the detecting kernels in three different directions respectively: Figs. 3(a), 3(b), and 3(c), respectively, represent 0 deg (90 deg is similar), 45 deg, and 135 deg detecting kernels. There are five straight lines in each of the kernels, is in the middle line, above it we have and , and under it we have and . Each grid in the figure represents a pixel and the label in each grid is for the pixel location in the image.
As the roads are straight lines or curves, using several pixels instead of a single pixel for the detection can increase the accuracy of the detection result. In a template, in the detection procedure for each kernel, the algorithm detects 3 to 4 pixels in a line as shown in Fig. 3, rather than detecting an isolated pixel,17,18 which can reduce noise. The postprocessing is for identifying the candidate lines. As an example of a template for an image , the detecting rules in four directions can be presented as
In the above equations, the situation in 135 deg is taken as an example. If ; ; ; , the algorithm calculates , where, (, 2, 3, 4) is the weight variable, such as set , . In the same way, , , can be obtained, too. is a preset threshold, and as increases, the number of ridge points decreases. If , this point is a candidate ridge point and its direction is marked. In the candidate points, many noise points may exist. To remove these noise ridge points, the following postprocedure is suggested.
First, the algorithm connects the near neighboring ridge points in the consistent direction, then removes some noise or isolated ridge points. After finding a short curve or a short line (e.g., less than 4 pixels), its two endpoints are detected. After that, the possible forward direction for each endpoint is estimated, and then in each direction, its two or more neighborhood points are smoothed as a line and its new direction is obtained. To easily understand the above algorithm, the algorithm working steps and the detection result are shown in Fig. 4.
Figure 5(a) presents a fuzzy aerial image including several roads. By comparing the above new algorithm to the four conventional edge detection algorithms, the testing results are shown in Figs. 5(b)–5(d). The Canny algorithm gives the binary image result which is similar to that obtained by using the Sobel algorithm. It also includes many noisy lines and partial bifilar lines, and creates troubles for detecting road locations accurately. The LOG algorithm can detect the main roads, but will miss some small or thin roads. The new ridge detection algorithm can detect all the roads in the image very well with a binary image result, unlike the LOG result which only presents the gradient images and also needs an appropriate dynamic threshold to turn a gradient image into a binary image. Although the result of the Canny algorithm is a binary image, it needs to predetermine two thresholds to make gradient image binarization. In addition, the road detected by the new algorithm is of one pixel width (not a bifilar line), so it can give the exact location of a road. Although the detection result of the studied algorithm includes some noise, most of noise can be filtered away according to predetermined thresholds of the length or area. The image in Fig. 6 is for a city block image, which is more complicated than that in Fig. 5; the Canny edge detector cannot detect all the roads, but when the new algorithm is used on it, the roads can be very well detected. After primary edge detection, nonroad edges or textures need to be filtered away. The noisy edges can be smoothed away by the procedure in Sec. 3.
Thin and Vague Road Identification in Aerial Images
After wide roads are detected by using other methods, thin or vague roads can be detected by the above new algorithm. To apply the above ridge edge detection algorithm to aerial images for thin and vague road detection and identification, it also needs to merge other image processing functions and algorithms in image preprocessing and postprocessing. In this way, the whole road detection method is constructed. Since the detected roads are thin lines or curves, the traditional algorithms are difficult to use. Figure 7 shows the workflow of the proposed method procedure.
(1) Input an aerial image, then smooth the image with a Gaussian filter ( kernel, it is the same kernel size as that in the above edge detection) to reduce noise. One approach to design the Gaussian filter is to directly calculate template weights from a discrete Gaussian distribution. For simplicity, the filter weights are integers. A specific template is as follows:
(2) Determine the scale of the image shrinking according to the average road width, with the principle to ensure ridge points exist. In order to reduce the workload of the ridge edge detection and the effects of nonedge points, the image is shrunk (, 2, 3) times. When the average road width is between 6 and 10 pixels, the image is shrunk one time, and when it is above 11, the image is shrunk two times. The purpose of the image shrinking is to make thick roads thin and to enhance vague roads. When shrinking an image, in the adjacent 4 or 9 points, the minimum or maximum (depending on whether the roads are valleys or ridge edges) gray scale value is chosen for the corresponding pixel in the shrunk image;
(3) In general cases, although the conventional first- and second-order differential algorithms can highlight the object border information in an image, they are often only available for the apparent boundaries. For those thin and vague borders and slight edges, they are unavailable to detect and enhance roads. A fractional differential (0.1 to 0.9 orders) algorithm can detect or partly detect the vague and slight edges of roads, so it can be applied to the vague aerial images. Tested for a number of aerial road images, here the fractional differential (Tiansi) kernel of order (Refs. 1213.–14) is used, as shown in Fig. 8.
(4) Detect ridge edge points [see Eqs. (1)–(4), the threshold is set as 6 for the image in this study: aerial images of resolution ], and then identify the objects by combining the ridge edge image with the original image. For identifying roads, first remove all the lines less than 4 pixels (the shortest lines), and then detect endpoints for the remaining lines or curves. Each road is a long-thin region, so the endpoints can be connected based on their distances and directions; finally take away all the lines if their endpoints cannot possible be connected to the others.
(5) Merge the line objects (by using Hough line detection). For the objects neighboring each other, first link the two linear objects that can be easily merged, and then merge the remaining linear objects. The merge rule is to detect endpoints for each object first, then to connect the two neighboring endpoints by using the difference thresholds of the distances and covertures/orientations, where the distance threshold is 10 pixels (about 1/2 of short road length), and the curvature/orientation difference threshold is 40 deg (less than 1/2 of 90 deg).
(6) Smooth away the nonlinear like objects. Since a linear object has a large ratio (say, three or more) between the length and width, nonlinear objects can be gradually taken away by the ratio threshold, e.g., a ratio less than three can be smoothed away.
(7) After the above steps, if the image has been shrunk, map the identification results (enlarge the image size to the same size of the original image) into the original image, then for each line point, test to see if there are other neighboring points that can replace it with the principle to make the edge smooth in the original image.
In experiments, a number of images are tested. In the following, the three typical representative aerial images and their testing results are presented. In the three images, one (Fig. 9) is a simple aerial image where several roads intersect and the image background is not complicated; the second image in Fig. 10 is a farmland image where the image surface is rough with different colors and textures; and the third image in Fig. 11 is more difficult to deal with than the others since it is more complicated. The proposed algorithm is compared to the conventional edge detection operators as above mentioned, but here the new algorithm is mainly compared to the Canny algorithm, because the final results of the two algorithms are binary images, not gradient magnitude images, and the Canny algorithm is the best one of the most conventional algorithms.
Figure 9(a) is for an aerial image with obvious roads, and the roads have similar colors as that of farmlands and their surrounding block objects, in which case the roads are hard to detect only based on colors or gray scales. The image histogram is shown as in Fig. 1.
In this situation, it seems that the result of the fractional differential algorithm shows the clear roads [Fig. 9(b)] without too much noise, unlike the traditional algorithms (e.g., the Canny algorithm). A more satisfactory result can be obtained with the new ridge edge detection algorithm and the postprocessing functions. Even though the ridge edges [Fig. 9(c)] are mixed with some noise, the noise can be removed according to the length threshold of the lines [Fig. 9(d)], and then the roads are easily extracted by a Hough line detection algorithm.
Figure 10(a) shows a fuzzy aerial image with complex scenes, including roads, houses, bushes, farmlands, forests, rivers, and sea, and its profile histogram features are shown in Fig. 2. To enhance the roads, the image is operated on with the fractional differential operator, and its result is presented in Fig. 10(b): all the roads become clearer. In the Canny edge detection result, partial roads are detected, but more than half of the roads are bifilar lines, which creates difficulty for postprocessing [Fig. 10(c)]. When using the new ridge edge detection algorithm and the road identification procedure (postprocessing) described in Fig. 7, the result is obtained as illustrated in Fig. 10(d), where, by comparison to the Canny edge detection algorithm, less noise is obtained, it shows the main road lines or curves clearly, and the resultant binary image is relatively simple. Therefore, it provides better conditions for postprocessing.
The image in Fig. 11 is a complex urban fuzzy aerial image, including large quantities of artificial facilities; the roads to be detected are not clear. So, first the roads are enhanced by using the fractional differential operator, and then the ridge edges are detected to initially extract partial roads with the new algorithm. In the image, some road information is missing [Fig. 11(c)], probably because of the weakness of the roads. Even for these kinds of images, the new ridge edge detection algorithm can detect most of the road information [Fig. 11(d)]. In Fig. 11(e), the small objects or noise are filtered away from the result of the new edge detection, and the result in Fig. 11(f) is the final road identification result where most roads are distinctly shown after road identifying is complete.
The image in Fig. 12(a) is an aerial image from Ref. 8. The network detected and identified by the algorithm in Ref. 8 is presented in Fig. 12(b), and the result using the new algorithm is illustrated in Fig. 12(c). By comparing the new method to the algorithm in Ref. 8, although the new algorithm only gives a single line (curve) for each road (not for road boundaries), it can produce better road connections in a general structure.
The difficulty for this new algorithm is shown in Fig. 13. Figure 13(a) is a very blurred and complicated aerial road network image of the same resolution of as that in the above images, where there are many buildings and streets among roads. A low threshold for the new algorithm can result in many noise edges, which make road extraction hard. A high threshold can produce less noise, but some road information is lost, which also makes road detection and connection hard.
This paper suggests a method for resolving the problem of the difficulty in extracting thin and vague road lines or curves from aerial images. Compared to some traditional algorithms, the main features of the new operator are: (1) Enhance images through a fractional differential template to highlight road details,; compared with the conventional Laplacian and high-pass filter operators, the new operator can enhance road information without adding too much noise; (2) Then get road lines or curves by using the new ridge edge detection algorithm based on ridge characteristics proposed in this study. The algorithm is different from general edge detection algorithms, in that it is designed not based on point detection, but based on lines of 3 to 4 points [see Eqs. (1)–(4)], and the detected edges are ridge edges. When comparing this method to the conventional edge detection algorithms (such as Canny, LOG, etc.), the results show that the proposed method can be applied for roughly detecting thin and vague straight and curved roads in high-resolution aerial images with satisfactory results. After edge detection, the road identifying procedure (postprocessing) includes a number of functions, such as Hough line detection, thinning, endpoint detection, endpoints connecting, removing small object noises, nonlinear object removal, etc., which makes the final road identification result good. If the width of roads is wide (say, more than 6 pixels on average), the original image is shrunk, and the detection result will be used to re-map the roads based on both detection results and the original image information.
Currently, a three-dimensional (3-D) laser technique has been used to recognize roads by combining image techniques.19,20 Therefore, further research will make full use of 3-D texture and color information to improve the fractional differential operator and the ridge edge detection algorithms for images. The postprocedures are very important for road mapping or identifying. A number of functions or algorithms require improvement for different types of aerial road images.
This research is financially supported by the National Natural Science Fund in China (grant no. 61170147) in China.
Xiuzhi Yang is an associate professor of information engineering. She received her MS degree in electronic and communication engineering from Fuzhou University in China. She is currently working at the College of Physics and Information Engineering at Fuzhou University, China. She has published over 40 papers in journals. Her research interests include image processing and digital television technology.
Weixing Wang is a professor of information engineering. He obtained his PhD degree in 1997 at Royal Institute of Technology in Sweden. Since 2001, he has been a PhD supervisor at Royal Institute of Technology. Now he is a visiting professor at Fuzhou University, and his interests involve information processing, pattern recognition, and artificial intelligence.