Glass insulators are widely used in power systems due to their high resistance, stable electrical strength, slow aging, etc. But the self-shattering, cracking, and serious contamination of insulators often appear under complex and harsh operating conditions, which will lead to serious accidents without timely detection and replacement. Thus, how to timely detect the insulator defects is an important task for the safe operation of transmission lines.
Insulator defects have been highly concerning in transmission lines and the related detection technologies have been developed consequently. Traditionally, insulator defects were inspected by the human patrol every one or more months, which is inefficient, subjective, and dangerous for the patroller. With the development of economy and technology, the detection technologies are becoming more and more intelligent. Now, machine visual and image processing have been developed in recent years with the computer vision techniques. More video monitoring devices (VMDs) have been installed on towers in China1 or unmanned aerial vehicles (UAVs) with cameras started patrolling transmission lines,23.–4 which are used to capture insulator images and monitor the running status of transmission lines, then the obtained images are often analyzed automatically using a customized image processing algorithm. For example, contamination detection, hydrophobic detection, and crack detection have been gradually applied in practice. Machine visual and image processing have the advantages of high speed, low cost, and high performance, which are regarded as the most attractive techniques for defect detection. This paper mainly focuses on self-shattered glass insulator, which refers to the phenomenon that the insulator self-shatters and falls off under the harsh environment. Once the glass insulator self-shatters, it will have great harm to the entire transmission lines. How to timely detect and replace the self-shattered insulator is an urgent and necessary task.
The detection technology of glass insulator defects mainly consists of two parts, which are the insulator identification and the defects’ localization. In terms of the insulator identification, many researchers have studied how to extract insulator by image processing, which mainly relies on the inner characteristics of the insulator such as color, shape, and texture. For example, the clear insulator strings are obtained by threshold segmentation and morphological corrosion of I-space images in hue, saturation, intensity (HSI) space.5 An improved algorithm is proposed to extract the S-space image in the HSI space and the maximum interclass variance threshold segmentation is performed for the extracted image.6 The maximum entropy threshold segmentation method is used to extract the insulators and the Hough linear detection and genetic algorithm are proposed to remove the interference.7 However, there are few studies that identify the defects of insulators. For example, the edge shape of the insulator is used to judge whether the insulator exists the self-shattered defect, but a complete edge is needed in the chain-code analysis.8 The self-shattered defect is judged by calculating the number of ellipses and the distance between the ellipse centers.9 But the aforementioned edge shape is obtained based on the insulator features, which are very difficult to obtain a complete edge under the complex background.8,9 The texture features of the insulator are extracted and the defects are judged by the analysis of the texture features, but the algorithm does not propose an effective segmentation scheme.10
Based on the analysis aforementioned, these algorithms can be only used for the ideal clear images. But it is difficult to capture the clear images by the VMD or UAV. Furthermore, these images are usually characterized by vague, color difference, deformation, and partial occlusion due to the dark environment, the varied topology, the shot angle, the camera resolution, and the contamination level. It is difficult to accurately recognize the defects of insulator strings, by relying only on certain insulator features, such as color, shape, and spatial sequence. In practice, we find that the insulator texture feature is relatively stable for most captured insulator images. Thus, an automatic identification and location technology of self-shattered glass insulator is presented, by which the self-shattered insulators can be accurately identified and located by extracting its local binary pattern (LBP) feature.1112.–13
Framework of Automatic Identification and Localization Technology
An automatic identification and localization technology of self-shattered glass insulators consists of three parts: cameras, which can be installed on towers or UAV, the 4G/OPGW communication network, and the monitoring center, where the identification and localization algorithm is embedded into the expert software, as shown in Fig. 1. The images of the glass insulator string are first captured, decompressed, and sent to the monitoring center by 4G/OPGW. The self-shattered insulators can be identified and located from the images in the monitoring center based on the identification and localization algorithm of self-shattered glass insulators built.
Algorithm for Identification and Localization Technology of Self-Shattered Glass Insulators
In most cases, a single insulator string or double insulator strings can be considered to be distributed as one or double straight lines. The contour of a single insulator is similar to an elliptical shape, and the number of insulators is mainly related to the running voltage of transmission lines. Thus, an insulator string in a given image [see Fig. 2(a)] can be considered as a row of approximate ellipses, whose length is the distance between the starting point and the end point along the row. Then, a linear-fitting straight line can be obtained to estimate the direction and length by fitting the center coordinate of each insulator by the least squares method.
Figure 2(b) shows the proposed mathematical model of the insulator string. We set the lower left coordinate of the image as the origin and the center of each insulator contour as (). When , it stands for the first insulator and is the last insulator.
The mathematical model of the insulator string can be obtained by a linear fitting step, which can be summarized as follows:
1. The marked center points of all insulator contours are used as the sampling points of the linear fitting;
2. The slope and intercept of the straight line can be calculated by the linear least squares fitting method.14 As shown in Fig. 2(b), the fitting equation of the straight line can be described as follows:4.1). The represents the slope of the straight line. The represents the intercept of a straight line. The principle of fitting is to minimize the error between the actual and the fitted with by Eq. (2)
Then, the partial derivations of and in Eq. (2) are calculated and simplified. The optimal and can be obtained by Eq. (3) and the straight-line equation by Eq. (1)
Then, the angle of the line can be obtained as
The length of insulator can be obtained as
The distance of two adjacent insulators is defined as and the average distance between two adjacent insulators can be obtained as
Identification and Localization of Self-Shattered Glass Insulators
Although the method of image processing to identify insulator defects has already been studied by some scholars, there are two difficulties related to this project.
1. Transmission lines often run in a complex natural environment such as mountains, trees, grassland, and complex lighting variance, and the segmentation of insulator images becomes a key problem, which will influence the accuracy of defect identification.
2. With the shooting angle of cameras changes, the insulator shape in images varies greatly, which has no stable shape features.
The available defect recognition algorithms often rely on color or shape of insulators, which is ineffective and unstable due to the limitations of insulator segmentation and unstable features. An automatic identification and location algorithm of self-shattered glass insulator is presented, which consists of identification of glass insulator string and detection of self-shattered insulators.
Identification of Glass Insulator String
A captured insulator image commonly consists of the insulator string and other background objects, such as tree, tower, conductor, hill, grass, and mountain. Especially, when the cameras work in the high voltage circumstance, the image quality is greatly influenced due to the electromagnetic interference. In addition to the complex background aforementioned, there are abundant noises in the insulator images. To accurately extract the insulator, the insulator region should be first identified and extracted. The HSI color space, which consists of image color space conversion and image segmentation, is adopted in this paper.
(1) Conversion of the color space
Because the glass insulators is light-green and translucent, the captured insulator images are easily impacted by similar background color, illumination, and color temperature, whose features are unstable and colors are different. Compared with RGB, HSI color space is closer to a human eye’s visual properties. The conversion from RGB to HSI is fulfilled, by which the color features are fully used and the interference of illumination is reduced.
(2) Image segmentation
Image segmentation is the premise of image analysis,15,16 which is key to extract the image features and identify the defects. The maximum interclass variance method (OTSU) is used to segment the insulators; we find that it is difficult for the segmentation by OTSU in RGB space to identify the insulators under the complex background, as shown in Fig. 3(a). Wrong segmentations exist in the H and S component images, as shown in Figs. 3(b) and 3(c), but the interference disappearing in Fig. 3(c) does not disappear in Fig. 3(b). Then, the idea of “and” operation is proposed to get the intersection of the segmentation of H and S components, by which more interference of background and noise can be reduced.
The “and” operation between image and image is defined as
As seen in Fig. 3(d), the most background objects can be removed by “and” operation, but there are still some weak background interferences and partial connections between insulators. The morphological open operation is used to solve the problem, by which not only these small noises are eliminated, but also the target connection or overlapping problem can be solved because the contours, the area, and the inclination of the foreground insulators and the noninsulators are obviously different. To remove the noninsulators, the histogram method is used to calculate the contour inclinations and the connected region in images. A clear insulator string is obtained by the method above, as shown in Fig. 3(e). The regional contour extraction of insulator string is shown in the Fig. 3(f). The segmented contour extraction can well reflect the direction and area information of the running insulators, which is important for the self-shattered detection.
(3) The fitting of insulator strings
According to the features of insulator string in the Fig. 3(e), the insulator string is almost linearly arranged. After the image segmentation and morphological open operation, a clear insulator string can be obtained. Then, the center of every insulator contour can be obtained as shown in Table 1. According to the mathematical model of insulator string established in Sec. 3, the least square method (LSM) is used to fit the central points of insulators to a straight line. Two insulator strings in Fig. 3 can be fitted and the fitting lines are shown in Fig. 4, whose equations are as follows:
Dimensional information of the insulator string.
|No.||Area (pixel)||Central point (pixel)||Distance (pixel)||Mean distance (pixel)|
Next, the distance of adjacent insulators can be calculated by both center points of adjacent insulator contours, as shown in Fig. 2. But the distances of adjacent insulators in the insulator strings may vary in a certain range, to ensure the accuracy of the calculation, the average distance of adjacent insulators is calculated with the maximum and the minimum distances eliminated. The calculated average distance is also close to the actual distance of adjacent insulators, which is beneficial to calculate the length of blocks and the size of template. Table 1 illustrates the dimensional information of the insulator string such as the center point of every insulator, the distance of adjacent insulators, and the calculated average distance of adjacent insulators for the whole string.
Detection of Self-Shattered Insulators
By matching the LBP histogram of blocks with that of sliding template, the self-shattered insulator can be recognized. The procedure consists of LBP histogram extraction based on blocks, determination of sliding blocks, and location of self-shattered insulators.
Local binary pattern histogram extraction based on blocks
Some image features such as color, shape, texture, and spatial sequence are commonly used to extract the insulators from images.17,18 It is found that the insulator texture feature is relatively stable, in a sense, the gray values of texture are also displayed regularly, which exists a certain gray relationship between two pixels in the image space. When the insulators self-shatter, the surface texture will be obviously changed. There are many well-established approaches to extract the texture feature, such as gray-level co-occurrence matrix (GLCM),19 gray differential statistics, and wavelets analysis.20 These methods have a certain effect in the texture analysis, while they are rarely used in actual applications. In recent years, the LBP method was proposed by Ojala et al.,21 which has a low-computational complexity, a multiscale feature, and a rotational invariable feature, and is widely used in the field of texture classification and face recognition. Then, the LBP method is proposed to identify the insulator string, which consists of three steps as follows: calculating the binary relationship of each pixel and its neighboring pixels in the image, forming the local binary model by weighted rules for the relationship, and getting binary image pattern by the multiregion histogram sequence.
(1) LBP feature
Set a window to gain the gray values of nine pixels, which are separately , , assume the window center as the threshold value, and binarize the rest pixels in the window. Define the texture as
The LBP value of the window is obtained by the weighted summation of different pixels, which is defined as
(2) LBP histogram extraction
Since the extracted insulator region is relatively clear, the single LBP block method is used to extract features of identical nonoverlapped blocks.22 The LBP texture feature can be obtained by the LBP operator aforementioned, by which the corresponding LBP map can be obtained as shown in Fig. 5(b). While the insulator feature cannot be directly reflected from the map, LBP histogram is adopted as the following matching data of the template and the sliding blocks, which is shown in Fig. 5(c). The histogram calculating equation is defined as
Determination of sliding blocks
For transmission lines, the function of the insulator string is to connect the tower and conductor. For the tension tower, the insulator string is nearly vertical. But for the suspension tower, the direction of the insulator string is at any angle. If the whole insulator can be analyzed timely, the sliding direction and the length of blocks are very important.
(1) Direction of sliding blocks
Because the installation positions of the cameras are different, the directions of the insulator strings in images are random. But we can use the fitting lines described in Sec. 4.1 and Eq. (4) to determine the direction of the insulator string, which can be regarded as the direction of the sliding blocks.
(2) Adaptive sliding blocks
The adaptive sliding blocks are necessary with the calculating time and the accuracy considered. By a large number of experiments, the length of normal blocks can be set 5d. But to ensure the accuracy of the calculation, the average distance of adjacent insulators is calculated with the maximum and the minimum distances eliminated. In addition, the number of blocks is mainly changed according to the length of the insulator string and the average distance of the insulators, which is defined as
Then, the length of normal blocks is and the length of the last smaller block is defined as
But for double insulator strings, the obtained region of insulator strings needs to be divided into two subsidiary parallelograms, which contain each row of insulators, which is two times of the number of adaptive blocks of single insulator string. The information of adaptive blocks for six test images is shown in Fig. 6.
Localization of self-shattered insulators
(1) Matching of the template and the sliding blocks
The length of the template is the same to that of the normal sliding blocks. The sliding template moves along the insulator string by step , then the LBP histogram is counted between the sliding template and the blocks.23,24 Figure 7 shows the sliding process of the template along the insulator string.
(2) Location of self-shattered insulators
Euclidean distance is used to calculate the histogram, which is defined in Eq. (13). By matching the LBP histogram of blocks with the LBP histogram of sliding template, the self-shattered insulator can be recognized
First, the average distance is obtained by Eq. (14). If , the matched block is not self-shattered block
Second, the difference between and is calculated by Eq. (15) which is normalized as by Eq. (16)
Finally, based on the normalized , every block can be recognized if it is a self-shattered block by threshold according to Eq. (17), then the relative displacement between the self-shattered block and start point is obtained, hence the position of self-shattered insulators is located
Application of the Identification and Localization Algorithm of Self-Shattered Glass Insulators
Figure 8 shows the whole process of the image processing algorithms aforementioned. The self-shattered insulator image is captured [see Fig. 8(a)], which has a complex background and great noise. The conversion of the color space and image segmentation is used to segment images [see Fig. 8(b)]. The idea of “and” operation is used to get the intersection of the segmentation of and components to obtain relative pure insulator strings as much as possible [see Fig. 8(c)]. Then, the morphological processing and contour detection are used to obtain the connected domains of single insulators [see Fig. 8(d)] and extracts the area and center. Based on the analysis of the previously mentioned processes, we establish the mathematical model of insulator string. Next, the LBP histograms based on adaptive block are extracted [see Figs. 8(e) and 8(f)], then the template and direction of sliding window are established [see Fig. 8(g)]. Finally, the LBP histogram of the regions between the sliding window and the template is calculated and analyzed to recognize and locate the self-shattered defects [see Fig. 8(h)], the left picture is the histogram of the template and the right is the histogram of a block. The recognition result is marked with a red rectangle [see Fig. 8(i)].
Experimental Results and Analysis
The performance of the proposed LBP histograms is analyzed in this section, which will be compared with the related well-established methods. All images used in this experiment are obtained from different insulator strings and different view angles under different conditions, which are captured by VMDs of transmission lines and whose size is . A dataset with 400 images is constructed to evaluate the performance, 200 images of which have one or more defects and 200 images no defect.
Comparison of Local Binary Pattern Histograms with Other Two Similar Methods
Two common texture feature extraction methods are selected to compare with the texture feature extraction based on the LBP histogram.
Method 1 is a feature extraction method based on the gray-level histogram;19 the color image is first converted to the gray image and count the gray image histogram, and then calculate the mean gray and calculate the dispersion, variance, skewness, kurtosis of gray histogram whose four feature vectors describe the image texture characteristics. Method 2 is a feature extraction method based on the GLCM.19 Because the calculation for the co-occurrence matrix is large, the gray scale is coarsely quantified from 256 levels into 16 levels, which can save computing time. Although the distortion exists after the image is quantified, it has little effect on the texture feature. Then, the GLCM of the four directions (0 deg, 45 deg, 90 deg, 135 deg) is calculated, generally, four commonly used characteristics are the contrast, entropy, energy, and correlation. Finally, the average value is calculated to describe the texture characteristic of image.
We extract the texture with a group of insulator images, which are the same in Fig. 6, and calculate the similarity distance between the feature vector of template insulator block and each block in the insulator image, but if the similarity distance is directly calculated, the difference of the value will affect the weights of feature vectors. Therefore, the feature vector is first normalized, and the normalized equation is defined as
The Euclidean distance is adopted to calculate the similarity distance. We assume that  represents the feature vector of the template block in every insulator image,  represents the feature vector of each block in the insulator image, the smaller similarity distance is, the higher the similarity between the template block and each block is. The range of normalized similarity distance is 0 to 1 and the similarity distance is defined as
Because the number of blocks in every image is different, the Euclidean distance between the template and the blocks can be calculated by these methods. As seen from Table 2, for most test images, the calculated distance based on LBP histogram is much smaller than that based on methods 1 and 2, which means that the proposed method has highest matching accuracy. Method 1 is the easiest and simplest to extract the texture feature, but its matching accuracy is the worst because it only relies on the gray histogram and lacks of the pixel space information. Method 2 is especially complex with co-occurrence matrix, but its matching accuracy is higher than the proposed method only in the No. 2 image. The proposed LBP histogram is very effective on the feature extraction of the insulators.
Comparison of the Euclidean distance with other two methods.
|The mean similarity distance/test images||1||2||3||4||5||6|
Identification Examples of Self-Shattered Insulators
By matching the LBP histogram of blocks with that of sliding template, the self-shattered insulator can be recognized effectively. When LBP histogram distance at some insulator is much bigger than others, this insulator maybe a self-shattered insulator. There are two examples used to illustrate the identification of self-shattered insulators. In addition, all the self-shattered insulators are labeled with a minimum red rectangle, by which it is easy to find and locate the self-shattered insulator.
Example 1. Identification of self-shattered insulator in single insulator string
Figure 9(a) is an image of single insulator string and Fig. 9(c) is the identification curve of self-shattered insulator. As seen from the normalized distance curves in Fig. 9(c), the normalized distance increases rapidly at number 11, which should be the self-shattered insulator. A red rectangle marks the location of self-shattered insulator, as shown in Fig. 9(a), whose central point coordinates are (160.381, 146.991) pixel.
Example 2. Identification of self-shattered insulator in double insulator strings
Figure 9(b) is an image of double insulator strings and Fig. 9(d) is the identification curves of self-shattered insulator. As seen from the normalized distance curves in Fig. 9(d), the normalized distance of the red curve, which represents the first insulator string changes smoothly. In contrast, the normalized distance of the blue curve, which represents the second insulator strings increases rapidly and peaks at number 20, which should be the self-shattered insulator. A red rectangle marks the location of self-shattered insulator, as shown in Fig. 9(b), whose central point coordinates are (167.5, 279.5) pixel.
Identification Performance of Self-Shattered Insulators
(1) Definition of , , and
To quantify the recognition effect of the algorithm, the plethora of metrics derived from confusion matrices (TP, FN, FP, and TN) is adopted. Then, accuracy (), precision (), and recall () are defined in pattern recognition and information retrieval communities as follows:25
(2) Identification performance
For transmission lines, the single insulator and the double insulator are designed and installed according to the running voltage in China. The following 400 test images are divided into four categories, which are the group of 100 normal single insulator images, the group of 100 defective single insulator images, the group of 100 normal double insulator images, and the group of 100 defective double insulator images, respectively.
Figure 10 shows the testing results of some typical images and Table 3 shows the recognition effect of the algorithm. As seen from Table 3, for single insulator images, , , and of the presented algorithm are 94.5%, 92.38%, and 96.78%, respectively. For double insulator images, , , and are 90.00%, 86.36%, and 93.23%, respectively. All running results show that the algorithm is effective and practical.
The recognition effect of the algorithm.
|Images||The number of test image||Ac (%)||Pr (%)||Rc (%)|
Comparison of Three Detection Methods
The proposed algorithm based on LBP histogram is characteristic of the texture feature. But the detection methods based on the edge or the shape are commonly used, by which the edge and the shape of normal region and abnormal region are fitted.
We adopt three different detection methods to identify and locate the self-shattered insulators. The experiments are carried on a laptop with an Intel(R) Core(TM) i5 CPU (2.67 GHz) and 4-GB memory. Three hundred images are selected to be analyzed. The mean running times from extracting insulator region to locating the defects are recorded. The number of successfully detecting the self-shattered insulator in 300 images is also recorded. The results are shown in Table 4.
Comparison of this method with other related methods.
|Method||Identification accuracy (%)||Mean running time (s)|
|Based on shape||71.45||2.4285|
|Based on edge shape||73.28||2.6071|
As seen from Table 4, the accuracy of the proposed method is up to 90.63%, which is much higher than that of other two methods. Because of the instability of the shape, the methods based on shape or edge shape are prone to change easily with the different shooting angles and the varied illumination. The proposed method based on texture is relatively stable, which is suitable for detecting the self-shattered glass insulator. But the mean running time of proposed method is bigger than other two methods. For transmission lines, the running time difference in the second level does not matter.
Field Running Tests
An automatic identification and localization technology, as shown in Fig. 1, is successfully developed, which was already applied to many 110 kV or more transmission lines belonging to the Guizhou power grid in China. The UAV with the high-definition camera installed was also applied to the Shaanxi power grid, as shown in Fig. 11.
Identification of the Self-Shattered Insulators by Expert Software
In this paper, the insulator recognition and localization algorithm is designed and embedded into the expert software installed in the monitoring center. The captured images are first compressed and then transferred to the monitoring center by 4G/OPGW. The images will be analyzed by the expert software, and then the identification results will be displayed in a timely fashion and sent to the staff by GSM SMS.
The expert software is developed combining VC++ and HALCON, whose core functions consist of the image reading, insulator recognition, and defect localization, etc. The expert software also has other functions such as file operation, display, algorithm and analysis, defect information display, as shown in Fig. 12. Figure 12(a) shows the recognition of the insulator string and Fig. 12(b) shows the detection and localization of the self-shattered insulator.
An automatic identification and localization technology of self-shattered glass insulators is proposed, by which the self-shattered insulator can be identified and replaced in a timely manner. The proposed algorithm based on LBP histogram is characteristic of the texture feature, whose accuracy is much higher than that of two methods based on shape or edge shape. The technology has been applied to many 110 kV or more transmission lines, which is effective and reliable.
However, we only aim at the kinds of images captured from natural illumination and complex background. In the case of images captured in fog, rain, and other complex climate, the method yields a low recognition rate. Meanwhile, compared with the actual world, the set coordinates still show certain differences. These situations deserve further research and improvement.
The authors thank the referees for many valuable comments given to help improve this paper. This paper was supported by the Project of Key Science and Technology Innovation Team of Shaanxi with the Grant No. 2014XT-07 and the Shaanxi Industrial Science and Technology tackling key problems fund with the Grant No. 2016GY-052.
Xinbo Huang received his BS and MS degrees in automation from Qingdao Technological University, Qingdao, China, in 1998 and 2001, respectively. He received his PhD in automation from Xidian University, Xi’an, China, in 2005. Currently, he is a professor at the School of Electronics Information, Xi’an Polytechnic University, and also a PhD supervisor at the School of Electro-Mechanical Engineering, Xidian University. His current research interests include online monitoring technology, image recognition technology, and the wireless network sensor.
Huiying Zhang received her BS degree in electronic information engineering from Xi’an Polytechnic University, Xi’an, China, in 2015. Currently, she is a graduate student, majoring in signal and information processing. Her research interest includes fault detection based on image processing.
Ye Zhang received her BS and MS degrees from Xi’an Polytechnic University, Xi’an, China, in 2011 and 2014, respectively. Currently, she is a PhD student in mechatronic engineering from Xidian University. Her main research interests have been focused on intelligent power and online monitoring technology.