2.1

Intelligent monitoring system

Dutta et al.¹⁰ studied the image processing technology in thermal state monitoring of power equipment. After converting RGB color model to HSI color model, they processed it and extracted different gradient edge descriptors. Edge operators such as Prewitt, Roberts and Sobel detected and recognized infrared thermal images. Finally, the Otsu image segmentation method based on clustering is adopted, which can be used to detect the thermal state of power equipment. Huda et al.¹¹ applied infrared thermography to thermal anomaly detection of electrical equipment. The imaging characteristics of different thermal anomalies of power equipment are analyzed and the corresponding discriminant methods are given. The effectiveness of this method is verified by experiments. By summarizing the development trend of power equipment fault diagnosis and related fault diagnosis techniques, Heano et al.¹² thoroughly studied the detail enhancement method of infrared images of power equipment, extracted the shape features of target equipment, and verified the feasibility and adaptability of the designed method in power equipment fault diagnosis.

2.2

Contextual mechanisms

The purpose of the context attention mechanism is not only to detect the target and the person in the image but also to infer the relationship between the person and the target. Kato et al.¹³ proposed visual composition learning for character interaction detection. In this model, the relationship between objects is decomposed into specific features of objects and verbs, and new interaction samples are formed in feature space by splicing the decomposed features. New interactive samples can be composed in feature space by splicing decomposed features. Xiong et al.¹⁴ proposed a new graph neural network using multidimensional edge features, which uses multidimensional edge feature information to construct the edge matrix in the graph. The implicit relationship between objects is obtained by updating the nodes and edge features. Cheng et al.¹⁵ proposed an end-to-end human interaction detection model. The model uses encoders to generate global memory features, explicitly models the relationships between different target features, and uses multi-layer perception to predict character interactions.

3.1

Human backbone feature extraction model

The paper uses the VGG-16 backbone to extract depth features from the input images. First, the color image is input to the feature extraction backbone network to generate a feature map F(H, W, C) containing high-level semantic information. All convolution kernels are set to size 3×3 to capture more information. In this paper, a small network is slid on the output convolutional feature map of the last shared convolutional layer. The small network takes as input a n×n space window of the input convolutional feature mapping. At each sliding window position, we predict multiple region proposals simultaneously. The maximum number of possible proposals for each position is denoted as k. Therefore, the coordinates of k candidate boxes are generated in the reg layer. Also the confidence level of each candidate box proposal object in the cls layer.

3.2

Contextual mechanisms

In this paper, we propose a hierarchical contextual feature extraction network for improving the performance of the model. First, in this paper, we apply the proposal generated by RPN to the feature map X to obtain the local RoI features x_local, which are as follows¹⁶:

where f_ROI–Align (∙) represents the ROI-Align operation function. h′ and w′ respectively represent the height and width of the region of interest.

To better utilize the contextual features, this paper uses the feature map F to obtain global-level contextual RoI features x_global, which are as follows.

where H and W respectively are the height and width of the input image.

Contextual features are obtained by fusing local contextual features with global contextual attention features through a convolutional layer, as shown in the following equation:

where f_couv represents the convolution operation, [:] represents the concatenation, and x_context represents the extraction of the contextual information.

To obtain a greater degree of the dynamic relationship between contextual features and targets, and to enhance the role of contextual information for target detection, a more relevant and reliable contextual feature for object detection can be obtained by equation 3.

where Ω[x_contest, x] represents the correlation between contextual features and target features. Ω[x_contest, x_contest] represents the correlation between contextual features. f_inr represents the dynamic encoding function. x_c represents the more relevant and reliable contextual information extracted by the network.

In this paper, a feature fusion strategy is proposed to achieve the fusion of the above information and the features extracted from the backbone network. The principle is shown in Figure 3.

Figure 3.

The hierarchical contextual feature extraction module.

where x_F, represents the fused features. |∙| represents the feature fusion operation.

3.3

Target detection model

This paper uses the two-layer convolutional layer to generate classification confidence for contextual features, which is fused with the classification confidence generated by ROI as follows:

The fused classification confidence is passed through the soft-max layer to generate new classification confidence. In this paper, we use G to denote the coordinates of the candidate frames. Assuming that the coordinates of the candidate frames are independent, the principle of obtaining the probability distribution can be expressed as:

where G_a represents the coordinates of the true bounding box. σ represents the standard deviation, which is used to measure the estimated uncertainty. When σ → 0, it means that the candidate box proposed by the identification framework in this paper is worth to be considered.

Not all the candidate frames obtained by the above process are valid. Therefore further screening of them is needed. In this paper, the non-maximum suppression (NMS) method is applied to the proposed region based on the cls scores of the proposed candidate frames. We set the IOU threshold of NMS to 0.5 and use the top n-ranked proposed regions for detection to finally achieve target detection.

4.1

Experimental setup

This experiment is run on the server based on Windows 10 system. Run Python3.7. The deep learning platform software used to compile the tests was configured as PyTorch and CUDA V1.0. Hardware configuration: GPU is RTX 1080 Ti. In this paper, 1000 images including insulators, shock hammers and spacers were collected to construct a dataset. These images are scaled to 227×227. This paper will choose four methods to compare with our model. These four methods are SSD, Faster R-CNN and YOLO-v4¹⁷. The SGD optimizer is used to train all models separately. Models are trained in a batch size of 10. The initial learning rate of the model is set to 0.01.

4.2

Qualitative experimental analysis

To verify that the proposed Detection of Abnormal Dangerous Behavior in Electrical Power Grids Based on the Fusion of Contextual Features model is a solution to the problem of low accuracy of target recognition in complex scenes. To evaluate the performance of the proposed algorithm, using more than 5000 images from the three datasets of Safety Helmet (Hardhat) Wearing Detect dataset, Hard Hat Workers dataset, Safety Helmet Detection dataset, the improved model can well identify the location and classification of people, heads, and helmets.

Figure 4 shows the results of the proposed method in this paper compared with the comparison method under the influence of different backgrounds and different occlusions. From the results, it can be seen that the algorithm proposed in this paper is able to identify the target accurately. Compared with other methods, the corresponding confidence level of the proposed method in this paper is optimal in all experiments. The experimental results show that the method proposed in this paper can accurately identify people, heads and helmets in the grid workspace, and can also identify the targets well in different backgrounds.

Figure 4.

Simulation experiment of target recognition in power network operation room.

4.3

Quantitative experimental analysis

When evaluating the algorithm model through the qualitative experimental results, due to the different observation habits or observation angles of different human individuals, the evaluation criteria of quantitative experiments are too subjective. Therefore, we present the qualitative comparison experimental data of the proposed algorithm and the comparison algorithm in this section. This article will qualitatively analyze the object recognition models of SSD, Faster R-CNN, YOLO-v4 and Ours from the objective data level. This article introduces AP and mAP object recognition evaluation standards to analyze the object recognition results of each object recognition model from objective data. The objective data corresponding to the different scene simulation experiments is shown in Table 1.

Table 1.

The objective data corresponding to the different scene simulation experiments.

The AP and mAP values for the different scene simulation experiments are shown in Table 1. From the data point of view, the algorithm in this paper is the best, and the AP is the highest in the Insulators, Shock Hammers and spacers experiments. The average recognition accuracy is 94.85% compared with other models, and the average recognition accuracy is improved by 1.02%.