3.1

Anomaly detection based on machine learning

In related paper, deep learning and ensemble learning are used for anomaly detection. Considering that the datasets are large and the operating environment does not have enough computing resources to train the deep learning network, the paper focus on ensemble learning. The tree model has a much better classification effect by using node decision method. According to the inconsistent scope of each feature in different anomalies, it can be subdivided into multiple decision nodes to achieve good prediction results. Therefore, XGBOOST and Random Forest are mainly used for training, supplemented by LightGBM method. The following takes the random forest model as an example to illustrate in detail.

As shown in Figure 2, the original data contains address, time, protocol and other data, so it is necessary to conduct coding processing on the original data to facilitate subsequent model training, and obtain the features after coding. In addition, training models using all features probably cannot obtain excellent results, so we need to select features here, and try to incorporate features containing a large amount of information but less noise into the model. Data normalization processing is required before the beginning of training to avoid the influence of extreme values on the training model. In general, we need to divide the original data into two parts: training set and test set. The training set is used for model training, while the test set is used for verifying the model. Here is the main choice of random forest model, used in the process of training without back into the way of sampling to select feature subset from the feature space, and then build a decision tree node by splitting algorithm, is adopted for the training set data is back on the way of sampling is selected from a training set of data the subset of data, usually two-thirds of them were randomly selected randomly selected from many times, to get more training subsets, take advantage of these subsets to build decision tree, which constitute a random forest, in the end for each training sample label a stochastic model of the forest. The prediction part is to pass the test set data through the trained random forest to obtain the probability that the test data belongs to a sample tag, and the tag value corresponding to its probability value is taken as the final tag result through the voting mechanism.

Figure 2.

Anomaly detection flow chart based on random forest.

After the random forest, XGBOOST and LightGBM models are well trained, this paper proposes a framework combined with logistic regression models. Logistic regression fits the linear relationship between several models and fully extracts the characteristics of each model. However, if the accuracy of a model is poor, the overall accuracy will be inferior to that of the sub-model after logistic regression. In order to avoid overfitting, L2 regularization can be considered.

3.2

Multiple classification to binary classification

In this paper, a model is subdivided into several sub-models by transforming multi-objective classification into binary classification. The specific transformation algorithm is shown in Algorithm 3.1:

This algorithm converts multiple categories into binary categories. In consideration of efficiency, a model is randomly selected for training in line 6. If the training results are better than the existing model, the existing model is replaced.

3.3

Multiple classification to binary classification

Different from the multi-classification to binary classification algorithm proposed in Section 3.2, the binary classification results need to be mapped back to the multiclassification. The mapping method we adopted was a voting method, and each model was used in turn to predict the test set to get the probability of abnormal traffic. The highest probability value was taken and marked as the corresponding label. The core algorithm is shown as follows:

Integrating with the multi-classification to dichotomy algorithm proposed in Section 3.2, the binary classification results need to be remapped to multi-classification. The mapping method we adopted is greedy voting method: each model predicts the test set in turn, obtains the probability of abnormal flow, and takes the maximum probability value as the corresponding label. Greed is embodied in this on the order of the multiple binary classification model, in this paper, the two classification models according to the accuracy high and low rank, high accuracy of the model to forecast first, predict corresponding labels on two classification model, such as a sample, “A” model to predict is abnormal probability is 80%, “B” probability model prediction was 90%, but because of “A” high accuracy, the sample will be labeled “A”. The core algorithm is as follows

Step 1: Use each sub-model to predict the test set and get the probability of being an exception, that is: p(i) = mod el_i (testset_j);

Step 2: Select the model with the probability greater than threshold, that is, which category should sample j be divided into, i.e.,

Step 3: Traverse all samples to get the final results.

4.1

Data processing

The main experimental environment of this paper is shown in Table 1.

Table 1.

The experimental environment of the paper.

This study is based on the traffic collected and processed by NCTU¹¹, which has been converted into text data, and on this basis, network anomaly detection is carried out. Firstly, the number and dimension of the data are concerned. The training set contains more than 9 million samples with 22 features, and there are five labels: ‘Normal’, ‘probing-nmap’, ‘ddos-smurf’, ‘probing-ip sweep’, ‘probing-port sweep’. The number of Normal samples is 892W at most, accounting for more than 90%. Secondly, we analyzed the correlation coefficient between the features of the training set, and the analysis results are shown in Figure 3. It can be seen from the figure that the maximum correlation is 0.8: CNT DST and CNT DST CONN; CNT SRC, CNT SRC SLOW, CNT SRC CONN, but considering the characteristics of the data set are relatively few, delete and merge are not carried out.

Figure 3.

Heat maps of features.

4.2

Model evaluation

This paper uses two methods to evaluate the effect of the proposed model. One is commonly used statistical indicators: macro precision, macro F1 score and macro recall, and the other is calculated by cost matrix.

The calculation equation of macro precision is as follows:

The calculation equation of macro recall is as follows:

The calculation equation of macro f1 is as follows:

The calculation method of the cost matrix is as follows:

Among them, α = 0.3, β = 2 and the cost matrix is shown in Table 2.

Table 2.

Cost matrix of the model.

Experimental results of common models are shown in Figure 4, and those calculated by cost matrix are shown in Figure 5. It can be observed from the two figures that XGBOOST and Random Forest have good effects. Therefore, the model in this paper adopts logical regression combination of XGBOOST and Random Forest, and then adopts multi-classification greedy transformation algorithm. The model in this paper is further improved in XGBOOST and Random Forest models, with macro precision, Macro recall and Macro F1 score reaching 0.99 and 0.93 by cost matrix calculation. The model in this paper has good effect.

Figure 4.

Statistical results of different models.

Figure 5.

The result of cost matrix evaluation.