Pipeline right-of-way (ROW) monitoring and safety pre-warning is an important way to guarantee a safe operation of oil/gas transportation. Any construction equipment or heavy vehicle intrusion is a potential safety hazard to the pipeline infrastructure. Therefore, we propose a novel technique that can detect and classify an intrusion on oil/gas pipeline ROW. The detection part has been done based on our previous work, where we built a robust feature set using a pyramid histogram of oriented gradients in the Fourier domain with corresponding weights. Then a support vector machine (SVM) with radial basis kernel is used to distinguish threat objects from background. For the classification part, the object can be represented by an integrated color, shape and texture (ICST) feature set, which is a combination of three different feature extraction techniques viz. the color histogram of HSV (hue, saturation, value), histogram of oriented gradient (HOG), and local binary pattern (LBP). Then two decision making models based on K-nearest neighbor (KNN) and SVM classifier are utilized for automatic object identification. Using real-world dataset, it is observed that the proposed method provides promising results in identifying the objects that are present on the oil/gas pipeline ROW.