The third-party interference, such as construction activities and man-made sabotage, has become the leading cause of pipeline accidents in the recent years. This work is devoted to a real-time surveillance system for safety monitoring and early warning of buried municipal pipelines subject to the most common abrupt intrusions based on distributed fiber optic sensors using the phase-sensitive optical time-domain reflectometry(φ-OTDR). A two-layer classifier based on convolutional neural network (CNN) is developed: one layer is used to discriminate the third-party threats from pedestrian and traffic noises; the other layer is to determine the specific type of third-party interference. To reduce false alarm times, the time-space matrixes are built to correct the possible errors. Field tests on an optical fiber cable buried in roads and residential areas are carried out to validate the two-stage surveillance system. It shows the first layer can effectively solve the problem of false alarm, while the second can accurately recongize the specific type of third-party interference.