We present a new method, based on the Fisher Vector (FV), for detecting buried explosive objects using ground- penetrating radar (GPR) data. First, low-level dense SIFT features are extracted from a grid covering a region of interest (ROIs). ROIs are identified as regions with high-energy along the (down-track, depth) dimensions of the 3-D GPR cube, or with high-energy along the (cross-track, depth) dimensions. Next, we model the training data (in the SIFT feature space) by a mixture of Gaussian components. Then, we construct FV descriptors based on the Fisher Kernel. The Fisher Kernel characterizes low-level features from an ROI by their deviation from a generative model. The deviation is the gradient of the ROI log-likelihood with respect to the generative model parameters. The vectorial representation of all the deviations is called the Fisher Vector. FV is a generalization of the standard Bag of Words (BoW) method, which provides a framework to map a set of local descriptors to a global feature vector. It is more efficient to compute than the BoW since it relies on a significantly smaller codebook. In addition, mapping a GPR signature into one global feature vector using this technique makes it more efficient to classify using simple and fast linear classifiers such as Support Vector Machines. The proposed approach is applied to detect buried explosive objects using GPR data. The selected data were accumulated across multiple dates and multiple test sites by a vehicle mounted mine detector (VMMD) using GPR sensor. This data consist of a diverse set of conventional landmines and other buried explosive objects consisting of varying shapes, metal content, and burial depths. The performance of the proposed approach is analyzed using receiver operating characteristics (ROC) and is compared to other state-of-the-art feature representation methods.