Object tracking is a core subject in computer vision and has significant meaning in both theory and practice. We propose a tracking method in which a robust discriminative classifier is built based on both object and context information. In this method, we consider multiple frames of local invariant features on and around the object and construct the object template and context template. To overcome the limitation of the invariant representations, we also design a nonparametric learning algorithm using transitive matching perspective transformation. This learning algorithm can keep adding object appearance and can avoid improper updating when occlusions appear. We also analyze the asymptotic stability of our method and prove its drift-free capability in long-term tracking. Extensive experiments using challenging publicly available video sequences that cover most of the critical conditions in tracking demonstrate the enhanced strength and robustness of our method.

Existing methods to detect vehicle tracks in coherent change detection images, a product of combining two synthetic aperture radar images taken at different times of the same scene, rely on simple and fast models to label track pixels. These models, however, are unable to capture natural track features, such as continuity and parallelism. More powerful but computationally expensive models can be used in offline settings. We present an approach that uses dilated convolutional networks consisting of a series of 3×3 convolutions to segment vehicle tracks. The design of our networks considers the fact that remote sensing applications tend to operate in low power and have limited training data. As a result, we aim for small and efficient networks that can be trained end-to-end to learn natural track features entirely from limited training data. We demonstrate that our six-layer network, trained on just 90 images, is computationally efficient and improves the F-score on a standard dataset to 0.992, up from 0.959 obtained by the current state-of-the-art method.

Classification of hyperspectral remote sensing imagery is one of the most popular topics because of its intrinsic potential to gather spectral signatures of materials and provides distinct abilities to object detection and recognition. In the last decade, an enormous number of methods were suggested to classify hyperspectral remote sensing data using spectral features, though some are not using all information and lead to poor classification accuracy; on the other hand, the exploration of deep features is recently considered a lot and has turned into a research hot spot in the geoscience and remote sensing research community to enhance classification accuracy. A deep learning architecture is proposed to classify hyperspectral remote sensing imagery by joint utilization of spectral–spatial information. A stacked sparse autoencoder provides unsupervised feature learning to extract high-level feature representations of joint spectral–spatial information; then, a soft classifier is employed to train high-level features and to fine-tune the deep learning architecture. Comparative experiments are performed on two widely used hyperspectral remote sensing data (Salinas and PaviaU) and a coarse resolution hyperspectral data in the long-wave infrared range. The obtained results indicate the superiority of the proposed spectral–spatial deep learning architecture against the conventional classification methods.

Hyperspectral anomaly detection (AD) is an important technique of unsupervised target detection and has significance in real situations. Due to the high dimensionality of hyperspectral data, AD will be influenced by noise, nonlinear correlation of band, or other factors that lead to the decline of detection accuracy. To overcome this problem, a method of hyperspectral AD based on stacked denoising autoencoders (AE) (HADSDA) is proposed. Simultaneously, two different feature detection models, spectral feature (SF) and fused feature by clustering (FFC), are constructed to verify the effectiveness of the proposed algorithm. The SF detection model uses the SF of each pixel. The FFC detection model uses a similar set of pixels constructed by clustering and then fuses the set of pixels by the stacked denoising autoencoders algorithm (SDA). The SDA is an algorithm that can automatically learn nonlinear deep features of the image. Compared with other linear or nonlinear feature extraction methods, the detection result of the proposed algorithm is greatly improved. Experiment results show that the proposed algorithm is an excellent feature learning method and can achieve higher detection performance.

Deep convolutional neural networks (CNNs) have shown outstanding performance in object recognition from natural images. In contrast, object recognition from remote sensing images is more challenging, due to the complex background and inadequate data for training a deep network with a huge number of parameters. We propose a unified deep CNN, called DeepPlane, to simultaneously detect the position and classify the category of aircraft in remote sensing images. This model consists of two correlative deep networks: the first one is designed to generate object proposals as well as feature maps and the second one is cascaded upon the first one to perform classification and box regression in one shot. The “inception module” is introduced to tackle the insufficient training data problem that is one of the most challenging obstacles of detection in remote sensing images. Extensive experiments demonstrate the efficiency of the proposed DeepPlane model. Specifically, DeepPlane could model detection and classification jointly and achieves 91.9% mAP in six categories of aircraft, which advances the state-of-the-art, sometimes considerably, for both tasks.

Recent advances in remote sensing technology have made multisensor data available for the same area, and it is well-known that remote sensing data processing and analysis often benefit from multisource data fusion. Specifically, low spatial resolution of hyperspectral imagery (HSI) degrades the quality of the subsequent classification task while using visible (VIS) images with high spatial resolution enables high-fidelity spatial analysis. A collaborative classification framework is proposed to fuse HSI and VIS images for finer classification. First, the convolutional neural network model is employed to extract deep spectral features for HSI classification. Second, effective binarized statistical image features are learned as contextual basis vectors for the high-resolution VIS image, followed by a classifier. The proposed approach employs diversified data in a decision fusion, leading to an integration of the rich spectral information, spatial information, and statistical representation information. In particular, the proposed approach eliminates the potential problems of the curse of dimensionality and excessive computation time. The experiments evaluated on two standard data sets demonstrate better classification performance offered by this framework.

Automated classification of images across image archives requires reducing the semantic gap between high-level features perceived by humans and low-level features encoded in images. Due to rapidly growing image archives in the Earth science domain, it is critical to automatically classify images for efficient sorting and discovery. In particular, classifying images based on the presence of Earth science phenomena allows users to perform climatology studies and investigate case studies. We present applications of deep learning-based classification of Earth science images.

In recent years, deep learning (DL), a rebranding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, and natural language processing. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV, e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should not only be aware of advancements such as DL, but also be leading researchers in this area. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools, and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as they relate to (i) inadequate data sets, (ii) human-understandable solutions for modeling physical phenomena, (iii) big data, (iv) nontraditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial, and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.

Ocean fronts have been a subject of study for many years, a variety of methods and algorithms have been proposed to address the problem of ocean fronts. However, all these existing ocean front recognition methods are built upon human expertise in defining the front based on subjective thresholds of relevant physical variables. This paper proposes a deep learning approach for ocean front recognition that is able to automatically recognize the front. We first investigated four existing deep architectures, i.e., AlexNet, CaffeNet, GoogLeNet, and VGGNet, for the ocean front recognition task using remote sensing (RS) data. We then propose a deep network with fewer layers compared to existing architecture for the front recognition task. This network has a total of five learnable layers. In addition, we extended the proposed network to recognize and classify the front into strong and weak ones. We evaluated and analyzed the proposed network with two strategies of exploiting the deep model: full-training and fine-tuning. Experiments are conducted on three different RS image datasets, which have different properties. Experimental results show that our model can produce accurate recognition results.

Automatic ship detection in optical remote sensing images has attracted wide attention for its broad applications. Major challenges for this task include the interference of cloud, wave, wake, and the high computational expenses. We propose a fast and robust ship detection algorithm to solve these issues. The framework for ship detection is designed based on deep convolutional neural networks (CNNs), which provide the accurate locations of ship targets in an efficient way. First, the deep CNN is designed to extract features. Then, a region proposal network (RPN) is applied to discriminate ship targets and regress the detection bounding boxes, in which the anchors are designed by intrinsic shape of ship targets. Experimental results on numerous panchromatic images demonstrate that, in comparison with other state-of-the-art ship detection methods, our method is more efficient and achieves higher detection accuracy and more precise bounding boxes in different complex backgrounds.

We investigate the effectiveness of deep neural network for cross-domain classification of remote sensing images in this paper. In the network, class centroid alignment is utilized as a domain adaptation strategy, making the network able to transfer knowledge from the source domain to target domain on a per-class basis. Since predicted labels of target data should be used to estimate the centroid of each class, we use overall centroid alignment as a coarse domain adaptation method to improve the estimation accuracy. In addition, rectified linear unit is used as the activation function to produce sparse features, which may improve the separation capability. The proposed network can provide both aligned features and an adaptive classifier, as well as obtain label-free classification of target domain data. The experimental results using Hyperion, NCALM, and WorldView-2 remote sensing images demonstrated the effectiveness of the proposed approach.

Deep convolutional neural networks (CNNs) have been widely used to obtain high-level representation in various computer vision tasks. However, for remote scene classification, there are not sufficient images to train a very deep CNN from scratch. From two viewpoints of generalization power, we propose two promising kinds of deep CNNs for remote scenes and try to find whether deep CNNs need to be deep for remote scene classification. First, we transfer successful pretrained deep CNNs to remote scenes based on the theory that depth of CNNs brings the generalization power by learning available hypothesis for finite data samples. Second, according to the opposite viewpoint that generalization power of deep CNNs comes from massive memorization and shallow CNNs with enough neural nodes have perfect finite sample expressivity, we design a lightweight deep CNN (LDCNN) for remote scene classification. With five well-known pretrained deep CNNs, experimental results on two independent remote-sensing datasets demonstrate that transferred deep CNNs can achieve state-of-the-art results in an unsupervised setting. However, because of its shallow architecture, LDCNN cannot obtain satisfactory performance, regardless of whether in an unsupervised, semisupervised, or supervised setting. CNNs really need depth to obtain general features for remote scenes. This paper also provides baseline for applying deep CNNs to other remote sensing tasks.

We evaluated how deep convolutional neural networks (DCNN) could assist in the labor-intensive process of human visual searches for objects of interest in high-resolution imagery over large areas of the Earth’s surface. Various DCNN were trained and tested using fewer than 100 positive training examples (China only) from a worldwide surface-to-air-missile (SAM) site dataset. A ResNet-101 DCNN achieved a 98.2% average accuracy for the China SAM site data. The ResNet-101 DCNN was used to process ∼19.6  M image chips over a large study area in southeastern China. DCNN chip detections (∼9300) were postprocessed with a spatial clustering algorithm to produce a ranked list of ∼2100 candidate SAM site locations. The combination of DCNN processing and spatial clustering effectively reduced the search area by ∼660X (0.15% of the DCNN-processed land area). An efficient web interface was used to facilitate a rapid serial human review of the candidate SAM sites in the China study area. Four novice imagery analysts with no prior imagery analysis experience were able to complete a DCNN-assisted SAM site search in an average time of ∼42  min. This search was ∼81X faster than a traditional visual search over an equivalent land area of ∼88,640  km2 while achieving nearly identical statistical accuracy (∼90% F1).

This paper proposes a model of dual-channel convolutional neural network (CNN) that is designed for change detection in SAR images, in an effort to acquire higher detection accuracy and lower misclassification rate. This network model contains two parallel CNN channels, which can extract deep features from two multitemporal SAR images. For comparison and validation, the proposed method is tested along with other change detection algorithms on both simulated SAR images and real-world SAR images captured by different sensors. The experimental results demonstrate that the presented method outperforms the state-of-the-art techniques by a considerable margin.

Feature extraction has always been a difficult problem in the classification performance of synthetic aperture radar automatic target recognition (SAR-ATR). It is very important to select discriminative features to train a classifier, which is a prerequisite. Inspired by the great success of convolutional neural network (CNN), we address the problem of SAR target classification by proposing a feature extraction method, which takes advantage of exploiting the extracted deep features from CNNs on SAR images to introduce more powerful discriminative features and robust representation ability for them. First, the pretrained VGG-S net is fine-tuned on moving and stationary target acquisition and recognition (MSTAR) public release database. Second, after a simple preprocessing is performed, the fine-tuned network is used as a fixed feature extractor to extract deep features from the processed SAR images. Third, the extracted deep features are fused by using a traditional concatenation and a discriminant correlation analysis algorithm. Finally, for target classification, K-nearest neighbors algorithm based on LogDet divergence-based metric learning triplet constraints is adopted as a baseline classifier. Experiments on MSTAR are conducted, and the classification accuracy results demonstrate that the proposed method outperforms the state-of-the-art methods.

High-resolution commercial synthetic aperture radar (SAR) satellites with resolutions of several meters have recently been used for effective disaster monitoring. One study reported the earthquake’s displacement using the pixel matching method with both pre- and postevent TerraSAR-X data, with a validated accuracy of ∼30  cm at global navigation satellite system (GNSS) Earth observation network (GEONET) reference points. However, it is insufficient to determine the accuracy using analysis of only a couple of data points per orbit. In addition, the errors were not reported because the number of data samples was too small to discuss the statistics. In order to better understand displacement accuracy, we analyzed displacement features using the pixel matching method to evaluate the relative geolocation accuracies of the TerraSAR-X product. First, we used fast Fourier transform oversampling 16 times to develop the pixel matching method for estimating the displacement at the subpixel level using the TerraSAR-X StripMap dataset. Second, we applied this methodology to 20 pairs of images from the Tokyo metropolitan area and calculated the displacement for each image pair. Third, we conducted spatial and temporal analyses in order to understand the displacement features. Finally, we evaluated the displacement accuracy by comparison with GEONET and solid earth tide data as a reference.

Research on the scattering behavior of terahertz (THz) waves plays a prominent part in the development of remote sensing and imaging systems. Due to the relatively short wavelengths of THz waves, materials that could be considered smooth at microwave band may begin to display rough surface in the THz region. To study the scattering characteristics and implement parameter estimation and imaging of rough surface rotating targets in the THz band, THz radar systems with carrier frequencies of 220 and 440 GHz were built, and experiments based on a series of rough surface cylinders were carried out. From the time–frequency distributions and imaging results, the transition of scattering characteristics from smooth surface to rough surface is clearly visible, which is consistent with the fact that as the roughness increases, there is an overall increase in diffuse scattering. In addition, methods of rotation period estimation and target size estimation were proposed and applied to experimental data of the THz radar systems, and their validities were verified by the high-precision estimation results.

Considering the sparseness of scatterers in the scene of a synthetic aperture radar (SAR) image, we propose a modified model for SAR images with enhanced features by automatically choosing variable lk-norm and regularization parameter. The approach is based on a regularized reconstruction of the scattering field, which employs prior information of the region of interest. It leads to an alternating iterative algorithm for the modeling. The method is constructed based on variable lk-norm and regularization parameter. Here, k is a function of the imaged region and it could be estimated during the iteration process to the scattering field. The regularization parameter is changing because it is being determined by k. Moreover, the parameter estimators of the presented model are derived by applying the method of log cumulants-based on Mellin transform. Compared to conventional SAR regularization methods, the proposed method reconstructs images with increased resolution, reduced clutter, and reduced computation cost. We demonstrate the performance of the method on real SAR scenes. The experiment results of measured SAR data prove the effectiveness.

Soil moisture plays an important role in the water cycle within the surface ecosystem, and it is the basic condition for the growth of plants. Currently, the spatial resolutions of most soil moisture data from remote sensing range from ten to several tens of km, while those observed in-situ and simulated for watershed hydrology, ecology, agriculture, weather, and drought research are generally <1  km. Therefore, the existing coarse-resolution remotely sensed soil moisture data need to be downscaled. This paper proposes a universal and multitemporal soil moisture downscaling method suitable for large areas. The datasets comprise land surface, brightness temperature, precipitation, and soil and topographic parameters from high-resolution data and active/passive microwave remotely sensed essential climate variable soil moisture (ECV_SM) data with a spatial resolution of 25 km. Using this method, a total of 288 soil moisture maps of 1-km resolution from the first 10-day period of January 2003 to the last 10-day period of December 2010 were derived. The in-situ observations were used to validate the downscaled ECV_SM. In general, the downscaled soil moisture values for different land cover and land use types are consistent with the in-situ observations. Mean square root error is reduced from 0.070 to 0.061 using 1970 in-situ time series observation data from 28 sites distributed over different land uses and land cover types. The performance was also assessed using the GDOWN metric, a measure of the overall performance of the downscaling methods based on the same dataset. It was positive in 71.429% of cases, indicating that the suggested method in the paper generally improves the representation of soil moisture at 1-km resolution.

The detailed three-dimensional modeling of buildings utilizing elevation data, such as those provided by light detection and ranging (LiDAR) airborne scanners, is increasingly demanded today. There are certain application requirements and available datasets to which any research effort has to be adapted. Our dataset includes aerial orthophotos, with a spatial resolution 20 cm, and a digital surface model generated from LiDAR, with a spatial resolution 1 m and an elevation resolution 20 cm, from an area of Athens, Greece. The aerial images are fused with LiDAR, and we classify these data with a multilayer feedforward neural network for building block extraction. The innovation of our approach lies in the preprocessing step in which the original LiDAR data are super-resolution (SR) reconstructed by means of a stochastic regularized technique before their fusion with the aerial images takes place. The Lorentzian estimator combined with the bilateral total variation regularization performs the SR reconstruction. We evaluate the performance of our approach against that of fusing unprocessed LiDAR data with aerial images. We present the classified images and the statistical measures confusion matrix, kappa coefficient, and overall accuracy. The results demonstrate that our approach predominates over that of fusing unprocessed LiDAR data with aerial images.

Identification of water extents in Landsat images is challenging due to surfaces with similar reflectance to water extents. The objective of this study is to provide stable and accurate methods for identifying water extents in Landsat images based on meta-heuristic algorithms. Then, seven Landsat images were selected from various environmental regions in Iran. Training of the algorithms was performed using 40 water pixels and 40 nonwater pixels in operational land imager images of Chitgar Lake (one of the study regions). Moreover, high-resolution images from Google Earth were digitized to evaluate the results. Two approaches were considered: index-based and artificial intelligence (AI) algorithms. In the first approach, nine common water spectral indices were investigated. AI algorithms were utilized to acquire coefficients of optimal band combinations to extract water extents. Among the AI algorithms, the artificial neural network algorithm and also the ant colony optimization, genetic algorithm, and particle swarm optimization (PSO) meta-heuristic algorithms were implemented. Index-based methods represented different performances in various regions. Among AI methods, PSO had the best performance with average overall accuracy and kappa coefficient of 93% and 98%, respectively. The results indicated the applicability of acquired band combinations to extract accurately and stably water extents in Landsat imagery.

The Doppler lidar technology is known for its ability to measure accurate winds with fine time and space resolutions. The derivation of turbulence parameters from lidar wind measurement has been attempted by several authors. All of them relate the turbulence parameters to long-time series (several tens of minutes) of wind measurements. The method presented here retrieves estimations of the atmospheric turbulence at much finer time scales. The technique is based on a wind reconstruction method applied to a five-beam wind Doppler lidar (namely the WindCube model by Leosphere). The method relies on a particle filter. The suggested reconstruction algorithm links the lidar observations to numerical particles to obtain turbulence estimations every time new observations are available. The high frequency of the estimations is an innovation and is detailed and discussed here. Moreover, the presented algorithm enables reconstruction of the wind in three dimensions in the observed volume. Thus, we locally have access to the spatial variability of the turbulent atmosphere. The suggested algorithm is applied to a set of real observations. The results show that the estimation of the turbulent parameters is significantly improved. They open the way to the use of lidars for scientific and industrial purposes such as site studies for wind farms.

Water monitoring is an important part of water resource management and has become an essential aspect of remote sensing. A number of indices have been developed for water extraction using satellite images. Even though all indices can extract the extent of a water body, none can do so without including a noise component, such as topographic shadows, cloud shadows, snow, ice, and buildup areas, all of which have spectrally similar characteristics under certain circumstances. In order to select the best index for water body extraction, several water indices have been compared. This paper proposes a method for extracting water bodies called the water extraction surface temperature index (WESTI). This method uses normalized difference water index (NDWI) and land surface temperature to eliminate the noise components, especially in mountainous and cold areas where other indices have very low accuracy. The results have shown that WESTI improves the NDWI results by removing more than 80% of topographic shadows, with an overall accuracy of 99% in all cases.

Updated and accurate information of rice-growing areas is vital for food security and investigating the environmental impact of rice ecosystems. The intent of this work is to explore the feasibility of dual-polarimetric C-band Radar Imaging Satellite-1 (RISAT-1) data in delineating rice crop fields from other land cover features. A two polarization combination of RISAT-1 backscatter, namely ratio (HH/HV) and difference (HH−HV), significantly enhanced the backscatter difference between rice and nonrice categories. With these inputs, a QUEST decision tree (DT) classifier is successfully employed to extract the spatial distribution of rice crop areas. The results showed the optimal polarization combination to be HH along with HH/HV and HH−HV for rice crop mapping with an accuracy of 88.57%. Results were further compared with a Landsat-8 operational land imager (OLI) optical sensor-derived rice crop map. Spatial agreement of almost 90% was achieved between outputs produced from Landsat-8 OLI and RISAT-1 data. The simplicity of the approach used in this work may serve as an effective tool for rice crop mapping.

Reducing the spectral dimension of hyperspectral data without loss of information is an important process in hyperspectral imagery. By adding the collaborative reconstructive information in linear discriminant analysis (LDA), we propose a discriminant collaborative neighborhood preserving embedding (DCNPE) for feature extraction from hyperspectral images. In the proposed DCNPE, an l2-graph is constructed based on the collaborative representation (CR). The edge weights of graph are calculated by l2-norm minimization using all samples to represent the pointed sample. The proposed DCNPE method aims to find a projection, which can preserve the CR-based reconstruction relationship of data as well as maximize the class discrimination of the data. DCNPE inherits the advantages of both LDA and CR. It not only overcomes the reduced dimension limitation of LDA but also preserves the collaborative neighborhood relations of data through l2-graph. Experimental results on three real HSI datasets certify its effectiveness of dimensionality reduction by showing a better classification performance.

Coffee is an important crop in tropical regions of the world; about 125 million people depend on coffee agriculture for their livelihoods. Understanding the spatial extent of coffee fields is useful for management and control of coffee pests such as Hypothenemus hampei and other pests that use coffee fruit as a host for immature stages such as the Mediterranean fruit fly, for economic planning, and for following changes in coffee agroecosystems over time. We present two methods for detecting Coffea arabica fields using remote sensing and geospatial technologies on WorldView-2 high-resolution spectral data of the Kona region of Hawaii Island. The first method, a pixel-based method using a maximum likelihood algorithm, attained 72% producer accuracy and 69% user accuracy (68% overall accuracy) based on analysis of 104 ground truth testing polygons. The second method, an object-based image analysis (OBIA) method, considered both spectral and textural information and improved accuracy, resulting in 76% producer accuracy and 94% user accuracy (81% overall accuracy) for the same testing areas. We conclude that the OBIA method is useful for detecting coffee fields grown in the open and use it to estimate the distribution of about 1050 hectares under coffee agriculture in the Kona region in 2012.

Advanced spaceborn thermal emission and reflection radiometer (ASTER) has fine spectral bands in short-wave infrared (SWIR) and thermal infrared (TIR) regions of the electromagnetic spectrum. The purpose behind the study is to explore the potential of ASTER for lithological and minerals detection; in comparison with Landsat-ETM+ Khaira Murat range (KMR) of Gali Jagir area, district Attock was selected as a test site; enriched with industrial minerals of the Eocene age. Maximum likelihood classification was applied on Landsat-ETM+ and ASTER images. Maximum likelihood classification on ASTER satellite image exhibits better discrimination among various lithologies as compared to Landsat-ETM+. Classified image of ASTER showed a correlation coefficient of 0.6 with the geological survey of Pakistan’s map while a classified image of Landsat-ETM+ exhibited a correlation of only 0.43. Landsat-ETM+ and ASTER satellite images were further investigated for minerals detection. Landsat-ETM+ band ratio detected clay. ASTER SWIR band ratios detected various clay and carbonate minerals. X’PertPRO diffractometer and differential thermal analysis of field samples verified the detected minerology. The results suggest that ASTER can be successfully used for lithological and minerals mapping of less-examined areas.