We present two generalized linear correlation filters (CFs) that encompass most of the state-of-the-art linear CFs. The common criteria that arc used in linear CF design are the mean squared error (MSE), output noise variance (ONV), and average similarity measure (ASM). We present a simple formulation that uses an optimal tradeoff among these criteria both constraining and not constraining the correlation peak value, and refer to them as generalized Constrained Correlation Filter (CCF) and Unconstrained Couelation Filter (UCF). We show that most state-of-the-art linear CFs arc subsets of these filters. We present a technique for efficient UCF computation. We also introduce the modified CCF (mCCF) that chooses a unique correlation peak value for each training image, and show that mCCF usually outperforms both UCF and CCF.

A research platform with four cameras in the infrared and visible spectral domains is under development at the Norwegian Defence Research Establishment (FFI). The platform will be mounted on a high-speed jet aircraft and will primarily be used for image acquisition and for development and test of automatic target recognition (ATR) algorithms. The sensors on board produce large amounts of data, the algorithms can be computationally intensive and the data processing is complex. This puts great demands on the system architecture; it has to run in real-time and at the same time be suitable for algorithm development. In this paper we present an architecture for ATR systems that is designed to be exible, generic and efficient. The architecture is module based so that certain parts, e.g. specific ATR algorithms, can be exchanged without affecting the rest of the system. The modules are generic and can be used in various ATR system configurations. A software framework in C++ that handles large data ows in non-linear pipelines is used for implementation. The framework exploits several levels of parallelism and lets the hardware processing capacity be fully utilised. The ATR system is under development and has reached a first level that can be used for segmentation algorithm development and testing. The implemented system consists of several modules, and although their content is still limited, the segmentation module includes two different segmentation algorithms that can be easily exchanged. We demonstrate the system by applying the two segmentation algorithms to infrared images from sea trial recordings.

In this paper, we present our implementation of a cascaded Histogram of Oriented Gradient (HOG) based pedestrian detector. Most human detection algorithms can be implemented as a cascade of classifiers to decrease computation time while maintaining approximately the same performance. Although cascaded versions of Dalal and Triggs's HOG detector already exist, we aim to provide a more detailed explanation of an implementation than is currently available. We also use Asymmetric Boosting instead of Adaboost to train the cascade stages. We show that this reduces the number of weak classifiers needed per stage. We present the results of our detector on the INRIA pedestrian detection dataset and compare them to Dalal and Triggs's results.

No-reference measurement for image quality, where an original error-free image is not provided as reference, plays an important role in image processing and analysis. This paper mainly investigates three no-reference image-quality metrics, which are based on the standard deviation, the maximum, and the mean of the magnitude of the intensity gradient of pixels. Each measurement metric is critically accessed using low resolution gray-scale images, which are acquired by unmanned aerial vehicles cruising over the city and aim to disclose the movement of vehicles such as a semi -truck, light colored cars, and dark colored cars, etc. The experimental results demonstrate that, compared to alternative schemes, the standard deviation based metric provides a more accurate measurement about the quality of images. In addition, standard deviation based scheme demonstrates superior correlation with alternative schemes to measure the quality of images.

In this paper, an unsupervised pedestrian detection algorithm is proposed. An input image is first divided into overlapping detection windows in a sliding fashion and Histogram of Oriented Gradients (HOG) features are collected over each window using non-overlapping cells. A distance metric is used to determine the distance between histograms of corresponding cells in each detection window and the average pedestrian HOG template (determined a priori). These distances over a group of cells are concatenated to obtain the feature vector pertaining to a block of cells. The feature vectors over overlapping blocks of cells are concatenated to form the distance feature vector of a detection window. Each window provides a data sample and the data samples extracted from the whole image are then modeled as a normalcy class using Support Vector Data Description (SVDD). The benefit of using the state-of-the-art SVDD technique to model the normalcy class is that it can be controlled by setting an upper limit on the permissible outliers during the modeling process. Assuming that most of the image is covered by background, the outliers that are detected during the modeling of the normalcy class can be hypothesized as detection windows that contain pedestrians in them. The detections are obtained at different scales in order to account for the different sizes of pedestrians. The final pedestrian detections are generated by applying non-maximal suppression on all the detections at all scales. The system is tested on the INRIA pedestrian dataset and its performance analyzed with respect to accuracy and detection rate.

This work extends earlier work on the real-time photogrammetric stitching of staring arrays of high resolution videos on commercial off the shelf hardware. The blending is both further optimised for Graphics Processor Unit (GPU) implementation and extended from one to two dimensions to allow for multiple layers or arbitrary arrangements of cameras. The incorporation of stabilisation inputs allows the stitching algorithm to provide space stabilised panoramas. The final contribution is to decrease the sensitivity to depth of the stitching procedure, especially for wide aperture baselines. Finally timing tests and some resultant stitched panoramas are presented and discussed.

We describe a Bayesian filtering process that estimates the pose (3-D position and orientation) of a moving rigid body using multiple cameras. The estimator also produces an arbitrary number of pose derivatives. We first discuss various ways to represent 3-D orientation. Unfortunately all 3-parameter representations have areas of instability. Higher dimensional representations are stable but require unwieldy constraints. Our combination of an axis-angle vector with a unit quaternion represents orientation minimally while remaining stable under realistic circumstances. Our dynamic model of rigid body motion can include an arbitrary number of derivatives, and we explicitly develop it up to the third order. Our observation model takes a predicted pose and produces the 2-D locations in each camera's image plane of the visible features on the body's surface. We provide noise terms for both the dynamic and observation models. We describe how our models are used in extended and unscented Kalman filters, and also in a particle filter. As a baseline we also describe a non-linear least squares method that uses just our observation model. We construct a synthetic testing scenario, and use root-mean-square error analysis to grade the relative performance of each model/filter combination. We derive the Cramer-Rao lower bound that gives the best achievable performance for our particular scenario. Our results show that adding derivatives to the state vector significantly improves the accuracy of pose estimates, and we also show that an unscented Kalman filter with a second order dynamic model is best suited to the task.

A motorboat’s wakes can cause difficulties as they appear in a track’s gate. While a motorboat is idle for long enough, the gate size reduces to its minimum. Immediately after, if motorboat makes a stop-move maneuver with maximum acceleration, true radar measurement can appear off the gate. In this case, the tracker can continue to update the track with target’s wake measurements falling in gate and this can mislead the tracker. In order not to miss the true measurement belonging to a track, a secondary gate should be created with larger size than that of the primary gate at the same gate center. However, this gate should only be created when an anomaly is detected at the tracker. This paper investigates when such secondary gates should be created. In this paper, a score-based method is presented to control gating in the presence of wake left behind a suddenly accelerated boat. This method is based on scoring the probability distributions characterizing the number radar measurements. The number of detections coming from clutter is K-distributed and the number of detections belonging to true target is Poisson distributed across the surveillance region. The method treats the motorboat’s wake as target originated measurement rather than clutter. Therefore, a probability distribution similar to true target’s with a different parameter is used to score the wake separately. Based on the scores, security gating decision is made. The real data experiments show that track continuity is maintained successfully with the use of score-based gating control method.

This paper describes a study into the impact of local environmental conditions on the detection of point targets using wide field of view infrared sensors on airborne platforms. A survey of the common complexity metrics for measuring IR clutter, and common point target detection algorithms was conducted. A quantitative evaluation was performed using 20 hours of infrared imagery collected over a three month period from helicopter flights in a variety of clutter environments. The research method, samples of the IR data sets, and results of the correlation between environmental conditions, scene complexity metrics and point target detection algorithms are presented. The key findings of this work are that variations in IR detection performance can be attributed to a combination of environmental factors (but no single factor is sufficient to describe performance variations), and that historical clutter metrics are insufficient to describe the performance of modern detection algorithms.

A recursive algorithm based on hidden Markov models is used to build a model of the identification target. The end result of the recursive matching is an optimal scene-to-model transformation, along with a recognition degree of suitability value between the scene and the model. The hierarchical structure of the model allows a maximization of the target identification probability.

Currently there is a huge demand for simple low cost IR cameras for both civil and military applications, among which one of the most common is the surveillance of restricted access zones. In the design of low cost IR cameras, it is necessary to avoid the use of several elements present in more sophisticated cameras, such as the refrigeration systems and the temperature control of the detectors, so as to prevent the use of a mechanical modulator of the incident radiation (chopper). Consequently, the detection algorithms must reliably separate the target signal from high noise and drift caused by temporal variations of the background image of the scene and the additional drift due to thermal instability detectors. A very important step towards this goal is the design of a preprocessing stage to eliminate noise. Thus, in this work we propose using the Empirical Mode Decomposition (EMD) method to attain this objective. In order to evaluate the quality of the reconstructed clean signal, the Average to Peak Ratio is assessed to evaluate the effectiveness in reconstructing the waveform of the signal from the target. We compare the EMD method with other classical method of noise cancellation based on the Discrete Wavelet Transform (DWT). The results reported by simulations show that the proposed scheme based on EMD performs better than traditional ones.

This paper presents a novel coastal region detection method for infrared search and track. The coastal region detection is critical to home land security and ship defense. Detected coastal region information can be used to the design of target detector such as moving target detection and threshold setting. We can detect coastal regions robustly by combining the infrared image segmentation and sensor line-of-sight (LOS) information. The K-means-based image segmentation can provide initial region information and the sensor LOS information can predict the approximate horizon location in images. The evidence of coastal region is confirmed by contour extraction results. The experimental results on remote coasts and near coasts validate the robustness of the proposed coastal region detector.

This paper addresses the detection and localization of persons in LWIR imagery which is useful especially in visual surveillance tasks such as intruder detection in military camps or for gaining situational awareness. A robust image retrieval function is used after a previous hot spot detection and localization step in LWIR using a suitable, extensive image data base that covers a variety of different shapes and appearances of persons in LWIR. The basic idea behind this approach is, in contrast to the visual optical band (VIS), that persons in thermal infrared exhibit somehow similar, weakly individualized signatures which can be matched to a sufficient degree to images in the data base and, thus, can be distinguished from background structures and other objects. Dedicated pre and post processing routines optimize the results and compensate for a possibly occuring lack of image features needed by the image retrieval function. The achieved results document the practical benefit and the robustness of the presented aproach.

Person recognition is a key issue in visual surveillance. It is needed in many security applications such as intruder detection in military camps but also for gaining situational awareness in a variety of different safety applications. A solution for LWIR videos coming from a moving camera is presented that is based on hot spot classification to distinguish persons from background clutter and other objects. We especially consider objects in higher distance with small appearance in the image. Hot spots are detected and tracked along the videos. Various image features are extracted from the spots and different classifiers such as SVM or AdaBoost are evaluated and extended to utilize the temporal information. We demonstrate that taking advantage of this temporal context can improve the classification performance.

Target detection and tracking with passive infrared (IR) sensors can be challenging due to significant degradation and corruption of target signature by atmospheric transmission and clutter effects. This paper summarizes our efforts in phenomenology modeling of boosting targets with IR sensors, and developing algorithms for tracking targets in the presence of background clutter. On the phenomenology modeling side, the clutter images are generated using a high fidelity end-to-end simulation testbed. It models atmospheric transmission, structured clutter and solar reflections to create realistic background images. The dynamics and intensity of a boosting target are modeled and injected onto the background scene. Pixel level images are then generated with respect to the sensor characteristics. On the tracking analysis side, a particle filter for tracking targets in a sequence of clutter images is developed. The particle filter is augmented with a mechanism to control particle flow. Specifically, velocity feedback is used to constrain and control the particles. The performance of the developed “adaptive” particle filter is verified with tracking of a boosting target in the presence of clutter and occlusion.

The Dynamic Data Driven Applications System (DDDAS) concept uses applications modeling, mathematical algorithms, and measurement systems to work with dynamic systems. A dynamic systems such as Automatic Target Recognition (ATR) is subject to sensor, target, and the environment variations over space and time. We use the DDDAS concept to develop an ATR methodology for multiscale-multimodal analysis that seeks to integrated sensing, processing, and exploitation. In the analysis, we use computer vision techniques to explore the capabilities and analogies that DDDAS has with information fusion. The key attribute of coordination is the use of sensor management as a data driven techniques to improve performance. In addition, DDDAS supports the need for modeling from which uncertainty and variations are used within the dynamic models for advanced performance. As an example, we use a Wide-Area Motion Imagery (WAMI) application to draw parallels and contrasts between ATR and DDDAS systems that warrants an integrated perspective. This elementary work is aimed at triggering a sequence of deeper insightful research towards exploiting sparsely sampled piecewise dense WAMI measurements – an application where the challenges of big-data with regards to mathematical fusion relationships and high-performance computations remain significant and will persist. Dynamic data-driven adaptive computations are required to effectively handle the challenges with exponentially increasing data volume for advanced information fusion systems solutions such as simultaneous target tracking and ATR.

Correlation filters have been shown to perform well for localization and classification tasks. In this paper, we investigate different techniques used to pre- and post-process the data to improve the correlation filter performance. In addition, we present an efficient method to use zero-mean, unit-norm test chips in a large test image. We compare the localization, classification, and recognition performance when we apply one or more of these methods.

Synthetic Discriminant Function (SDF) formulation of correlation filters provides constraints for forming target subspaces for a target set. In this paper we extend the SDF formulation to include quadratic constraints and use this solution to form nonlinear manifolds in the target space. The theory for forming these manifolds will be developed and demonstrated with data.

Discrete Fourier transforms (DFTs) are typically used to compute correlations and implementing correlation filters (CFs). Because of the properties of DFTs, resulting correlations are actually circular (also known as periodic) correlations. Using current CF design techniques, it is not possible to design a CF that produces exactly the desired linear correlation output. There are several techniques that may be used to reduce the effects of circular correlation. In this paper, we describe these techniques and provide some experimental results that compare these techniques.

This work is sponsored by the Air Force Research Laboratory (AFRL). The views and conclusions contained in this document are those of the authors and should not be interpreted as representing offcial policies, either expressed or implied, of AFRL, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein. This document is approved for public released via PA#: 88ABW-2013-1359.

A vision system was designed for people detection to provide support to SWAT team members operating in challenging environments such as low-to-no light, smoke, etc. When the vision system is mounted on a mobile robot platform: it will enable the robot to function as an effective member of the SWAT team; to provide surveillance information; to make first contact with suspects; and provide safe entry for team members. The vision task is challenging because SWAT team members are typically concealed, carry various equipment such as shields, and perform tactical and stealthy maneuvers. Occlusion is a particular challenge because team members operate in close proximity to one another. An uncooled electro-opticaljlong wav e infrared (EO/ LWIR) camera, 7.5 to 13.5 m, was used. A unique thermal dataset was collected of SWAT team members from multiple teams performing tactical maneuvers during monthly training exercises. Our approach consisted of two stages: an object detector trained on people to find candidate windows, and a secondary feature extraction, multi-kernel (MK) aggregation and classification step to distinguish between SWAT team members and civilians. Two types of thermal features, local and global, are presented based on ma ximally stable extremal region (MSER) blob detection. Support vector machine (SVM) classification results of approximately [70, 93]% for SWAT team member detection are reported based on the exploration of different combinations of visual information in terms of training data.

The present work is a part of a larger project on recognizing and identifying weapons from a single image and assessing threats in public places. Methods of populating the weapon ontology have been shown. A clustering-based approach of constructing visual hierarchies on the base of extracted geometric features of weapons has been proposed. The convergence of a sequence of visual hierarchy trees to a conceptual hierarchy tree has been discussed. For illustrative purposes, from the growing conceptual ontology, a conceptual hierarchy tree has been chosen as a point of convergence for a sequence of visual hierarchy trees. A new approach is defined, on the base of the Gonzalez' algorithm, to generate the visual hierarchies. The closest visual hierarchy tree is selected as the search environment for a query weapon. A method of threat assessment is proposed. This method uses the attribute-rich conceptual hierarchy tree to evaluate the results from the visual hierarchy tree search. The two trees are linked at the leaf-level, because the visual hierarchy closest to the conceptual has the same distribution of the leaf nodes. A set of experimental results are reported to validate the theoretical concepts. A portion of the existing weapon ontology is used for this purpose.

This paper mainly discusses the chipping and segmentation of target-of-interest (TOI) from lowresolution gray-scale electro-optical (EO) data, which is acquired by unmanned aerial vehicle (UAV) hovering above the city. As the processing for automatic target recognition, chipping and segmentation of TOI consist of the following two steps: regional chipping and target segmentation. Regional chipping is dedicated to obtaining the minimal region surrounding the TOI so that it can be accurately and efficiently recognized. Target segmentation is dedicated to isolating a TOI from the background and is implemented using the diffusion algorithm. The whole work is accomplished in MATLAB and is critically assessed using the given EO data.

Real-time detection of objects in video sequences captured from an aerial platforms is a key task for surveillance applications. It is common to perform expensive frame to frame registration as preprocessing to moving object detection in this type of application, and there is no principled approach to the detection of stationary targets.We explore the Spectral Residual algorithm,⁶ a fast linearithmic run time saliency model which requires no training and has no temporal dependencies, and is capable of detecting proto-objects in a single image. In this paper we describe methods for enhancing the Spectral Residual saliency algorithm to generate candidate object detections from video sequences captured from moving platforms. These object candidates can then be passed to a classification module for further processing. We describe a method that makes the Spectral Residual algorithm more robust to natural variances in color images, and a pyramidal approach to make the processes more robust to objects of varying size. Furthermore we describe a technique for processing the resulting saliency map into a set of tight bounding boxes suitable for extracting image regions for classification. These methods result in a system that is fast, robust, and efficient with reliable performance for low SWaP surveillance platforms.

We propose a methodology for using sensor metadata and targeted preprocessing to optimize which selection from a large suite of algorithms are most appropriate for a given data set. Rather than applying several general purpose algorithms or requiring a human operator to oversee the analysis of the data, our method allows the most effective algorithm to be automatically chosen, conserving both computational, network and human resources. For example, the amount of video data being produced daily is far greater than can ever be analyzed. Computer vision algorithms can help sift for the relevant data, but not every algorithm is suited to every data type nor is it efficient to run them all. A full body detector won’t work well when the camera is zoomed in or when it is raining and all the people are occluded by foul weather gear. However, leveraging metadata knowledge of the camera settings and the conditions under which the data was collected (generated by automatic preprocessing), face or umbrella detectors could be applied instead, increasing the likelihood of a correct reading. The Lockheed Martin AKITA™ system is a modular knowledge layer which uses knowledge of the system and environment to determine how to most efficiently and usefully process whatever data it is given.

This paper describes a fully automated system to align an infra-red laser beam with a small retro-reflector over a wide range of distances. The component development and test were especially used for an open-path spectrometer gas detection system. Using blob detection under OpenCV library, an automatic alignment algorithm was designed to achieve fast and accurate target detection in a complex background environment. Test results are presented to show that the proposed algorithm has been successfully applied to various target distances and environment conditions.

The problem of distributed estimation of a parametric function in space is stated as a maximum likelihood estimation problem. The function can represent a parametric physical ¯eld generated by an object or be a deterministic function that parameterizes an inhomogeneous spatial random process. In our formulation, a sparse network of homogeneous sensors takes noisy measurements of the function. Prior to data transmission, each sensor quantizes its observation to L levels. The quantized data are then communicated over parallel noisy channels to a fusion center for a joint estimation. The numerical examples are provided for the cases of (1) a Gaussian-shaped ¯eld that approximates the distribution of pollution or fumes produced by an object and (2) a radiation ¯eld due to a spatial counting process with the intensity function decaying according to the inverse square law. The dependence of the mean- square error on the number of sensors in the network, the number of quantization levels, and the SNR in observation and transmission channels is analyzed. In the case of Gaussian-shaped ¯eld, the performance of the developed estimator is compared to unbiased Cramer-Rao Lower Bound.

In this paper we derive an explicit, single expression for a complex-valued, orthonormal basis well localized in time-frequency domain. We construct it from a single real function Φ(x) which is a Gaussian divided by the square root of a Jacobi theta θ₃ function. Then we simplify Φ(x) to the form of inverse square root of a Jacobi theta θ₃ function. We show that the shape of Φ(x) can be changed from Gaussian-like to rectangular-like with a single parameter. The basis generating function Φ(x) and its Fourier transform Φ have exponential decay. We also show how to modify a standard I and Q processor to compute complex-valued time-frequency expansion coefficients.

A synthetic aperture radar system operating in congested frequency bands suffers from radio frequency inter­ ference (RFI) from narrowband sources. When RFI interference is suppressed by frequency notching, gaps are introduced into the fast time phase history. This results in a missing data spectral estimation problem, where the missing data increases sidelobe energy and degrades image quality. The adaptive spectral estimation method Iterative Adaptive Approach (IAA) has been shown to provide higher resolution and lower sidelobes than comparable methods, but at the cost of higher computationally complexity. Current fast IAA algorithms reduce the computational complexity using Toeplitz /Vandermonde structures, but are not applicable for missing data cases because these structures are lost. When the number of missing data samples is small, which often is the case in SAR with RFI, we use a low rank completion to restore the Toeplitz/ Vandermonde structures. We show that the computational complexity of the proposed algorithm is considerably lower than the state-of-the-art and demonstrate the utility on a simulated frequency notched SAR imaging problem.

We have created a new autonomous (computer-free) adaptive optics system using holographic modal wavefront sensing and closed-loop control of a MEMS deformable mirror (DM). A multiplexed hologram is recorded using the maximum and minimum actuator positions on the deformable mirror as the "modes". On reconstruction, an input beam is diffracted into pairs of focal spots and the ratio of the intensities of certain pairs determines the absolute wavefront phase at a particular actuator location. We present the results from an ultra-compact, 32-actuator prototype device operating at 100 kHz. It is largely insensitive to obscuration and has a speed independent of the number of actuators.

This paper describes the development and implementation of evaluation metrics for group state estimator (GSE, i.e. group tracking) algorithms. Key differences between group tracker metrics and individual tracker metrics are the method used for track-to-truth association and the characterization of group raid size. Another significant contribution of this work is the incorporation of measured radar performance in assessing tracker performance. The result of this work is a set of measures of performance derived from canonical individual target tracker metrics, extended to characterize the additional information provided by a group tracker. The paper discusses additional considerations in group tracker evaluation, including the definition of a group and group-to-group confusion. Metrics are computed on real field data to provide examples of real-world analysis, demonstrating an approach which provides characterization of group tracker performance, independent of the sensor's performance.

Target detection is difficult when the target is concealed or placed under ground or water. To detect and identify concealed objects from a distance, the analysis of the HyperSpectral Imaging (HSI) and Wideband (WB) data is studied. While the HSI analysis may render surface information about objects, the WB data can reveal information about inner layers of the object and its content. Two of the challenging issues with object identification using HSI are (i) computational complexity of the analysis and (ii) signature mismatch. Here, the robust matched filter is emphasized for HSI processing. In addition, the wideband technology is utilized to provide more information about concealed target, and to support spectral processing for object uncovering more effectively. During simulation, electromagnetic waves and propagation areas are modeled. In fact, an object is modeled as different layers with different thicknesses. The existence of a target is estimated by the detection of spectral signatures relating to materials used in the target. In other words, the simultaneous presence of spectral signatures corresponding to the main materials of the target in the hyperspectral data helps detecting the target. The reflected higher frequency signals provide information about exterior layers of both an object and the background; in addition, the reflected lower frequency signals provide information about interior layers of the object. To identify different objects, the simulation is performed using HSI, and WB technology at different frequencies (MHz- GHz) and powers. Based on simulation, the proposed method can be a promising approach to detect targets.

In previous work, we have given a method for obtaining propagation invariant features for classification of underwater objects from their sonar backscatter in dispersive but rangG'-independent environments. In this paper we consider the derivation of invariant features for classification in range­ dependent environments, based on the parabolic equation.

We describe how one can obtain the phase space differential equation for joint position-wavenumber distributions for pulse propagation with dispersion and attenuation. We show that there are many advantages to the phase space equation both from the point of view of insight and practical calculation. We use the method to obtain new approximations for pulse propagation. The phase space distributions we use are the Wigner distribution and spectrogram.

This PDF file contains the front matter associated with SPIE Proceedings Volume 8744, including the Title Page, Copyright Information, Table of Contents, and the Conference Committee listing.