Detection of targets can be difficult using thermal camera during thermal cross-over periods. Under these conditions, the target takes on the apparent temperature of objects in the foreground or background and becomes undetectable. It is commonly said that the thermal imagery is “washed-out”. When the thermal contrast of an object against its background is zero, many times a polarization contrast of the same object is non-zero. In this paper, we introduce a camera that measures thermal and polarization images in both the mid-wave infrared (MWIR) and long-wave infrared (LWIR). We also show example images and derive a simple equation that explains the conditions under which a polarization signature of an object can be expected.
Polaris Sensor Technologies reports on the development of Pedestrian Automated System for Enforcement and Safety (PASES), a radar and video based system used to monitor vehicle and pedestrian traffic with the intent of improving pedestrian safety. Data is fused from a system of multiple sensors and multiple sensor modalities to identify vehicular violations of pedestrian right of way. A focus was placed on the selection of low cost COTS sensors to make the system more widely available to state and local DOTs with limited budgets. Applications include automated enforcement, adaptive traffic control, and improved intersection and crosswalk design based on high quality data available for traffic engineering. We discuss early results with high fidelity sensors, and the performance trades made in order to make the system affordable. A discussion of the system processing architecture is included which highlights the treatment of each sensor data type, and the means of combining the processed data products into state information related to traffic incidents involving vehicles and pedestrians.
Improved situational awareness results not only from improved performance of imaging hardware, but also when the
operator and human factors are considered. Situational awareness for IR imaging systems frequently depends on the
contrast available. A significant improvement in effective contrast for the operator can result when depth perception is
added to the display of IR scenes. Depth perception through flat panel 3D displays are now possible due to the number of
3D displays entering the consumer market. Such displays require appropriate and human friendly stereo IR video input in
order to be effective in the dynamic military environment. We report on a stereo IR camera that has been developed for
integration on to an unmanned ground vehicle (UGV). The camera has auto-convergence capability that significantly
reduces ill effects due to image doubling, minimizes focus-convergence mismatch, and eliminates the need for the
operator to manually adjust camera properties. Discussion of the size, weight, and power requirements as well as
integration onto the robot platform will be given along with description of the stand alone operation.
Polaris Sensor Technologies (PST) has developed a stereo vision upgrade kit for TALON® robot systems comprised of a
replacement gripper camera and a replacement mast zoom camera on the robot, and a replacement display in the
Operator Control Unit (OCU). Harris Corporation has developed a haptic manipulation upgrade for TALON® robot
systems comprised of a replacement arm and gripper and an OCU that provides haptic (force) feedback. PST and Harris
have recently collaborated to integrate the 3D vision system with the haptic manipulation system. In multiple studies
done at Fort Leonard Wood, Missouri it has been shown that 3D vision and haptics provide more intuitive perception of
complicated scenery and improved robot arm control, allowing for improved mission performance and the potential for
reduced time on target. This paper discusses the potential benefits of these enhancements to robotic systems used for the
domestic homeland security mission.
Polaris Sensor Technologies has developed numerous 3D display systems using a US Army patented approach. These
displays have been developed as prototypes for handheld controllers for robotic systems and closed hatch driving, and as
part of a TALON robot upgrade for 3D vision, providing depth perception for the operator for improved manipulation
and hazard avoidance. In this paper we discuss the prototype rugged 3D laptop computer and its applications to defense
missions. The prototype 3D laptop combines full temporal and spatial resolution display with the rugged Amrel laptop
computer. The display is viewed through protective passive polarized eyewear, and allows combined 2D and 3D
content. Uses include robot tele-operation with live 3D video or synthetically rendered scenery, mission planning and
rehearsal, enhanced 3D data interpretation, and simulation.
Polaris Sensor Technologies has developed a Stereovision Upgrade Kit for TALON robot to provide enhanced depth
perception to the operator. This kit previously required the TALON Operator Control Unit to be equipped with the
optional touchscreen interface to allow for operator control of the camera convergence angle adjustment. This
adjustment allowed for optimal camera convergence independent of the distance from the camera to the object being
viewed. Polaris has recently improved the performance of the stereo camera by implementing an Automatic
Convergence algorithm in a field programmable gate array in the camera assembly. This algorithm uses scene content to
automatically adjust the camera convergence angle, freeing the operator to focus on the task rather than adjustment of the
vision system. The autoconvergence capability has been demonstrated on both visible zoom cameras and longwave
infrared microbolometer stereo pairs.
In this paper, we report on the development of a high definition stereoscopic liquid crystal display for use in a variety of
applications. The display technology provides full spatial and temporal resolution on a liquid crystal display panel
consisting of 1920×1200 pixels at 60 frames per second. Applications include training, mission rehearsal and planning,
and enhanced visualization. Display content can include mixed 2D and 3D data. Source data can be 3D video from
cameras, computer generated imagery, or fused data from a variety of sensor modalities. Recent work involving
generation of 3D terrain from aerial imagery will be demonstrated. Discussion of the use of this display technology in
military and medical industries will be included.
In this paper, we report on the development of a 3D vision field upgrade kit for TALON robot consisting of a
replacement flat panel stereoscopic display, and multiple stereo camera systems. An assessment of the system's use for
robotic driving, manipulation, and surveillance operations was conducted. The 3D vision system was integrated onto a
TALON IV Robot and Operator Control Unit (OCU) such that stock components could be electrically disconnected and
removed, and upgrade components coupled directly to the mounting and electrical connections. A replacement display,
replacement mast camera with zoom, auto-focus, and variable convergence, and a replacement gripper camera with fixed
focus and zoom comprise the upgrade kit. The stereo mast camera allows for improved driving and situational awareness
as well as scene survey. The stereo gripper camera allows for improved manipulation in typical TALON missions.
Uncalibrated stereo imagery experimental and analytical results are presented for path planning and navigation. An
Army Research and Development Engineering Command micro-size UAV was outfitted with two commercial cameras
and flown over varied landscapes. Polaris Sensor Technologies processed the data post flight with an image
correspondence algorithm of their own design. Stereo disparity (depth) was computed despite a quick assembly, image
blur, intensity saturation, noise and barrel distortion. No camera calibration occurred. Disparity maps were computed at
a processing rate of approximately 5 seconds per frame to improve perception. Disparity edges (treeline to ground, voids
and plateaus) were successfully observed and confirmed to be properly identified. Despite the success of localizing this
disparity edges sensitivity to saturated pixels, lens distortion and defocus were strong enough to overwhelm more subtle
features such as the contours of the trees, which should be possible to extract using this algorithm. These factors are
being addressed. The stereo data is displayed on a flat panel 3D display well suited for a human machine interface in
field applications. Future work will entail extraction of intelligence from acquired data and the overlay of such data on
the 3D image as displayed.
In this paper, we report on the development of a high definition stereoscopic liquid crystal display for use in training
applications. The display technology provides full spatial and temporal resolution on a liquid crystal display panel
consisting of 1920×1200 pixels at 60 frames per second. Display content can include mixed 2D and 3D data. Source data
can be 3D video from cameras, computer generated imagery, or fused data from a variety of sensor modalities.
Discussion of the use of this display technology in military and medical industries will be included. Examples of use in
simulation and training for robot tele-operation, helicopter landing, surgical procedures, and vehicle repair, as well as for
DoD mission rehearsal will be presented.
In this paper, we report on the use of a 3D vision field upgrade kit for TALON robot consisting of a replacement flat
panel stereoscopic display, and multiple stereo camera systems. An assessment of the system's use for robotic driving,
manipulation, and surveillance operations was conducted. A replacement display, replacement mast camera with zoom,
auto-focus, and variable convergence, and a replacement gripper camera with fixed focus and zoom comprise the
upgrade kit. The stereo mast camera allows for improved driving and situational awareness as well as scene survey. The
stereo gripper camera allows for improved manipulation in typical TALON missions.
The use of tele-operated Unmanned Ground Vehicles (UGVs) for military uses has grown significantly in recent years
with operations in both Iraq and Afghanistan. In both cases the safety of the Soldier or technician performing the mission
is improved by the large standoff distances afforded by the use of the UGV, but the full performance capability of the
robotic system is not utilized due to insufficient depth perception provided by the standard two dimensional video
system, causing the operator to slow the mission to ensure the safety of the UGV given the uncertainty of the perceived
scene using 2D. To address this Polaris Sensor Technologies has developed, in a series of developments funded by the
Leonard Wood Institute at Ft. Leonard Wood, MO, a prototype Stereo Vision Upgrade (SVU) Kit for the Foster-Miller
TALON IV robot which provides the operator with improved depth perception and situational awareness, allowing for
shorter mission times and higher success rates. Because there are multiple 2D cameras being replaced by stereo camera
systems in the SVU Kit, and because the needs of the camera systems for each phase of a mission vary, there are a
number of tradeoffs and design choices that must be made in developing such a system for robotic tele-operation.
Additionally, human factors design criteria drive optical parameters of the camera systems which must be matched to the
display system being used. The problem space for such an upgrade kit will be defined, and the choices made in the
development of this particular SVU Kit will be discussed.
In September 2009 the Fort Leonard Wood Field Element of the US Army Research Laboratory - Human Research and
Engineering Directorate, in conjunction with Polaris Sensor Technologies and Concurrent Technologies Corporation,
evaluated the objective performance benefits of Polaris' 3D vision upgrade kit for the TALON small unmanned ground
vehicle (SUGV). This upgrade kit is a field-upgradable set of two stereo-cameras and a flat panel display, using only
standard hardware, data and electrical connections existing on the TALON robot. Using both the 3D vision system and a
standard 2D camera and display, ten active-duty Army Soldiers completed seven scenarios designed to be representative
of missions performed by military SUGV operators. Mission time savings (6.5% to 32%) were found for six of the seven
scenarios when using the 3D vision system. Operators were not only able to complete tasks quicker but, for six of seven
scenarios, made fewer mistakes in their task execution. Subjective Soldier feedback was overwhelmingly in support of
pursuing 3D vision systems, such as the one evaluated, for fielding to combat units.
The flow of information among our armed forces is greater than ever and the workload on the
warfighter is increasing. A novel, stereo-based 3D display has been developed to aid the warfighter
by displaying information in a more intuitive fashion by exploiting depth perception. The flat panel
display has a footprint consistent with current and future vehicles, unmanned systems, and aircraft
and is capable of displaying analog 3D video and OpenGL 3D imagery. A description of the display
will be given along with discussion of the applications evaluated to date.
A flat panel stereoscopic display has been developed and tested for application in unmanned ground systems.
The flat panel display has a footprint that is only slightly thicker than the same size LCD display and has been
installed in the lid of a TALON OCU. The approach uses stacked LCD displays and produces live stereo
video with passive polarized glasses but no spatial or temporal multiplexing. The analog display, which is
available in sizes from 6.4" diagonal to 17" diagonal, produces 640 × 480 stereo imagery. A comparison of
soldiers' performance using 3D vs. 2D using live stereo video will be given. A description of the display will
be given along with discussion of the testing.
In this paper, we report on the development of a 3D vision system consisting of a flat panel stereoscopic display and auto-converging stereo camera and an assessment of the system's use for robotic driving, manipulation, and surveillance operations. The 3D vision system was integrated onto a Talon Robot and Operator Control Unit (OCU) such that direct comparisons of the performance of a number of test subjects using 2D and 3D vision systems were possible. A number of representative scenarios were developed to determine which tasks benefited most from the added depth perception and to understand when the 3D vision system hindered understanding of the scene. Two tests were conducted at Fort Leonard Wood, MO with noncommissioned officers ranked Staff Sergeant and Sergeant First Class. The scenarios; the test planning, approach and protocols; the data
analysis; and the resulting performance assessment of the 3D vision system are reported.
Polaris Sensor Technologies, Inc. is identifying target pixels in IR imagery at signal to noise (SNR) ranges from 1.25 to
3 with a mixed set of algorithms that are candidates for next generation focal planes. Some of these yield less than 50
false targets and a 95% probability of detection in this low SNR range. What has been discovered is that single frame
imagery combined with IMU data can be input into a host of algorithms like Neural Networks and filters to isolate
signals and cull noise. Solutions for nonlinear thresholding approaches can be solved using both genetic algorithms and
neural networks. What is being addressed is how to implement these approaches and apply them to point target
detection scenarios. The large format focal planes will flood the down stream image processing pipelines used in real
time systems, and this team wonders if data can be thinned near the FPA using one of these techniques. Delivering all
the target pixels with a minimum of false positives is the goal addressed by the group. Algorithms that can be digitally
implemented in a ROIC are discussed as are the performance statistics Probability of Detection and False Alarm Rate.
Results from multiple focal planes for varied scenarios will be presented.
Using matched filters to find targets in cluttered images is an old idea. Human operators can interactively find threshold
values to be applied to the correlation surface that will do a good job of binarizing it into signal/non-signal pixel regions.
Automating the thresholding process with nine measured image statistics is the goal of this paper. The nine values are
the mean, maximum, and standard deviation of three images: the input image presumed to have some signal, an NxN
matched filter kernel in the shape of the signal, and the correlation surface generated by convolving the input image with
the matched filter kernel. Several thousand input images with known target locations and reference images were run
through a correlator with kernels that resembled the targets. The nine numbers referred to above were calculated in
addition to a threshold found with a time consuming brutal algorithm. Multidimensional radial basis functions were
associated with each nine number set. The bump height corresponded to the threshold value. The bump location was
within a nine dimensional hypercube corresponding to the nine numbers scaled so that all the data fell within the interval
0 to 1 on each axis. The sigma (sharpness of the radial basis function) was calculated as a fraction of the squared
distance to the closest neighboring bump. A new threshold is calculated as a weighted sum of all the Gaussian bumps in
the vicinity of the input 9D vector. The paper will conclude with a table of results using this method compared to other
Pulse Coupled Neural Networks (PCNNs) have been shown to be of value in image processing applications, especially
at identifying features of small spatial extent at low signal to noise ratio. In our use of the PCNN, every pixel in a scene
feeds a neuron in a fully connected lateral neural network. Nearest neighbor neurons contribute to the output of any
given neuron using weights that link the neuron and its neighborhood in both a linear and a non-linear fashion. The
network is pulsed, and the output of the network at each pulse is a binary mask of neurons that are active. Pulsing drives
the network to evaluate its state. The multi-dimensionality and the non-linear nature of the network make selecting
weights using trial and error a non-trivial problem. It is important that the desired features of the input are identified on
a predictable pulse, a problem that has yet to be sufficiently addressed by proponents of the PCNN. Our method to
overcome these problems is to use a Genetic Algorithm to select the set of PCNN coefficients which will identify the
pixels of interest on a predetermined pulse. This method enables PCNNs to be trained, which is a novel capability and
renders the method of use for applications.