Man-portable Unmanned Ground Vehicles (UGVs) have been fielded on the battlefield with limited computing power.
This limitation constrains their use primarily to teleoperation control mode for clearing areas and bomb defusing. In
order to extend their capability to include the reconnaissance and surveillance missions of dismounted soldiers, a
separate processing payload is desired. This paper presents a processing architecture and the design details on the
payload module that enables the PackBot to perform sophisticated, real-time image processing algorithms using data
collected from its onboard imaging sensors including LADAR, IMU, visible, IR, stereo, and the Ladybug spherical
cameras. The entire payload is constructed from currently available Commercial off-the-shelf (COTS) components
including an Intel multi-core CPU and a Nvidia GPU. The result of this work enables a small UGV to perform
computationally expensive image processing tasks that once were only feasible on a large workstation.
Large gains in the automation of human detection and tracking techniques have been made over the past several years.
Several of these techniques have been implemented on larger robotic platforms, in order to increase the situational
awareness provided by the platform. Further integration onto a smaller robotic platform that already has obstacle
detection and avoidance capabilities would allow these algorithms to be utilized in scenarios that are not plausible for
larger platforms, such as entering a building and surveying a room for human occupation with limited operator
However, transitioning these algorithms to a man-portable robot imparts several unique constraints, including limited
power availability, size and weight restrictions, and limited processor ability. Many imaging sensors, processing
hardware, and algorithms fail to adequately address one or more of these constraints.
In this paper, we describe the design of a payload suitable for our chosen man-portable robot, the iRobot Packbot. While
the described payload was built for a Packbot, it was carefully designed in order to be platform agnostic, so that it can be
used on any man-portable robot. Implementations of several existing motion and face detection algorithms that have
been chosen for testing on this payload are also discussed in some detail.
Recently, there has been an increasing interest in using panoramic images in surveillance and target tracking
applications. With the wide availability of off-the-shelf web-based pan-tilt-zoom (PTZ) cameras and the advances of
CPUs and GPUs, object tracking using mosaicked images that cover a scene of 360° in near real-time has become a
reality. This paper presents a system that automatically constructs and maps full view panoramic mosaics to a cube-map
from images captured from an active PTZ camera with 1-25x optical zoom. A hierarchical approach is used in storing
and mosaicking multi-resolution images captured from a PTZ camera. Techniques based on scale-invariant local features
and probabilistic models for verification are used in the mosaicking process. Our algorithm is automatic and robust in
mapping each incoming image to one of the six faces of a cube with no prior knowledge of the scene structure. This
work can be easily integrated to a surveillance system that wishes to track moving objects in its 360° surrounding.