The Robotics Collaborative Technology Alliances (RCTA) program, which ran from 2001 to 2009, was funded by the
U.S. Army Research Laboratory and managed by General Dynamics Robotic Systems. The alliance brought together a
team of government, industrial, and academic institutions to address research and development required to enable the
deployment of future military unmanned ground vehicle systems ranging in size from man-portables to ground combat
vehicles. Under RCTA, three technology areas critical to the development of future autonomous unmanned systems
were addressed: advanced perception, intelligent control architectures and tactical behaviors, and human-robot
interaction. The Jet Propulsion Laboratory (JPL) participated as a member for the entire program, working four tasks in
the advanced perception technology area: stereo improvements, terrain classification, pedestrian detection in dynamic
environments, and long range terrain classification. Under the stereo task, significant improvements were made to the
quality of stereo range data used as a front end to the other three tasks. Under the terrain classification task, a multi-cue
water detector was developed that fuses cues from color, texture, and stereo range data, and three standalone water
detectors were developed based on sky reflections, object reflections (such as trees), and color variation. In addition, a
multi-sensor mud detector was developed that fuses cues from color stereo and polarization sensors. Under the long
range terrain classification task, a classifier was implemented that uses unsupervised and self-supervised learning of
traversability to extend the classification of terrain over which the vehicle drives to the far-field. Under the pedestrian
detection task, stereo vision was used to identify regions-of-interest in an image, classify those regions based on shape,
and track detected pedestrians in three-dimensional world coordinates. To improve the detectability of partially
occluded pedestrians and reduce pedestrian false alarms, a vehicle detection algorithm was developed. This paper
summarizes JPL's stereo-vision based perception contributions to the RCTA program.
Stereo is a key component in many autonomous navigation tasks. These applications demand real-time performance and consequently, the state-of-the-art uses local correlation-based algorithms that lend themselves to algorithmic and hardware optimization. These systems perform well in simple terrain or on open ground. However, when discrete objects such as trees are present in the scene, correlation-based approaches exhibit inherent difficulties. Some of these difficulties are introduced during the preprocessing stage that attempts to compensate for photometric variations between the cameras. Other difficulties occur during the correlation stage due to occlusion. As a result, object portions appear enlarged, contracted, or missing, as the range data bleeds between the foreground object and the background. This complicates subsequent obstacle detection, representation and modeling. These problems have been addressed by more sophisticated stereo algorithms based on energy minimization and global optimization schemes. Such complex algorithms, however, are computationally demanding and not amenable to real-time implementation. Our solution uses a better preprocessing method, intelligent use of edge cues, and a variation of the traditional shiftable window approach to enhance the stereo correlation at and near depth discontinuities. There is additional computational overhead involved, but we are able to maintain real-time performance. We present details of our new algorithm and several results in complex natural environments.
Binocular, correlation based stereo has been a key component in many efforts at autonomous vehicle navigation. However, estimation of ground truth range data, especially in field conditions, remains a challenge. We present a 5 camera, multibaseline stereo system and demonstrate its use as a passive ground truthing mechanism for binocular stereo. In this paper, we provide both a system description and a detailed overview of a novel depth-based multibaseline stereo algorithm. Our new algorithm avoids the need for pairwise camera rectification. We conclude with several simulations and real world experiments to verify our results.