Simulation of robots in a virtual domain has multiple benefits. End users can use the simulation as a training tool to
increase their skill with the vehicle without risking damage to the robot or surrounding environment. Simulation allows
researchers and developers to benchmark robot performance in a range of scenarios without having the physical robot or
environment present. The simulation can also help guide and generate new design concepts. USARSim (Unified
System for Automation and Robot Simulation) is a tool that is being used to accomplish these goals, particularly within
the realm of search and rescue. It is based on the Unreal Tournament 2004 gaming engine, which approximates the
physics of how a robot interacts with its environment. A family of vehicles that can benefit from simulation in
USARSim are Whegs<sup>TM</sup> robots. Developed in the Biorobotics Laboratory at Case Western Reserve University,
Whegs<sup>TM</sup> robots are highly mobile ground vehicles that use abstracted biological principles to achieve a robust level of
locomotion, including passive gait adaptation and enhanced climbing abilities. This paper describes a Whegs<sup>TM</sup> robot
model that was constructed in USARSim. The model was configured with the same kinds of behavioral characteristics
found in real Whegs<sup>TM</sup> vehicles. Once these traits were implemented, a validation study was performed using identical
performance metrics measured on both the virtual and real vehicles to quantify vehicle performance and to ensure that
the virtual robot's performance matched that of the real robot.
The Autonomy Levels for Unmanned Systems (ALFUS) workshop series was convened to address the
autonomous nature of unmanned, robotic systems, or unmanned systems (UMS). Practitioners have
different perceptions or different expectations for these systems. The requirements on human interactions,
the types of tasks, the teaming of the UMSs and the humans, and the operating environment are just a few
of the issues that need to be clarified. Also needed is a set of definitions and a model with which the
autonomous capability of the UMS can be described. This paper reports the current results and status of the
ALFUS framework, which practitioners can apply to analyze the autonomy requirements and to evaluate
the performance of their robotic programs.
In this paper, we describe work in performance standards for urban search and rescue (USAR) robots, begun in 2004 by
the Department of Homeland Security. This program is being coordinated by the National Institute of Standards and
Technology and will result in consensus standards developed through ASTM International, under the Operational
Equipment Subcommittee of their Homeland Security Committee. A comprehensive approach to performance
requirements and standards development is being used in this project. Formal test methods designed by several
working groups in the standards task group are validated by the stakeholders. These tests are complemented by regular
exercises in which responders and robot manufacturers work together to apply robots within realistic training scenarios.
This paper recaps the most recent exercise, held at the Federal Emergency Management Agency (FEMA) Maryland Task
Force 1 training facility, at which over twenty different robots were operated by responders from various FEMA Task
Forces. The exercise included candidate standard test methods being developed for requirements in the areas of
communications, mobility, sensors, and human-system interaction for USAR robots.
Urban Search and Rescue Simulation (USARSim) is an open source package that provides a high-resolution,
physics based simulation of robotic platforms. The package provides models of several common robotic platforms
and sensors as well as sample worlds and a socket interface into a commonly used commercial-off-the-shelf (COTS)
simulation package. Initially introduced to support the development of search and rescue robots, USARSim has
proved to be a tool with a broader scope, from robot education to human robot interfaces, including cooperation,
and more. During Robocup 2006, a new competition based on USARSim will be held in the context of the urban
search and rescue competitions.
The Mobility Open Architecture Simulation and Tools (MOAST) is a framework that builds upon the 4-D
Real-time Control Systesm (4D/RCS) architecture to analyze the performance of autonomous vehicles and multiagent
systems. MOAST provides controlled environments that allow for the transparent transference of data
between a matrix of real and virtual components. This framework is glued together through well-defined interfaces
and communications protocols, and detailed specifications on individual subsystem input/output (IO). This
allows developers to freely swap components and analyze the effect on the overall system by means of comparison
to baseline systems with a limited set of functionality. When taken together, the combined USARSim/MOAST
system may be used to provide a comprehensive development and testing environment for complex robotic
This paper will provide an overview of each system and describe how the combined system may be used for
stand-alone simulated development and test, or hardware-in-the-loop development and testing of autonomous
mobile robot systems.
In this paper, we describe work in performance standards for urban search and rescue (USAR) robots begun in 2004 by the Department of Homeland Security. This program is being coordinated by the National Institute of Standards and Technology and will result in consensus standards developed through ASTM International, under the Operational Equipment Subcommittee of their Homeland Security Committee. The first phase of the program involved definition of requirements by subject matter experts. Responders participated in a series of workshops to identify deployment categories for robots, performance categories, and ranges of acceptable or target performance in the various categories. Over one hundred individual requirements were identified, within main categories such as Human-System Interaction, Logistics, Operating Environment, and System (which includes Chassis, Communications, Mobility, Payload, Power, and Sensing). To ensure that the robot developers and eventual end users work closely together, "responders meet robots" events at situationally relevant sites are being held to refine and extend the performance requirements and develop standard test methods. The results of these standard performance tests will be captured in a compendium of existing and developmental robots with classifications and descriptors to differentiate particular robotic capabilities. This, along with ongoing efforts to categorize situational USAR constraints such as building collapse types or the presence of hazardous materials, will help responders match particular robotic capabilities to response needs. In general, these efforts will enable responders to effectively use robotic tools to enhance their effectiveness while reducing risk to personnel during disasters.
The initial construct of the framework for the Autonomy Levels of Unmanned Systems (ALFUS) was presented in the 2004 SPIE Defense and Security Symposium. This paper describes the continuing development effort and further accomplishments made by the Ad Hoc working group. We focus on two elements of the ALFUS product set, namely, the detailed model that is being implemented as a spreadsheet-based tool and the summary model. We also discuss identified challenges.
This paper will describe how the Mobility Open Architecture Tools and Simulation (MOAST) framework can facilitate performance evaluations of RCS compliant multi-vehicle autonomous systems. This framework provides an environment that allows for simulated and real architectural components to function seamlessly together. By providing repeatable environmental conditions, this framework allows for the development of individual components as well as component performance metrics. MOAST is composed of high-fidelity and low-fidelity simulation systems, a detailed model of real-world terrain, actual hardware components, a central knowledge repository, and architectural glue to tie all of the components together. This paper will describe the framework’s components in detail and provide an example that illustrates how the framework can be utilized to develop and evaluate a single architectural component through the use of repeatable trials and experimentation that includes both virtual and real components functioning together
This paper describes NIST’s efforts in evaluating what it will take to achieve autonomous human-level driving skills in terms of time and funding. NIST has approached this problem from several perspectives: considering the current state-of-the-art in autonomous navigation and extrapolating from there, decomposing the tasks identified by the Department of Transportation for on-road driving and comparing that with accomplishments to date, analyzing computing power requirements by comparison with the human brain, and conducting a Delphi Forecast using the expert researchers in the field of autonomous driving. A detailed description of each of these approaches is provided along with the major finding from each approach and an overall picture of what it will take to achieve human level driving skills in autonomous vehicles.
This paper presents a cost-based adaptive planning agent that is operating at the route-segment level of a deliberative hierarchical planning system for autonomous road driving. At this level, the planning agent is responsible for developing fundamental driving maneuvers that allow a vehicle to travel safely amongst moving and stationary objects. This is facilitated through the use of an incrementally expanded planning graph that provides the ability to implement a dynamic cost function. This cost function varies to comply with particular road, regional, or event driven situations, and when coupled with the incremental graph expansion allows for the agent to implement hard and soft system constraints.
The viability of Unmanned Systems as tools is increasingly recognized in many domains. As technology advances, the autonomy on board these systems also advances. In order to evaluate the systems in terms of their levels of autonomy, it is critical to have a set of standard definitions that support a set of metrics. As autonomy cannot be evaluated quantitatively without sound and thorough technical basis, the development of autonomy levels for unmanned systems must take into account many factors such as task complexity, human interaction, and environmental difficulty. An <i>ad hoc</i> working group assembled by government practitioners has been formed to address these issues. The ultimate objectives for the working group are: (1) To determine the requirements for metrics for autonomy levels of unmanned systems. (2) To devise methods for establishing metrics of autonomy for unmanned systems. (3) To develop a set of widely recognized standard definitions for the levels of autonomy for unmanned systems. This paper describes the interim results that the group has accomplished through the first four workshops that the group held. We report on the initial findings of the workshops toward developing a generic framework for the Autonomy Levels for Unmanned Systems (ALFUS).
Urban search and rescue (USAR) is one of the most dangerous and time-critical non-wartime activities. Researchers have been developing hardware and software to enable robots to perform some search and rescue functions so as to minimize the exposure of human rescue personnel to danger and maximize the survival of victims. Significant progress has been achieved, but much work remains. USAR demands a blending of numerous specialized technologies. An effective USAR robot must be endowed with key competencies, such as being able to negotiate collapsed structures, find victims and assess their condition, identify potential hazards, generate maps of the structure and victim locations, and communicate with rescue personnel. These competencies bring to bear work in numerous sub-disciplines of intelligent systems (or artificial intelligence) such as sensory processing, world modeling, behavior generation, path planning, and human-robot interaction, in addition to work in communications, mechanism design and advanced sensors. In an attempt to stimulate progress in the field, reference USAR challenges are being developed and propagated worldwide. In order to make efficient use of finite research resources, the robotic USAR community must share a common understanding of what is required, technologically, to attain each competency, and have a rigorous measure of the current level of effectiveness of various technologies. NIST is working with partner organizations to measure the performance of robotic USAR competencies and technologies. In this paper, we describe the reference test arenas for USAR robots, assess the current challenges within the field, and discuss experiences thus far in the testing effort.
This paper describes a world model that combines a variety of sensed inputs and a priori information and is used to generate on-road and off-road autonomous driving behaviors. The system is designed in accordance with the principles of the 4D/RCS architecture. The world model is hierarchical, with the resolution and scope at each level designed to minimize computational resource requirements and to support planning functions for that level of the control hierarchy. The sensory processing system that populates the world model fuses inputs from multiple sensors and extracts feature information, such as terrain elevation, cover, road edges, and obstacles. Feature information from digital maps, such as road networks, elevation, and hydrology, is also incorporated into this rich world model. The various features are maintained in different layers that are registered together to provide maximum flexibility in generation of vehicle plans depending on mission requirements. The paper includes discussion of how the maps are built and how the objects and features of the world are represented. Functions for maintaining the world model are discussed. The world model described herein is being developed for the Army Research Laboratory's Demo III Autonomous Scout Vehicle experiment.
One approach to measuring the performance of intelligent systems is to develop standardized or reproducible tests. These tests may be in a simulated environment or in a physical test course. The National Institute of Standards and Technology has developed a test course for evaluating the performance of mobile autonomous robots operating in an urban search and rescue mission. The test course is designed to simulate a collapsed building structure at various levels of fidelity. The course will be used in robotic competitions, such as the American Association for Artificial Intelligence (AAAI) Mobile Robot Competition and the RoboCup Rescue. Designed to be repeatable and highly reconfigurable, the test course challenges a robot's cognitive capabilities such as perception, knowledge representation, planning, autonomy and collaboration. The goal of the test course is to help define useful performance metrics for autonomous mobile robots which, if widely accepted, could accelerate development of advanced robotic capabilities by promoting the re-use of algorithms and system components. The course may also serve as a prototype for further development of performance testing environments which enable robot developers and purchasers to objectively evaluate robots for a particular application. In this paper we discuss performance metrics for autonomous mobile robots, the use of representative urban search and rescue scenarios as a challenge domain, and the design criteria for the test course.
This paper outlines the goals and work accomplished thus far for both the man-machine interface and mission planning elements of the experimental unmanned vehicle program. It is the gaol of the XUV program to make available to the user an interface and tools that will allow for seamless transition between mission planning, rehearsal, and execution on multiple collaborating autonomous vehicles in a platoon group.