An important class of robotic applications potentially involves multiple, cooperating robots: security or military surveillance, rescue, mining, etc. One of the main challenges in this area is effective cooperative control: how does one determine and orchestrate individual robot behaviors which result in a desired group behavior? Cognitive (planning) approaches allow for explicit coordination between robots, but suffer from high computational demands and a need for a priori, detailed world models. Purely reactive approaches such as that of Brooks are efficient, but lack a mechanism for global control and learning. Neither approach by itself provides a formalism capable of a sufficiently rapid and rich range of cooperative behaviors. Although we accept the usefulness of the reactive paradigm in building up complex behaviors from simple ones, we seek to extend and modify it in several ways. First, rather than restricting primitive behaviors to fixed input-output relationships, we include memory and learning through feedback adaptation of behaviors. Second, rather than a fixed priority of behaviors, our priorities are implicit: they vary depending on environmental stimuli. Finally, we scale this modified reactive architecture to apply not only for an individual robot, but also at the level of multiple cooperating robots: at this level, individual robots are like individual behaviors which combine to achieve a desired aggregate behavior. In this paper, we describe our proposed architecture and its current implementation. The application of particular interest to us is the control of a team of mobile robots cooperating to perform area surveillance and target acquisition and tracking.