The design and development of decision systems capable of adaptively learning in the operational environment is presented. Innovative adaptive learning concepts and methodologies are offered that are designed for enhancing the performance of decision systems, such as automatic target recognition systems, wherein robustness of performance is a significant issue. The fundamental concept underlying this design is that of learning in partially exposed environments, wherein, at the start, the system is not necessarily aware of all the pattern classes that may be encountered in the future phase of operations. The decision system is based on a variant to the widely popular nearest-neighbor concept. Several stages of sophistication of the system design are presented. The potential problem of increase in computational loads is addressed in detail by exploring the benefits of employing the recently proposed concept of minimal consistent set. The effectiveness of the system design is experimentally illustrated using two data sets, the now classical IRIS data and some real-world TV image data.