In this paper, we propose a dynamic approach to feature and classifier selection. In our approach, based on performance, visual features and classifiers are selected automatically. In earlier work, we presented the Visual Apprentice, in which users can define visual object models via a multiple- level object definition hierarchy. Visual Object Detectors are learned, using various learning algorithms - as the user provides examples from images or video, visual features are extracted and multiple classifiers are learned for each node of the hierarchy. In this paper, features and classifiers are selected automatically at each node, depending on their performance over the training set introduce the concept of Recurrent Visual Semantics and show how it can be used to identify domains in which performance-based learning techniques such as the one presented can be applied. We then show experimental results in detecting Baseball video shots, images that contain handshakes,and images that contain skies. These result demonstrate the importance, feasibility, and usefulness of dynamic feature/classifier selection for classification of visual information, and the performance benefits of using multiple learning algorithms to build classifiers. Based on our experiments, we also discuss some of the issues that arise when applying learning techniques in real-world content-based applications.