This paper presents a comprehensive approach to the design of machine vision software for roving robots and autonomous vehicles. Various techniques are proposed for solving the important problems of directional guidance, obstacle avoidance, and object identification. Artificial intelligence and knowledge-base concepts form the basis of the vision system design. The principle of texture invariance is introduced for shadow analysis and discrimination. The idea of scene layout footprints and 3-D maps for landmarks is proposed as a means of orientation determination for the guidance and navigation of roving robots and autonomous vehicles. The vision system performs three phases of visual processing: the initialization phase, the "walking" phase, and the warning phase. The visual processing and interpretation are monitored by the knowledge access and inference routine.