Thermal infrared images of the ocean obtained from satellite sensors are widely used for the study of ocean dynamics. The derivation of mesoscale ocean information from satellite data depends to a large extent on the correct interpretation of infrared oceanographic images. The difficulty of the image analysis and understanding problem for oceanographic images is due in large part to the lack of precise mathematical descriptions of the ocean features, coupled with the time varying nature of these features and the complication that the view of the ocean surface is typically obscured by clouds, sometimes almost completely. Towards this objective, the present paper describes a hybrid technique that utilizes a nonlinear probabilistic relaxation method and an expert system for the oceanographic image interpretation problem. A unified mathematical framework that helps in solving the problem is presented. This paper highlights the advantages of using the contextual information in the feature labeling algorithm. The paper emphasizes the need for the feedback from the high level modules to the intermediate modules in an automatic image interpretation system. The paper presents some important results of the series of experiments conducted at Remote Sensing Branch, NORDA, on the NOAA AVHRR imagery data. Key words: feature labeling, feature extraction, oceanic features, edge detection, knowledge based systems, expert system, relaxation, infrared imagery.