In global classifications using Markov Random Field (MRF) modelling, the neighbourhood form is generally considered as independent of its location in the image. Such an approach may lead to classification errors for pixels located at the segment borders. The solution proposed here consists in relaxing the assumption of fixed-form neighbourhood. However this non-stationary neighbourhood modelling is useful only if an efficient heuristic can be defined to perform the optimization. Ant colony optimization (ACO) is currently a popular algorithm. It models upon the behavior of social insects for computing strategies: the information gathered by simple autonomous mobile agents, called ants, is shared and exploited for problem solving. Here we propose to use the ACO and to exploit its ability of self-organization. The ants collect information through the image, from one pixel to the others. The choice of the path is a function of the pixel label, favouring paths within a same image segment. We show that this corresponds to an automatic adaptation of the neighbourhood to the segment form. Performance of this new approach is illustrated on a simulated image and on actual remote sensing images, SPOT4/HRV, representing agricultural areas. In the studied examples, we found that it outperforms the fixed-form neighbourhood used in classical MRF classifications. The advantage of having a neighborhood shape that automatically adapts to the image segment clearly appears in these cases of images containing fine elements, lanes or thin fields, but also complex natural landscape structures.