The study presented in this paper tests the hypothesis that the combination of a global similarity transformation and local free form deformations can be used for the accurate segmentation of internal structures in MR images of the brain. To quantitatively evaluate our approach, the entire brain, the cerebellum and the head of the caudate have been segmented manually on one of the volumes and mapped back onto all the other volumes using the computed transformations. The contours so obtained have been compared to contours drawn manually around the structures of interest in each individual brain. Manual delineation was repeated to test intra-rater variability. Contours were quantitatively compared using a similarity index defined as 2 times the area encircled by both contours divided by the sum of the areas encircled by each contour. This index ranges from 0 to 1 with zero indicating zero overlap and one indicating a perfect agreement between two contours. It is sensitive to both displacement and differences in shape and it is thus preferable to a simple area comparison. Results indicate that the method we proposed can be used to segment accurately and fully automatically large and small structures in high resolution 3D images of the brain. The average similarity indices between the manual and automatic segmentations are 0.96, 0.97, and 0.845 for the whole head, the cerebellum, and the head of the caudate respectively. These numbers are 0.97, 0.97, and 0.88 when two manual delineations are compared. Statistical analysis reveals that the differences in mean similarity indices between the two manual delineations and between the manual delineations and the automatic segmentation method are statistically significant for the whole head and the caudate but not for the cerebellum. It is shown, however, that similarity indices in the range of 0.85 correspond to contours that are virtually undistinguishable.