We present a new method for evaluating the performance of nonrigid image registration algorithms by analyzing the invertibility and transitivity properties of the transformations that they produce. The invertibility and transitivity of transformations computed using a unidirectional and a consistent linear-elastic registration algorithm are evaluated. The invertibility of the transformations is evaluated by comparing the composition of transformations from images A to B and B to A to the identity mapping. The transitivity of the transformations is evaluated by measuring the difference between the identity mapping and the composition of the transformations from images A to B, B to C, and C to A. Transformations are generated by matching three computer-generated phantoms, three computed tomography (CT) data of infant heads, and 23 magnetic resonance imaging (MRI) data of adult brains. In all cases, the inverse consistency constraint (ICC) algorithm out-performs the unidirectional algorithm by producing transformations that have less inverse consistency error and less transitivity error. For the MRI brain data, the ICC algorithm reduced the maximum inverse consistency error by 205 times, the average transitivity error by 50%, and the maximum transitivity error by 37% on average compared to the unidirectional algorithm.