A feature-based, nonrigid image registration method using a Hausdorff distance-based matching measure is presented. The proposed method is robust to outliers and missing features, as no correspondence is established between the features. Utilizing a B-spline-based, nonrigid deformation model, the proposed method is able to handle nonrigid deformations between the images to be registered. A gradient descent-based optimization procedure is developed to maximize the matching measure under the nonrigid transformation model. With adaptively adjustable step sizes, the optimization procedure works in a coarse-to-fine manner so that first large nonrigid deformations are compensated and then the transformation parameters are gradually refined. In addition, two acceleration techniques are devised to greatly speed up the registration method, making it more practical for real applications. The performance of the proposed method is validated in various experiments from synthetic image registration to hand-drawn Chinese character registration and brain outline registration. The limitation of the method is also analyzed and exemplified. To partly alleviate the limitation of the method, we incorporate landmark information into the method and achieve promising results.