As high-resolution conservation images, acquired using various imaging modalities, become more widely available, it is
increasingly important to achieve accurate registration between the images. Accurate registration allows information
unavailable in any one image to be compiled from several images and then used to provide a better understanding of
how a painting was constructed. We have developed an algorithm that solves several important conservation problems:
1) registration and mosaicking of multiple X-ray films, ultraviolet images, and infrared reflectograms to a color
reference image at high spatial-resolution (200 to 500 dpi) of paintings (both panel and canvas) and of works on paper,
2) registration of the images within visible and infrared multispectral reflectance and luminescence image cubes, and 3)
mosaicking of hyperspectral image cubes (400 to 2500 nm).
The registration/mosaicking algorithm corrects for several kinds of distortion, small rotation and scale errors, and
keystone effects between the images. Thus images acquired with different cameras, illumination, and geometries can be
registered/mosaicked. This automatic algorithm for registering/mosaicking multimodal conservation images is expected
to be a valuable tool for conservators attempting to answer questions regarding the creation and preservation history of
paintings. For example, an analysis of the reflectance spectra obtained from the sub-pixel registered multispectral image
cubes can be used to separate, map, and identify artist materials in situ. And, by comparing the corresponding images in
the X-ray, visible, and infrared regions, conservators can obtain a deeper understanding of compositional changes.
To improve the spatial sampling of scanning hyperspectral cameras, it is often necessary to capture numerous overlapping image cubes and later mosaic them to form the overall image cube. For hyperspectral camera systems having broad-area coverage, whisk-broom scanning using an external mirror is often employed. Creating the final image cube mosaic requires sub-pixel correction of the scan-mirror distortion, as well as alignment of the individual image cubes. For systems lacking geo-positional information that relates sensor to scene, alignment of the image scans is nontrivial. Here we present a novel algorithm that removes scan distortion and aligns hyperspectral image cubes based on correlation of the cubes’ image content with a reference image.
The algorithm is able to provide robust results by recognizing that the cubes’ image content will not always match identically with that of the reference image. For example, in cultural heritage applications, the reference color image of the finished painting need not match the under-painting seen in the SWIR. Our approach is to identify a corresponding set of points between the cubes and the reference image, using a subset of wavelet scales, and then filtering out matches that are inconsistent with a map of the distortion. The filtering is performed by removing points iteratively according to their proximity to a function fit to their disparity (distance between the matched points). Our method will be demonstrated and our results validated using hyperspectral image cubes (976-1680 nm) and visible reference images from the fields of remote sensing and cultural heritage preservation.
We present an algorithm for automatically selecting and matching control points for the purpose of registering
images acquired using different imaging modalities. The modulus maxima of the wavelet transform were used to
define a criterion for identifying control points. This criterion is capable of selecting points based on the size of
features in the image. This technique can be tailored, by adjusting the scale of the filters in the modulus calculation,
to the specific objects or structures known to occur in each image being registered. The control-point matching
technique includes an iterative method for reducing the set of control-point pairs using the horizontal and vertical
disparities between the matched pairs of points. Least-squares planes are fit to the horizontal and vertical disparity
data, and control-point pairings are deleted based on their distances from those planes. The remaining points are
used to recompute the planes. The process is iterated until the remaining points fall within a certain distance from
the planes. Finally, a spatial transformation is performed on the template image to bring it into alignment with the
reference image. The result of the control-point pair reduction is a more accurate alignment than what would have
been produced using the initial control-point pairs. These techniques are applicable to medical images, but
examples are given using images of paintings.
As high-resolution images of paintings, acquired using various imaging modalities (e.g. X-ray, luminescence, visible and
infrared reflection) become more available, it is increasingly useful to have accurate registration between them. Accurate
registration allows new information to be compiled from the several multimodal images. This leads to a better
understanding of how the painting was constructed and of any compositional changes that have occurred. To that end,
we have produced an automatic image registration algorithm that is capable of aligning X-ray, color, and infrared
images, as well as multispectral luminescence and reflectance image sets, or cubes. The key steps of the algorithm
include identifying large sets of candidate control points in the reference image, then pairing them with potential points
in a second image using cross-correlation. Finally, after selecting the best set of control point pairs, the second image is
transformed to be in register with the reference image. Tests show the algorithm to be capable of achieving sub-pixel
registration across these various image modalities.