Optimal full motion video (FMV) registration is a crucial need for the Geospatial community. It is required for
subsequent and optimal geopositioning with simultaneous and reliable accuracy prediction. An overall approach being
developed for such registration is presented that models relevant error sources in terms of the expected magnitude and
correlation of sensor errors. The corresponding estimator is selected based on the level of accuracy of the a priori
information of the sensor’s trajectory and attitude (pointing) information, in order to best deal with non-linearity effects.
Estimator choices include near real-time Kalman Filters and batch Weighted Least Squares. Registration solves for
corrections to the sensor a priori information for each frame. It also computes and makes available a posteriori
accuracy information, i.e., the expected magnitude and correlation of sensor registration errors. Both the registered
sensor data and its a posteriori accuracy information are then made available to “down-stream” Multi-Image
Geopositioning (MIG) processes. An object of interest is then measured on the registered frames and a multi-image
optimal solution, including reliable predicted solution accuracy, is then performed for the object’s 3D coordinates. This
paper also describes a robust approach to registration when a priori information of sensor attitude is unavailable. It
makes use of structure-from-motion principles, but does not use standard Computer Vision techniques, such as
estimation of the Essential Matrix which can be very sensitive to noise. The approach used instead is a novel, robust,
direct search-based technique.