|
A general approach to the problem of reconstructing a non-negative function from finitely many linear functional values is to select from among all data consistent functions that one closest to a prior estimate, according to some measure of directed distance. Minimizing the cross entropy (MCE), or more generally, minimizing the Kullback-Leibler distortion, are examples of this approach. Shore and Johnson have characterized the MCE approach for the reconstruction of probability densities in terms of axioms of probabilistic inference. Many of the applications of these methods involve the reconstruction of functions that are not essentially probabilistic in nature, such as energy distributions in space, x-ray attenuation functions, and so on. The properties of the reconstructions, as approximations of the true solution, are not easily determined from these axioms. Because the basic problem is how to approximate one function by others, we adopt axioms that govern the approximation theoretic behavior of the directed distances. The first three axioms merely impose reasonable behavior on the directed distances. The fourth, directed orthogonality, characterizes a wide class of directed distances, including the Kullback-Leibler and the Itakura-S alto distortion measures. The addition of a fifth axiom, invariance to scale, uniquely characterizes the Kuilback-Leibler distortion, which reduces to MCE (for probability densities) when the prior estimate is constant.
|
