The goal of this book is to present a unified information-theoretic approach to solve estimation problems commonly faced in signal-processing applications. We provide both new approaches to this problem as well as new interpretations for existing techniques. Major applications of this work include image restoration, communication channel estimation, text restoration, and system modeling. A general approach to solving a number of detection and estimation problems utilizing concepts from information theory is developed. The theoretical development of the approach as well as important applications is given. Additionally, the work of other researchers is shown to be special cases of this general technique.

Decision making in the context of detection and estimation problems requires assumptions about the probability model. The requirement for assumptions derives from the lack of sufficient information to arrive at the "right" solution. By making appropriate assumptions one can choose a single best estimate of the "right" solution from the available set of candidate solutions. In other words, these assumptions fill the gap, or resolve the uncertainty, that is left by the unavailability of the required information.

Information theory offers a powerful approach to the problem of assigning probability models. In Part II, we examine the use of information theory for probability modeling and establish estimates of the error bounds on the resulting models. In Part III, we consider the application of information-theoretic functional to the problem of estimation. Specifically, this research concerns the application of information-theoretic concepts to problems of deconvolution and system modeling. We develop a general iterative mapping derived using the mutual information functional to establish a criterion for optimality in solving problems of signal and image restoration and resolution enhancement. The mathematical structure is defined and the convergence of the mapping is established both in the general formulation and for particular examples. Some popular techniques for signal restoration, such as that proposed by Van Cittert and derived independently of information theory, are shown to be special cases of this general approach. Additionally, the approach allows the inclusion of prior partial knowledge of the desired solution and testing of various hypotheses concerning the solution characteristics. The work herein offers both new approaches to these problems as well as a theoretically sound general framework unifying the work of previous researchers.

Joseph Noonan and Prabahan Basu

Medford, MA

December 2011