Maximum auto-mutual-information analysis
Allan A. Nielsen
Technical University of Denmark
DTU Compute - Applied Mathematics and Computer Science
DK-2800 Kgs. Lyngby, Denmark
Based on the information theoretical measure mutual information derived from entropy and Kullback-Leibler divergence, an alternative to maximum autocorrelation factor analysis is sketched.
In signal and image processing principal component analysis  (PCA) is often used for dimensionality reduction and feature extraction in pre-processing steps to for example classication.
In remote sensing image analysis PCA is often replaced by maximum autocorrelation factor  (MAF) or minimum noise fraction  (MNF) analysis. This is done because MAF and MNF analyses incorporate spatial information in the orthogonalization of the multivariate data which is conceptually more satisfactory and which typically gives better results.
In this contribution, autocorrelation between the multivariate data and a spatially shifted version of the same data in the MAF analysis is replaced by the information theoretical, entropy and Kullback-Leibler divergence based measure mutual information [4-8]. This potentially gives a more detailed decomposition of the data. Also, the orthogonality between already found components and components of higher order requested in the MAF analysis is replaced by a requirement of minimum mutual information between components. These ideas resulting in maximum auto-mutual-information analysis are in turn based on [9-11]. The sketched methods are used on the well-known AVIRIS  (https://aviris.jpl.nasa.gov/) Indian Pines data.
 Hotelling, H., “Analysis of a complex of statistical variables into principal components," Journal of Educational Psychology 24, 417-441 (1933).
 Switzer, P. and Green, A. A., “Min/max autocorrelation factors for multivariate spatial imagery," Tech. Rep. 6, Department of Statistics, Stanford University (1984).
 Green, A. A., Berman, M., Switzer, P., and Craig, M. D., “A transformation for ordering multispectral data in terms of image quality with implications for noise removal," IEEE Transactions on Geoscience and Remote Sensing 26, 65-74 (Jan. 1988).
 Shannon, C. E., “A mathematical theory of communication," Bell System Technical Journal 27(3), 379-423 and 623-656 (1948).
 Hyvärinen, A., Karhunen, J., and Oja, E., Independent Component Analysis, J. Wiley (2001).
 Mackay, D. J. C., Information Theory, Inference and Learning Algorithms, Cambridge University Press (2003).
 Bishop, C. M., Pattern Recognition and Machine Learning, Springer (2007).
 Canty, M. J., Image Analysis, Classication and Change Detection in Remote Sensing. With Algorithms for ENVI/IDL and Python, Taylor & Francis, CRC Press, third ed. (2014).
 Yin, X., “Canonical correlation analysis based on information theory," Journal of Multivariate Analysis 91, 161-176 (2004).
 Karasuyama, M. and Sugiyama, M., “Canonical dependency analysis based on squared-loss mutual information," Neural Networks 34, 46-55 (2012).
 Vestergaard, J. S. and Nielsen, A. A., “Canonical information analysis," ISPRS Journal of Photogrammetry and Remote Sensing 101, 1-9 (2015). http://www.imm.dtu.dk/pubdb/p.php?6270, Matlab code at https://github.com/schackv/cia.
 Vane, G., Green, R. O., Chrien, T. G., Enmark, H. T., Hansen, E. G., and Porter, W. M., “The airborne/infrared imaging spectrometer (AVIRIS)," Remote Sensing of Environment 44, 127-143 (1993).
Allan A. Nielsen, "Maximum auto-mutual-information factor analysis (Conference Presentation)," Proc. SPIE 10427, Image and Signal Processing for Remote Sensing XXIII, 104270J (Presented at SPIE Remote Sensing: September 12, 2017; Published: 19 October 2017); https://doi.org/10.1117/12.2278290.5618090324001.
Conference Presentations are recordings of oral presentations given at SPIE conferences and published as part of the conference proceedings. They include the speaker's narration along with a video recording of the presentation slides and animations. Many conference presentations also include full-text papers. Search and browse our growing collection of more than 12,000 conference presentations, including many plenary and keynote presentations.