Quadratic mutual information for dimensionality reduction and classification

David M. Gray; José C. Principe

doi:10.1117/12.852760

13 May 2010 Quadratic mutual information for dimensionality reduction and classification

David M. Gray, José C. Principe

Proceedings Volume 7696, Automatic Target Recognition XX; Acquisition, Tracking, Pointing, and Laser Systems Technologies XXIV; and Optical Pattern Recognition XXI; 76960D (2010) https://doi.org/10.1117/12.852760
Event: SPIE Defense, Security, and Sensing, 2010, Orlando, Florida, United States

Abstract

A research area based on the application of information theory to machine learning has attracted considerable interest in the last few years. This research area has been coined information-theoretic learning within the community. In this paper we apply elements of information-theoretic learning to the problem of automatic target recognition (ATR). A number of researchers have previously shown the benefits of designing classifiers based on maximizing the mutual information between the class data and the class labels. Following prior research in information-theoretic learning, in the current results we show that quadratic mutual information, derived using a special case of the more general Renyi's entropy, can be used for classifier design. In this implementation, a simple subspace projection classifier is formulated to find the optimal projection weights such that the quadratic mutual information between the class data and the class labels is maximized. This subspace projection accomplishes a dimensionality reduction of the raw data set wherein information about the class membership is retained while irrelevant information is discarded. A subspace projection based on this criterion preserves as much class discriminability as possible within the subspace. For this paper, laser radar images are used to demonstrate the results. Classification performance against this data set is compared for a gradient descent MLP classifier and a quadratic mutual information MLP classifier.

Citation Download Citation

David M. Gray and José C. Principe "Quadratic mutual information for dimensionality reduction and classification", Proc. SPIE 7696, Automatic Target Recognition XX; Acquisition, Tracking, Pointing, and Laser Systems Technologies XXIV; and Optical Pattern Recognition XXI, 76960D (13 May 2010); https://doi.org/10.1117/12.852760

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available