Improving learning speed in multilayer perceptrons through principal component analysis

Francesco Masulli; Massimo Penna

doi:10.1117/12.235905

22 March 1996 Improving learning speed in multilayer perceptrons through principal component analysis

Francesco Masulli, Massimo Penna

Proceedings Volume 2760, Applications and Science of Artificial Neural Networks II; (1996) https://doi.org/10.1117/12.235905
Event: Aerospace/Defense Sensing and Controls, 1996, Orlando, FL, United States

Abstract

This paper describes an application of Principal Component Analysis to the speeding-up the learning of a Multi-Layer Perceptron (MLP). A training algorithm, called the Incremental Input Dimensionality (IID) method, is presented that is constituted by some training steps, in each of which the dimension of the principal component subspace is increased. For each training step, some training epochs (presentations of the training set), using the Back- Propagation algorithm, are performed in order to reduce the mean square error on the test set. In this way, the last training step is performed with a subspace corresponding to the assigned reconstruction error. The performance of the MLP using IID, in the case of handwritten digit classification, are reported. For our data-base a choice of a reconstruction error rate of 5% in the IID algorithm implies a maximum dimension of the principal components subspace equal to 37. In the experiments reported in this paper, Back-Propagation using IID has turned out to be faster than standard Back-Propagation with a speed-up of about 73%. Moreover, as the IID method concerns only data representation, it can be combined with other speed-up techniques for MLP learning, and can be used by other classifiers.

Citation Download Citation

Francesco Masulli and Massimo Penna "Improving learning speed in multilayer perceptrons through principal component analysis", Proc. SPIE 2760, Applications and Science of Artificial Neural Networks II, (22 March 1996); https://doi.org/10.1117/12.235905

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available