A general fine-tune method for catastrophic forgetting

Yang Tao; Mingming Zhu; Hao Li; Cao Yuan

doi:10.1117/12.2540725

14 February 2020 A general fine-tune method for catastrophic forgetting

Yang Tao, Mingming Zhu, Hao Li, Cao Yuan

Proceedings Volume 11429, MIPPR 2019: Automatic Target Recognition and Navigation; 1142912 (2020) https://doi.org/10.1117/12.2540725
Event: Eleventh International Symposium on Multispectral Image Processing and Pattern Recognition (MIPPR2019), 2019, Wuhan, China

Abstract

When the model begins a new task, the challenge of naming the "catastrophic forgetting" limits the scalability of the deep learning network, which quickly forgets the learning capabilities it has. The fine-tuning method recommends that the original feature extraction be retained to extract the features of the new task and to achieve the purpose of learning the new class. However, this method degrades performance on previously learned tasks because the shared parameters change without new guidance for the original task-specific prediction parameters. This paper proposes general fine-tune method to reduce catastrophic forgetting in sequential task learning scenarios. The critical idea of the method is fine-tuning the parameters in each layer, unlike the traditional fine tuning only for the last layer. The experimental results show that the new method is superior to fine-tune, in the accuracy of the old task and the performance of the new task is better than that of the EWC. A distinct advantage is that old tasks do not limit the performance of new tasks but provide some support for new tasks.

Citation Download Citation

Yang Tao, Mingming Zhu, Hao Li, and Cao Yuan "A general fine-tune method for catastrophic forgetting", Proc. SPIE 11429, MIPPR 2019: Automatic Target Recognition and Navigation, 1142912 (14 February 2020); https://doi.org/10.1117/12.2540725

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Data modeling

Neurons

Neural networks

Feature extraction

Sensors

Computer science

Mathematics

Show All Keywords

Keywords/Phrases

Search In:

Publication Years