Overcome STSF phenomenon in catastrophic forgetting

Yifan Chang; Qifan Zhao

doi:10.1117/12.2674765

23 May 2023 Overcome STSF phenomenon in catastrophic forgetting

Yifan Chang, Qifan Zhao

Proceedings Volume 12604, International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID 2022); 126043Y (2023) https://doi.org/10.1117/12.2674765
Event: 2nd International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID 2022), 2022, Guangzhou, China

Abstract

Catastrophic forgetting is an undesirable phenomenon in convolution neural networks and iCarl is an effective algorithm for preventing catastrophic forgetting. However, a potential defect that more similar tasks result in severer catastrophic forgetting (STSF) in iCarl is explored in this work. The reason of STSF is that similar tasks are prone to occupy similar feature channels and similar feature representations, thus they can be replaced by each other easily. Based on these findings, a novel method Similar Margin Loss (SML) is proposed. SML aims to make feature representations of samples from the same task compact while making feature representations from the different tasks differentiable in the feature space. Experiment results show that SML is effective in alleviating STSF.

Citation Download Citation

Yifan Chang and Qifan Zhao "Overcome STSF phenomenon in catastrophic forgetting", Proc. SPIE 12604, International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID 2022), 126043Y (23 May 2023); https://doi.org/10.1117/12.2674765

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Education and training

Cerium

Convolution

Feature extraction

Neural networks

Matrices

Evolutionary algorithms

Show All Keywords

Keywords/Phrases

Search In:

Publication Years