A sample survey study of poly-semantic neurons in deep CNNs

Changwan Zhang; Yue Wang

doi:10.1117/12.2674650

23 May 2023 A sample survey study of poly-semantic neurons in deep CNNs

Changwan Zhang, Yue Wang

Proceedings Volume 12604, International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID 2022); 126043A (2023) https://doi.org/10.1117/12.2674650
Event: 2nd International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID 2022), 2022, Guangzhou, China

Abstract

Although deep CNN networks have excellent image classification performance, they do not provide interpretability, and furthermore existing work reveals that these models have complex internals, for example, mysterious polysemantic neurons activate to multiple features. In this work, we analyze the intermediate data of the network dissection paper made by Bau et al. to understand to what extent polysemantic neurons exist. We divide the polysemantic neurons into five types and calculate the percentage of each type by sampling. We find that above 50% neurons identify one concept but there are a quite proportion of neurons that recognize two or more features. This can explain the high classification accuracy and some capacity saving of a deep CNN. By case studies, we draw some conclusions and hypotheses: First, unlike the human visual system, a CNN cannot distinguish detailed features (metaphor: a CNN is like a nearsighted eye). Second, the reason that the CNN is prone to adversarial attacks may be partially due to the polysemantic neurons. Third, polysemantic neurons may partially explain why people wrongly visualize one thing as another in neuroscience.

Citation Download Citation

Changwan Zhang and Yue Wang "A sample survey study of poly-semantic neurons in deep CNNs", Proc. SPIE 12604, International Conference on Computer Graphics, Artificial Intelligence, and Data Processing (ICCAID 2022), 126043A (23 May 2023); https://doi.org/10.1117/12.2674650

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Neurons

Visualization

Image classification

Education and training

Semantics

Statistical modeling

Neuroscience

Show All Keywords

Keywords/Phrases

Search In:

Publication Years