Spectral-spatial adaptive transformer model for hyperspectral image classification

Dong Wang; Sitian Liu; Chunli Zhu

doi:10.1117/12.2687107

27 November 2023 Spectral-spatial adaptive transformer model for hyperspectral image classification

Dong Wang, Sitian Liu, Chunli Zhu

Proceedings Volume 12767, Optoelectronic Imaging and Multimedia Technology X; 1276709 (2023) https://doi.org/10.1117/12.2687107
Event: SPIE/COS Photonics Asia, 2023, Beijing, China

Abstract

Hyperspectral imagery (HSI) classification, with the goal of assigning an appropriate land cover label to each hyperspectral pixel, is a challenging part of hyperspectral remote sensing. Recently, convolutional neural network-based HSI classification methods have shown superior performance due to their excellent locally contextual modeling ability. However, the ability of these methods to obtain deep semantic features is limited, and the computational cost increases markedly as the number of layers increases. In this work, we propose a novel spectral–spatial adaptive transformer (SSAT) model to adapt a pre-trained model for effective HSI classification. The main architecture of SSAT is based on vision transformer, which could aggregate features at different levels. Furthermore, we have designed an adaptive encoder block including spectral adaption, spatial adaption, and joint adaptation to extract HSI features in the spectral–spatial domains. Finally, the classification map is obtained from the fully connected layer. Extensive experiments have been conducted to validate the effectiveness of the proposed SSAT compared with seven typical HSI classification methods. Results demonstrate that the key classification evaluation index overall accuracy (OA) outperforms other comparative methods by at least 2.03%. Classification maps reveal the superior visualization effect, demonstrating that SSAT is an efficient tool for HSI classification.

Citation Download Citation

Dong Wang, Sitian Liu, and Chunli Zhu "Spectral-spatial adaptive transformer model for hyperspectral image classification", Proc. SPIE 12767, Optoelectronic Imaging and Multimedia Technology X, 1276709 (27 November 2023); https://doi.org/10.1117/12.2687107

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
8 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Transformers

Feature extraction

Image classification

Data modeling

Education and training

Hyperspectral imaging

Principal component analysis

Show All Keywords

Keywords/Phrases

Search In:

Publication Years