CA-fuse-MIL: cross-attention fusion of handcrafted and deep features for whole slide image classification

Paras Goel; Saarthak Kapse; Pushpak Pati; Prateek Prasanna

doi:10.1117/12.3008533

3 April 2024 CA-fuse-MIL: cross-attention fusion of handcrafted and deep features for whole slide image classification

Paras Goel, Saarthak Kapse, Pushpak Pati, Prateek Prasanna

Proceedings Volume 12933, Medical Imaging 2024: Digital and Computational Pathology; 1293306 (2024) https://doi.org/10.1117/12.3008533
Event: SPIE Medical Imaging, 2024, San Diego, California, United States

Abstract

Whole Slide Image (WSI) analysis plays a pivotal role in computer-aided diagnosis and disease prognosis in digital pathology. While the emergence of deep learning and self-supervised learning (SSL) techniques helps capture relevant information in WSIs, directly relying on deep features overlooks essential domain-specific information captured by traditional handcrafted features. To address this issue, we propose fusing handcrafted and deep features in the multiple instance learning (MIL) framework for WSI classification. Inspired by advancements in transformers, we propose a novel cross-attention fusion mechanism “CA-Fuse-MIL,” to learn complementary information from handcrafted and deep features. We demonstrate that Cross-Attention fusion outperforms WSI classification using either just handcrafted or deep features. On the TCGA Lung Cancer dataset, our proposed fusion technique boosts the accuracy by upto 5.21% and 1.56% over two different set of deep features baseline. We also explore a variant of CA-Fuse-MIL which utilizes multiple cross-attention layers.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Paras Goel, Saarthak Kapse, Pushpak Pati, and Prateek Prasanna "CA-fuse-MIL: cross-attention fusion of handcrafted and deep features for whole slide image classification", Proc. SPIE 12933, Medical Imaging 2024: Digital and Computational Pathology, 1293306 (3 April 2024); https://doi.org/10.1117/12.3008533

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
7 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Feature extraction

Feature fusion

Education and training

Solid state lighting

Head

Binary data

Image analysis

Show All Keywords

Keywords/Phrases

Search In:

Publication Years