CAFformer: channel-based adaptive fusion with transformer for reference-based super-resolution

Shuxiang Li; Yuhan Dong; Yuhong Yuan

doi:10.1117/12.3033660

13 June 2024 CAFformer: channel-based adaptive fusion with transformer for reference-based super-resolution

Shuxiang Li, Yuhan Dong, Yuhong Yuan

Proceedings Volume 13180, International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024); 1318076 (2024) https://doi.org/10.1117/12.3033660
Event: International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024), 2024, Guangzhou, China

Abstract

Reference-based Super-Resolution (RefSR) has garnered significant attention for its capacity to leverage external a priori information. RefSR involves the intricate process of transferring texture details from a Reference (Ref) image to a LowResolution (LR) image, relying on corresponding pixel or patch relationships. Despite the proliferation of Convolutional Neural Network (CNN) or Transformer-based models aimed at enhancing RefSR performance, a substantial number of approaches neglect the distinct roles played by LR and Ref images within the reconstruction process. Specifically, LR images inherently contain fundamental structural information corresponding to High-Resolution (HR) images, while Ref images encapsulate potential high-frequency details. In this paper, we introduce a novel module for channel-based adaptive fusion, specifically designed to integrate features extracted from LR and Ref images. The proposed module adeptly combines LR and Ref image features along the channel dimension, enabling the efficient utilization of Transformer long-range modeling capabilities across the width and height dimensions of the feature map. The incorporation of this innovative module results in superior performance compared to state-of-the-art Transformer-based methods, concurrently demonstrating an improvement in inference speed. Rigorous experimental results validate the efficacy of our approach, Channelbased Adaptive Fusion with Transformer for Reference-based Super-Resolution (CAFformer), outperforming existing methods in both quantitative and qualitative evaluations. This contribution holds promise for advancing the field of superresolution reconstruction by comprehensively addressing the inherent distinctions between LR and Ref images.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Shuxiang Li, Yuhan Dong, and Yuhong Yuan "CAFformer: channel-based adaptive fusion with transformer for reference-based super-resolution", Proc. SPIE 13180, International Conference on Image, Signal Processing, and Pattern Recognition (ISPP 2024), 1318076 (13 June 2024); https://doi.org/10.1117/12.3033660

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Transformers

Image fusion

Deformation

Super resolution

Feature fusion

Image processing

Image restoration

Show All Keywords

Keywords/Phrases

Search In:

Publication Years