Plug-and-play illumination correction for illumination variant optical flow estimation

Jinxuan Wu; Daoyu Li; Zhijie Gao

doi:10.1117/12.3036382

22 November 2024 Plug-and-play illumination correction for illumination variant optical flow estimation

Jinxuan Wu, Daoyu Li, Zhijie Gao

Proceedings Volume 13239, Optoelectronic Imaging and Multimedia Technology XI; 132391H (2024) https://doi.org/10.1117/12.3036382
Event: SPIE/COS Photonics Asia, 2024, Nantong, Jiangsu, China

Conference Poster

Abstract

Optical flow estimation is widely used in various fields, such as motion scene understanding, autonomous driving, and object tracking. Despite great advances over the last decade, handling illumination variations in optical flow remains an open problem. Conventional optical flow estimation techniques build upon the brightness constancy assumption. Thus, variation in the lighting within the scene can affect the accuracy of optical flow algorithms. To tackle this challenge, here we report a plug-and-play illumination correction technique for robust optical flow estimation under variant illumination scenarios. This technique includes a motion-illumination decoupling strategy for the two images used to compute the optical flow, and an image brightness correction strategy. We trained a UNET-based neural network with Swin Transformer layer as basic block to decouple the motion and luminance information of the two images. Then we applied the decoupled luminance information from the reference image to the source image and adjusted its brightness. This technique makes both images maintain the same luminance for robust optical flow estimation under variant illumination conditions. We applied the reported technique on both traditional optical flow algorithms and deep learning-based optical flow algorithms, and the experiment results validated that it enables to enhance the algorithms’ illumination robustness, and achieves competitive evaluation results on the MPI-Sintel dataset and real-captured data.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Jinxuan Wu, Daoyu Li, and Zhijie Gao "Plug-and-play illumination correction for illumination variant optical flow estimation", Proc. SPIE 13239, Optoelectronic Imaging and Multimedia Technology XI, 132391H (22 November 2024); https://doi.org/10.1117/12.3036382

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available