Snapshot-to-video autoencoder for compressed ultra-high-speed imaging

Xianglei Liu; João Monteiro; Isabela Albuquerque; Yingming Lai; Cheng Jiang; Shian Zhang; Tiago Falk; Jinyang Liang

doi:10.1117/12.2610281

2 March 2022 Snapshot-to-video autoencoder for compressed ultra-high-speed imaging

Xianglei Liu, João Monteiro, Isabela Albuquerque, Yingming Lai, Cheng Jiang, Shian Zhang, Tiago Falk, Jinyang Liang

Proceedings Volume 12019, AI and Optical Data Sciences III; 1201903 (2022) https://doi.org/10.1117/12.2610281
Event: SPIE OPTO, 2022, San Francisco, California, United States

Abstract

Single-shot two-dimensional (2D) optical imaging of transient scenes is indispensable for numerous areas of study. Among existing techniques, compressed optical-streaking ultrahigh-speed photography (COSUP) uses a cost-efficient design to endow ultra-high frame rates with off-the-shelf CCD and CMOS cameras. Thus far, COSUP’s application scope is limited by the long processing time and unstable image quality in existing analytical-modeling-based video reconstruction. To overcome these problems, we have developed a snapshot-to-video autoencoder (S2V-AE)—a new deep neural network that maps a compressively recorded 2D image to a movie. The S2V-AE preserves spatiotemporal coherence in reconstructed videos and presents a flexible structure to tolerate changes in input data. Implemented in compressed ultrahigh-speed imaging, the S2V-AE enables the development of single-shot machine-learning assisted real-time (SMART) COSUP, which features a reconstruction time of 60 ms and a large sequence depth of 100 frames. SMART COSUP is applied to wide-field multiple-particle tracking at 20 thousand frames-per-second. As a universal computational framework, the S2V-AE is readily adaptable to other modalities in high-dimensional compressed sensing. SMART COSUP is also expected to find wide applications in applied and fundamental sciences.

Conference Presentation

Citation Download Citation

Xianglei Liu, João Monteiro, Isabela Albuquerque, Yingming Lai, Cheng Jiang, Shian Zhang, Tiago Falk, and Jinyang Liang "Snapshot-to-video autoencoder for compressed ultra-high-speed imaging", Proc. SPIE 12019, AI and Optical Data Sciences III, 1201903 (2 March 2022); https://doi.org/10.1117/12.2610281

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available