Open Access
13 November 2021 Fast 3D super-resolution imaging using a digital micromirror device and binary holography
Jialong Chen, Zhiqiang Fu, Bingxu Chen, Shih-Chi Chen
Author Affiliations +
Abstract

Significance: High-speed three-dimensional (3D) super-resolution microscopy is a unique tool to investigate various biological phenomena; yet the technology is not broadly adopted due to its high cost and complex system design.

Aim: We present a compact, low-cost, and high-speed 3D structured illumination microscopy (SIM) based on a digital micromirror device and binary holography to visualize fast biological events with super-resolution.

Approach: The 3D SIM uses a digital micromirror device to generate three laser foci with individually controllable positions, phases, and amplitudes via binary holography at the back aperture of objective lens to form optimal 3D structured patterns. Fifteen raw images are sequentially recorded and processed by the 3D SIM algorithm to reconstruct a super-resolved image.

Results: Super-resolution 3D imaging at a speed of 26.7 frames per second is achieved with a lateral and axial resolution of 155 and 487 nm, which corresponds to a 1.65- and 1.63-times resolution enhancement, respectively, comparing with standard deconvolution microscopy.

Conclusions: The 3D SIM realizes fast super-resolution imaging with optimal 3D structured illumination, which may find important applications in biophotonics.

1.

Introduction

Structured illumination microscopy (SIM) is a fast and effective method to achieve super-resolution imaging and optical cross-sectioning and thus has been widely used in biological studies. In general, a structured light illumination system modulates and shifts the high-frequency contents of a specimen to expand the optical transfer function such that finer features can be resolved by frequency demodulations.1 The first three-dimensional (3D) SIM was developed in 2008 that doubles the spatial resolution in all three dimensions.2 In this system, a grating is employed to generate 3D structured illumination via three-beam interference, where the light intensity varies both laterally and axially. The phase shifts and rotations of the illumination for frequency demodulations are achieved by mechanical translation and rotations of the grating, which requires careful tuning and has limited imaging speed.

To address the issue, fast 3D SIMs have been developed by replacing the grating with programmable spatial light modulators (SLM) or fast scanning devices;35 for example, liquid crystal-based SLMs have been used in 3D SIM to generate high-resolution 3D structured patterns via phase modulations; notably, the temporal resolution is limited by the SLM pattern rate to 60  Hz.3 To improve speed, a pair of galvanometric scanners and a piezoelectric scanner are combined with a half-wave plate to realize fast control of 3D structured patterns at kilohertz range at the expense of complex system design and higher cost. Compared with these two solutions, DMDs, i.e., a binary SLM, present unique advantages in 3D SIM in terms of speed (up to 32.5 kHz), cost, and compact system design.5 The use of DMDs can be divided into projection and holography modes. In the projection mode, the desired grating pattern is directly displayed on the DMD to generate a structured pattern. However, in this system, the diffracted zeroth-order beam has much higher intensity than the ±1st beams, resulting in low-contrast structured patterns that are difficult to reconstruct high-frequency features.5 In the holography mode, designed binary holograms are displayed on the DMD to manipulate the wavefront of the incident laser to perform fast beam shaping and random-access scanning.69 In comparison to the projection mode, the holography mode offers improved flexibility and precision in generating and controlling the position, phase, and intensity of multiple foci at the back focal plane of an objective lens, which may be used to design and create complex two-dimensional (2D) and 3D structured patterns with improved contrast.

In this work, we present a 3D SIM based on binary holography to realize super-resolution imaging on a wide-field microscope, which achieves a lateral and axial resolution of 155 and 487 nm, respectively, and an imaging speed of 26.7 frames/s (limited by the camera). In this system, a DMD is used to generate 3D structured patterns by the interference of three laser foci with designed positions, phases, and intensities encoded in the hologram. The new 3D SIM system presents improved flexibility in adjusting and fine-tuning the geometry, periods, orientations, and contrast of the 3D structured pattern, leading to a compact DMD-based 3D SIM system with improved imaging speed and signal-to-noise ratio (SNR).

2.

System Design

2.1.

Optical Configuration of the Holography-Based 3D SIM

Figure 1 shows the optical configuration of the holography-based 3D SIM system. The laser source is a 488-nm continuous-wave laser (MLD, Cobolt). First, a quarter-wave plate (WPQ10M-488, Thorlabs) is used to generate a circularly polarized beam, which ensures the same modulation depth can be achieved for structured patterns of different orientations.10 Next, the laser beam passes through a beam expander and a high reflectivity mirror (M1) to fully fill the aperture of a DMD (DLP 4500, Texas Instrument) with uniform illumination. To optimize the power efficiency, the laser incident angle is set to 21.42 deg in reference to the DMD normal to meet the blazing condition.6,7 As the designed wavefront information is encoded in the non-zeroth-order diffractions, a spatial filter is employed to spatially select the -1st-order diffraction beam. Next, the selected diffraction beam is relayed to the back focal plane of the objective lens (OBJ, UPLSAPO 60XW, NA = 1.2, Olympus), thereby forming the designed 3D structured pattern via a three-beam interference at the front focal plane. For detection, the emissions from the specimens are first collected by the objective lens, then pass through a dichroic mirror (DM, T495lpxr, Chroma), tube lens (L4), notch filter (NF, NF488-15, Thorlabs), and lastly recorded by a scientific complementary metal–oxide–semiconductor (sCMOS, C13440, Hamamatsu) camera. A precision XYZ stage (LPS-45, Physik Instrumente) is used to maneuver the specimen for 3D imaging.

Fig. 1

Optical configuration of the holography-based 3D SIM. QWP, quarter-wave plate; BE, beam expander; M1-M2, high reflectivity mirrors; L1–L4, lenses (fL1=400  mm, fL2=fL3=200  mm, fL4=400  mm); SF, spatial filter; DM, dichroic mirror; OBJ, objective lens; NF, notch filter; and sCMOS, scientific complementary metal-oxide semiconductor camera.

JBO_26_11_116502_f001.png

2.2.

Generation of Optimal 3D Structured Patterns

In this section, we present the generation and optimization of three laser foci based on binary holography for forming the 3D structured patterns, where the weighted Gerchberg–Saxton (WGS)11 algorithm is applied to improve the quality of the laser foci. To begin, the lateral position of a single laser focus can be mathematically described and controlled by a phase φ(x,y) equation, i.e., tilted wavefront, as expressed as

Eq. (1)

φ(x,y)=2πλfL1[x(xb+Δx)+y(yb+Δy)]+θ,
where λ is the excitation wavelength; xb and yb are the applied biased positions in the x and y directions of the 1st diffraction beam for separating the different diffraction orders; Δx and Δy are the applied positions of the laser focus at the back focal plane; and θ is a term to adjust the phase of the focus. To generate and control multiple laser foci, the required phase φ(x,y) can be calculated by superposing the single-focus complex phases.11 Next, the WGS algorithm is applied to each designed complex phases, where a weighting factor, wn, is introduced to optimize the accuracy of the laser focus intensities, as expressed in Eq. (2):

Eq. (2)

φ(x,y)=arg[nwnejφn(x,y)],
where φn(x,y) is the phase of the n’th focus obtained from Eq. (1). To obtain the designed intensity ratio, the optimal wn is iteratively updated with respect to the complex amplitudes Vn of each focus through Eqs. (25):

Eq. (3)

H(i,j)={1,q2φ(x,y)2π+kq20,otherwise,

Eq. (4)

Vn=i=1Ij=1JH(i,j)ejφn(x,y),

Eq. (5)

wn,s=wn,s1|Vn/Vt,n||Vn/Vt,n|,
where H(i,j) (1iI; 1jJ) is the binary hologram generated based on φ(x,y) and Lee holography;12 1 and 0 refer to “on” and “off” states of the DMD pixel; I and J are the size of DMD chip; x=i p; y=j·p; p is the DMD pixel size; k is an integer; q is a constant that determines the duty cycle of binary patterns; wn,s is the updated weighting of the n’th focus in the s’th iteration; Vt,n represents the target (normalized) amplitude of the n’th focus; and · denotes the average operation. To begin, we first assign the target amplitude in Eq. (2), i.e., the initial value of wn,0 is set based on Vt,n. After 5 to 10 iterations, the optimal wn can be identified to improve the accuracy of the amplitude |Vn/Vt,n| among the three laser foci with a converging condition that max||Vn/Vt,n|1|<0.01. Last, the calculated wn are substituted into Eq. (3) to generate the optimal binary hologram for imaging experiments.

3.

Experiments

We first perform imaging experiments to demonstrate the advantages of holography-based DMD 3D SIM over projection-based systems. Figures 2(a) and 2(b) show the measured power spectra and in-focus emissions of the structured patterns with the same modulated second-order frequency at 2.6  μm1, generated by the projection-based and holography-based DMD 3D SIM, respectively. In the projection mode, the amplitude ratio between the center focus (i.e., zeroth-order beam) and outer focus (i.e., ±1st-order beam) is π/2 according to the grating theory; in the holography mode, the amplitude ratio can be arbitrarily set; and an optimal value of 0.7 is used based on Ref. 2. From the results in Figs. 2(a) and 2(b), one may find that the holography-based SIM produces a four times stronger second-order modulation of the structured pattern in comparison to the projection-based SIM. This ensures good reconstruction of high-frequency features in 3D SIM and improved robustness against noises in high-speed imaging applications. Figures 2(c)2(e) show 3D imaging experiments on microtubules (U2OS cells stained with Alexa Fluor 488) at an image acquisition rate of 400 frames/s (i.e., 26.7 frames/s after image reconstruction) via the Wiener filter algorithm, projection-based and holography-based approaches, respectively. In the experiment, 15 raw images illuminated with different structured patterns (i.e., five equally spaced phase shifts and each with three equally spaced orientations) are sequentially recorded and processed by an open-source 3D SIM algorithm, i.e., fairSIM in ImageJ.13 To better compare the imaging results, Figs. 2(f)2(h) show zoom-in views of the red boxes in Figs. 2(c)2(e), respectively; and Fig. 2(i) shows the intensity profiles along the color-labeled dashed lines in Figs. 2(f)2(h). From the results, one may find that the holography-based SIM, shown in Figs. 2(e), 2(h), and 2(i), generates super-resolved images with higher quality and reduced artifacts; this also shows the importance of second-order modulation for 3D SIM reconstruction.

Fig. 2

Comparison between projection-based and holography-based DMD 3D SIM. (a), (b) Power spectra and the in-focus emissions of the generated structured patterns via projection-based and holography-based methods, respectively. (c)–(e) Imaging results of the microtubules in U2OS cells (stained with Alexa Fluor 488) reconstructed by the Wiener filter algorithm, projection-based, and holography-based methods, respectively. (f)–(h) Enlarged views of the red boxes in (c)–(e), respectively. (i) Intensity profiles of the color-labeled dashed lines in (f)–(h), respectively. Scale bar: 1  μm.

JBO_26_11_116502_f002.png

To characterize the resolution of the holography-based 3D SIM, we perform imaging experiments on fluorescent microspheres (Ø=100  nm, F8803, ThermoFisher), where the raw images are acquired at 100 frames/s (i.e., 6.7 frames/s after reconstruction). In the experiment, 15 raw images with different structured patterns and a modulated second-order frequency of 2.6  μm1 are used to reconstruct a super-resolved image. Figures 3(a)3(c) show the imaging results in the XY and XZ planes based on the Wiener filter algorithm, projection-based and holography-based 3D SIM, respectively. Figures 3(d) and 3(e) show the lateral and axial intensity profiles across the dashed lines in Figs. 3(a) and 3(b), respectively. To quantify the resolution, we measured the full-width at half-maximum of intensity profiles in Figs. 3(d) and 3(e); the results indicate that the holography-based 3D SIM achieves a lateral and axial resolution of 155 and 487 nm, respectively; the projection-based 3D SIM achieves a lateral and axial resolution of 159 and 482 nm, respectively; and the conventional Wiener filter algorithm achieves a lateral and axial resolution of 256 and 796 nm, respectively. Note that it is expected that projection-based and holography-based 3D SIM have similar resolution under proper illumination; and the holography-based system will have more advantage in low-light or high-speed imaging conditions due to enhanced second-order modulation, which has been demonstrated in Fig. 2. Comparing with the Wiener filter method (i.e., standard deconvolution microscopy), the holography-based 3D SIM has effectively improved the lateral and axial resolution by a factor of 1.65 and 1.63, respectively.

Fig. 3

Imaging results of 100-nm-diameter fluorescent microspheres: XY and XZ cross-sectional views of 100 nm microspheres based on the (a) Wiener filter, (b) projection-based 3D SIM, and (c) holography-based 3D SIM; (d) lateral and (e) axial intensity profiles of the dashed lines in (a)–(c). Scale bar: 300 nm.

JBO_26_11_116502_f003.png

Next, we perform imaging experiments on biological specimens, i.e., microtubules in U2OS cells stained with Alexa Fluor 488, to verify the resolution. Before image reconstruction, the collected raw images are first diagnosed by the SIMcheck14 toolbox in ImageJ, which is a common practice for assessing the quality of the raw data for SIM. Figures 4(a)4(c) show three SIMcheck results including the channel intensity profile, modulation contrast-to-noise ratio, and raw Fourier projection. The channel intensity profile compares the average intensity in different recorded frames, and a total intensity variation of 3.16% indicates high excitation stability and low photobleaching. The MCN compares the pattern strength and noise strength, which are related to the quality and reliability of the reconstructed super-resolved features; an average MCN value of 5.54 suggests our raw data are adequate for SIM reconstruction. The raw Fourier projection calculates the maximum intensity projection of the Fourier transform of raw data. In Fig. 4(c), the ±1st and ±2nd orders are clearly visible and clean, indicating good robustness of our system. Figures 4(d) and 4(e) show the reconstructed images based on the Wiener filter algorithm and 3D SIM, respectively. Figures 4(f) and 4(g) show zoom-in views of the red boxes in Figs. 4(d) and 4(e), respectively. From the results, one can clearly observe that the background emissions are better suppressed in the 3D SIM system. Last, Figs. 4(h) and 4(i) show the intensity profiles across the orange and red lines in Figs. 4(f) and 4(g), respectively. The results confirm that the holography-based 3D SIM has effectively improved the imaging resolution to resolve fine features with sizes down to 150  nm. This is consistent with the resolution characterization results shown in Fig. 3.

Fig. 4

Imaging results of microtubules: (a)–(c) SIMcheck output of channel intensity profile; MCN ratio; and raw Fourier projection, respectively; (d) and (e) reconstructed images based on the Wiener filter and 3D SIM, respectively; (f) and (g) zoom-in views of the boxed regions in (d) and (e), respectively; (h) and (i) intensity profiles of the orange and red lines in (f) and (g), respectively. Scale bars: 10  μm in (d) and (e) and 1  μm in (f) and (g).

JBO_26_11_116502_f004.png

Last, 3D imaging is performed on the microtubules specimen. The raw data consist of 30 optical cross-sections, where the distance between adjacent sections is 200 nm. Figure 5(a) shows the reconstructed 3D image in color-coded 2D maps based on the Wiener filter algorithm (left) and 3D SIM (right), where different colors represent optical sections obtained at different depths. From the XZ and YZ cross-sectional views shown in Figs. 5(b) and 5(c), one can observe the substantial resolution enhancement in both lateral and axial directions.

Fig. 5

3D imaging of microtubules specimen stained with Alexa Fluor 488: (a) reconstructed 3D images based on the Wiener filter (left) and 3D SIM (right), where the color bar indicates depth; (b) XZ and (c) YZ cross-sectional views along the red and orange dashed lines in (a). Scalebar=5  μm.

JBO_26_11_116502_f005.png

For efficiency, the overall power efficiency is measured to be 1% (i.e., 100 mW for the input and 1 mW for the output beam after the objective lens), where most energy is lost at the DMD due to diffraction.7 For imaging applications, the illumination power can be easily compensated by increasing the laser power. Note that all imaging results in Figs. 2Fig. 3Fig. 45 were obtained at a power level of 0.2 to 0.5 mW, measured at the focal region.

4.

Conclusion

We have presented a new 3D SIM achieving a lateral and axial resolution of 155 and 487 nm, respectively, and an imaging speed of 26.7 frames/s. In the system, a DMD is used to generate 3D structured illumination based on binary holography. To generate the optimal structure patterns (and thereby the SNR and imaging resolution), the WGS algorithm has been applied to improve the accuracy of the intensity ratio among the three laser foci at the back focal plane of the objective lens. 2D and 3D imaging experiments on fluorescent microspheres and microtubules have been carefully devised and performed on both the holography-based 3D SIM and deconvolution microscopy to measure the 3D point-spread function as well as to demonstrate the resolution enhancement in both lateral and axial directions. These promising results suggest our new 3D SIM platform, which is simple and compact, may find important applications in field of biophotonics.

Disclosures

The authors have no relevant financial interests in this article and no potential conflicts of interest to disclose.

Acknowledgments

This research was supported by HKSAR Research Grants Council, General Research Fund (14203020) and by the Innovation and Technology Commission, Innovation Technology Fund (ITS/178/20FP). We would like to thank Professor Yuhui Zhang and Dr. Meng Zhang at the Britton Chance Center for Biomedical Photonics, Wuhan National Laboratory for Optoelectronics-Huazhong University of Science and Technology, for providing the U2OS cell specimens.

References

1. 

M. G. L. Gustafsson, “Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy,” J. Microsc., 198 (2), 82 –87 (2000). https://doi.org/10.1046/j.1365-2818.2000.00710.x JMICAR 0022-2720 Google Scholar

2. 

M. G. L. Gustafsson et al., “Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination,” Biophys. J., 94 (12), 4957 –4970 (2008). https://doi.org/10.1529/biophysj.107.120345 BIOJAU 0006-3495 Google Scholar

3. 

L. Shao et al., “Super-resolution 3D microscopy of live whole cells using structured illumination,” Nat. Methods, 8 (12), 1044 –1046 (2011). https://doi.org/10.1038/nmeth.1734 Google Scholar

4. 

W. Liu et al., “Three-dimensional super-resolution imaging of live whole cells using galvanometer-based structured illumination microscopy,” Opt. Express, 27 (5), 7237 –7248 (2019). https://doi.org/10.1364/OE.27.007237 OPEXFF 1094-4087 Google Scholar

5. 

S. Shin et al., “Super-resolution three-dimensional fluorescence and optical diffraction tomography of live cells using structured illumination generated by a digital micromirror device,” Sci. Rep., 8 (1), 9138 (2018). https://doi.org/10.1038/s41598-018-27399-w Google Scholar

6. 

J. Cheng et al., “High-speed femtosecond laser beam shaping based on binary holography using a digital micromirror device,” Opt. Lett., 40 (21), 4875 –4878 (2015). https://doi.org/10.1364/OL.40.004875 OPLEDP 0146-9592 Google Scholar

7. 

Q. Geng et al., “Digital micromirror device-based two-photon microscopy for three-dimensional and random-access imaging,” Optica, 4 (6), 674 –677 (2017). https://doi.org/10.1364/OPTICA.4.000674 Google Scholar

8. 

C. Wen et al., “Compressive sensing for fast 3-D and random-access two-photon microscopy,” Opt. Lett., 44 (17), 4343 –4346 (2019). https://doi.org/10.1364/OL.44.004343 OPLEDP 0146-9592 Google Scholar

9. 

J. Chen et al., “Holography-based structured light illumination for temporal focusing microscopy,” Opt. Lett., 46 (13), 3143 –3146 (2021). https://doi.org/10.1364/OL.431161 OPLEDP 0146-9592 Google Scholar

10. 

M. Brunstein et al., “Full-field dual-color 100-nm super-resolution imaging reveals organization and dynamics of mitochondrial and ER networks,” Opt. Express, 21 (22), 26162 –26173 (2013). https://doi.org/10.1364/OE.21.026162 OPEXFF 1094-4087 Google Scholar

11. 

R. D. Leonardo, F. Ianni and G. Ruocco, “Computer generation of optimal holograms for optical trap arrays,” Opt. Express, 15 (4), 1913 –1922 (2007). https://doi.org/10.1364/OE.15.001913 OPEXFF 1094-4087 Google Scholar

12. 

W. Lee, “Binary synthetic holograms,” Appl. Opt., 13 (7), 1677 –1682 (1974). https://doi.org/10.1364/AO.13.001677 APOPAI 0003-6935 Google Scholar

13. 

M. Müller et al., “Open-source image reconstruction of super-resolution structured illumination microscopy data in ImageJ,” Nat. Commun., 7 (1), 10980 (2016). https://doi.org/10.1038/ncomms10980 Google Scholar

14. 

G. Ball et al., “SIMcheck: a toolbox for successful super-resolution structured illumination microscopy,” Sci. Rep., 5 (1), 15915 (2015). https://doi.org/10.1038/srep15915 SRCEC3 2045-2322 Google Scholar

Biography

Jialong Chen received his BE degree in the Department of Precision Instruments in 2017 from the Tsinghua University, Beijing, China. He is currently a PhD student in the Department of Mechanical and Automation Engineering of the Chinese University of Hong Kong. His research interests include ultrafast laser applications and structured illumination microscopy.

Zhiqiang Fu received his BE degree in mechatronics engineering from Zhejiang University, Zhejiang, China, in 2018. He is currently a PhD student in the Department of Mechanical and Automation Engineering of Chinese University of Hong Kong. His research interests include ultrafast laser applications, nanofabrication, and 3D microscopy.

Bingxu Chen received his BE degree in the Department of Precision Instruments from the Tsinghua University, Beijing, China, in 2019. He is currently a PhD student in the Department of Mechanical and Automation Engineering of the Chinese University of Hong Kong. His research interests include ultrafast spectroscopy and laser microfabrication.

Shih-Chi Chen is a professor in the Department of Mechanical and Automation Engineering at the Chinese University of Hong Kong. He received his BS degree in mechanical engineering from the National Tsing Hua University, Taiwan, in 1999; and his SM and PhD degrees in mechanical engineering from Massachusetts Institute of Technology, Cambridge, in 2003 and 2007, respectively. Following his graduate work, he entered a postdoctoral fellowship at Wellman Center for Photomedicine, Harvard Medical School, where his research focused on biomedical optics and endomicroscopy. From 2009 to 2011, he was a senior scientist at Nano Terra, Inc., a start-up company founded by Prof. George Whitesides at Harvard University, to develop precision instruments for novel nanofabrication processes. His current research interests include ultrafast laser applications, biomedical optics, precision engineering, and nanomanufacturing. He is a member of the American Society for Precision Engineering (ASPE), American Society of Mechanical Engineers (ASME), SPIE, and the Optical Society (OSA); and currently serves as an associate editor for Journal of Micro and Nano-Manufacturing, IEEE Transactions on Nanotechnology, and HKIE Transactions. In 2003 and 2018, he received the prestigious R&D 100 awards for developing a six-axis nanopositioner and an ultrafast nanoscale 3D printer, respectively.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Jialong Chen, Zhiqiang Fu, Bingxu Chen, and Shih-Chi Chen "Fast 3D super-resolution imaging using a digital micromirror device and binary holography," Journal of Biomedical Optics 26(11), 116502 (13 November 2021). https://doi.org/10.1117/1.JBO.26.11.116502
Received: 16 June 2021; Accepted: 25 October 2021; Published: 13 November 2021
Lens.org Logo
CITATIONS
Cited by 1 scholarly publication.
Advertisement
Advertisement
KEYWORDS
3D image processing

Holography

Digital micromirror devices

Binary data

Stereoscopy

Super resolution

Image resolution

Back to Top