2.1.

In-Line DHM

In this review, we refer to Gabor holography as in-line holography.⁵⁷ In-line DHM is a type of microscopy without objective lenses, and as illustrated in Fig. 1, a single light beam directed onto a pinhole of a diameter of the order of a wavelength illuminates the object, typically several thousand wavelengths away from the pinhole, and the object beam is the part of the incident light that is scattered by the object and the unscattered remainder is taken as the reference beam. If the light source is coherent, the pinhole is not necessary. The object field is automatically in alignment with the reference beam, and the interference of the object and reference beams results in the holographic diffraction pattern, which is recorded by CCD camera and then transferred to a computer for numerical reconstruction.

Fig. 1

In-line digital holographic microscopy (DHM) setup. MO, microscope objectives; P, pinhole; S, sample object.

By removing background effects first and then reconstructing the object field at different planes, a 3-D image can be built up from a single two-dimensional image, and the entire hologram pixel count is utilized, which also leads to shorter minimum distances for reconstruction and higher resolution of the resultant image. With the simplicity of the apparatus and large depth of field, the in-line DHM is particularly useful for particle image analysis. In conjunction with reconstruction algorithm, the 3-D position of a particle can be determined and then applied to particle velocimetry, such as tracking small particles or swimming cells in a liquid flow.

2.2.

Off-Axis DHM

In an off-axis DHM setup, a Mach-Zehnder interferometer is chosen since it offers flexibility in alignment, especially when microscopic imaging optics is used. One microscope objective lens (MO) is needed for object magnification and another one is used in the reference arm to match the curvatures of the object and reference wave fronts. We illustrated the off-axis holographic microscopy setup in Fig. 2. The object arm contains a sample stage and an MO that projects a magnified image of the object onto a CCD camera. The reference arm similarly contains another MO, and the reference and object waves are offset by an angle to avoid the overlap of the reference and the twin images, so that the holographic interference pattern contains fringes due to interference between the diffracted object field and the off-axis reference field. The captured holographic image is numerically converted into Fourier domain to obtain the angular spectrum,^44,45 and a spatial filter is then applied to retain the real image peak alone. The filtered angular spectrum is propagated to appropriate distance and, by an inverse Fourier transform, reconstructed as an array of complex numbers containing the amplitude and phase images of the sample. Both amplitude and phase profiles can be further analyzed to quantitatively determine the positions of objects in a 3-D space.

Fig. 2

Off-axis DHM setup. M's, mirrors; BS’s, beam splitters; MO’s, microscope objectives; S, sample object.

2.3.

Quantitative Phase Microscopy by Digital Holography

Off-axis DHM is a very effective process for achieving high-precision QPM since it allows measurement of optical thickness with nanometer-scale accuracy by single-shot, wide-field acquisition, and it yields phase profiles without some of the complications of other phase imaging methods. The phase image is immediately and directly available on calculating the two-dimensional complex array of the holographic image, and the phase profile conveys quantitative information about the physical thickness and index of refraction of cells. We have recently utilized DH-QPM to study the wrinkling of a silicone rubber film by motile fibroblasts.⁵⁸ The wrinkle formation has been visualized, quantitative measures of surface deformation have been extracted, and cellular traction force has been estimated in a direct and straightforward manner. A nonwrinkling substrate, collagen-coated polyacrylamide (PAA) was also employed to make direct measurement of elastic deformations, albeit at discrete locations.⁵⁹ The Young’s modulus of PAA can be adjusted by controlling the concentrations of the monomer and cross-linker. DH-QPM, with its capacity of yielding quantitative measures of deformation directly, has been employed to measure the Young’s modulus of PAA,⁶⁰ which provided a very effective process for achieving high-precision quantitative phase microscopy. We have utilized DH-QPM to study motile fibroblasts deforming PAA gel,⁶¹ where the cell-substrate adhesion has been visualized and quantitative measures of surface deformation have been extracted. The substrate stiffness and quantitative measures of substrate deformation have been combined to produce estimates of the traction forces and characterize how these forces vary depending on the substrate rigidity.

2.4.

Twin-Beams DHM

The twin-beams DHM setup is illustrated in Fig. 3.⁶² Two beams, coming from the same laser source and slightly off-axis, enter into the $20\times$ microscope objective and result in an image on a CCD plane. Theoretically, the two beams have the same focal plane; however, the twin beams can experience aberrations resulting in some focus-shift. As shown in Fig. 3, each particle of the sample appears as two projections on the CCD array. The tracking is performed by evaluating the two out-of-focus projections of the particles due to the twin-beams onto the CCD plane, and the separation between the two projections is a function of the longitudinal position of the particle. QPM images of the micro-objects can be directly obtained from the holograms recorded by the same configuration.

Fig. 3

Twin-beams DHM setup (reprinted from Ref. 62 by permission of OSA).

3.1.

Quantification of Image Sharpness and Peak Searching

To localize an object in a 3-D volume, an automated data analysis algorithm to accurately identify depth position is required. For example, pure amplitude objects located out of focal plane appear as gray patches without sharp structures, while pure phase objects show minimum visibility when they are in focus.^63–65 Thus, focusing metrics based on the quantification of image sharpness are applied by searching on the reconstructed images for the planes that contain the objects with the peak and sharpest details to establish the all-in-focus profile and its corresponding depth map.⁶⁶ At each pixel, the intensity variation in longitudinal direction is investigated to find out the peak intensity. The peak intensity value becomes a pixel of an all-in-focus profile and its location in longitudinal direction is recorded as a corresponding depth map value. The combination of resultant all-in-focus profile and depth map enables one to produce the 3-D visualization of objects, and object properties can then be extracted.

We recently utilized off-axis DHM and numerical autofocusing algorithm based on peak searching to image suspended polymer microspheres (9.6 μm in diameter) in suspension.²¹ Figure 4(a) presents the hologram of microspheres in suspension captured by off-axis DHM. Angular spectrum method^44,45 was applied and the reconstructed amplitude image was shown in Fig. 4(b), where several in-focus and out-of-focus microspheres were seen at this reconstruction plane. The field of view was $90\times 90{\mu \text{m}}^2$ with $464\times 464\text{pixels}$. The microsphere focused the incoming light and formed a bright spot along the optical path. Therefore, the pattern of the microsphere appeared to have a maximum intensity at the center near the in-focus plane and the surrounding was dark. Then, the autofocusing algorithm based on peak searching was applied to identify the in-focus position of each particle. The hologram was numerically reconstructed in longitudinal direction $Z$ from 100 to $-100\mu \text{m}$, in $-2\text{-}\mu \text{m}$ steps. The autofocusing algorithm was then applied by searching on the reconstructed images for the planes that contained the objects with the peak and sharpest details to establish the all-in-focus profile. The peak intensity at each pixel along the longitudinal direction was chosen and this value became a pixel of an in-focus profile; meanwhile, its location in longitudinal direction was also recorded as a corresponding depth map value. The combination of all-in-focus intensity profiles [Fig. 4(c)] and the corresponding depth maps [Fig. 4(d)] enabled to produce the 3-D visualization of microspheres and a threshold on the intensity allowed for distinguishing objects from other elements, which is illustrated in Fig. 4(e). In fact, we took every center position of brightest points in Fig. 4(c) as $X\text{-}Y$ locations of particles, and Fig. 4(d) was used to determine the $Z$-location of those particles identified in Fig. 4(c). Figure 4(d) is mostly noisy and $Z$ information of only the locations where the particles were present was actually used. Figures 4(f) to 4(h) show the $XY$, $XZ$, and $YZ$ views of the 3-D profile, respectively. This straightforward autofocusing algorithm is able to determine focal planes for all the objects in the reconstructed volume automatically. The combination of depth map $Z$ and axial $X\text{-}Y$ position of objects allows for quantitative 3-D profiling. This method has been widely used by researchers, and more applications will be discussed in Sec. 4.

Fig. 4

Three-dimensional (3-D) profiling of suspended microspheres. Field of view of (a) to (d) is $90\times 90{\mu \text{m}}^2$ with $464\times 464\text{pixels}$. (a) Hologram. (b) Amplitude image. (c) All-in-focus intensity profile. (d) Depth position profile. (e) 3-D profile ($90\times 90\times 120{\mu \text{m}}^3$) and gray-scale representation of intensity. (f) $XY$ view of three-dimensional profile ($90\times 90{\mu \text{m}}^2$). (g) $XZ$ view of three-dimensional profile ($90\times 120{\mu \text{m}}^2$). (h) $YZ$ view of three-dimensional profile ($90\times 120{\mu \text{m}}^2$).

3.2.

3-D Deconvolution Methods

Overviews of the deconvolution methods regarding the out-of-focus problem in optical microscopy can be found in Refs. 6768.69. to 70. In Ref. 71, 3-D deconvolution methods, including instant and iterative deconvolution, have been applied to restore particle distribution from holograms and the corresponding reconstructions. Instant 3-D deconvolution could be applied on separated particles spread in a certain volume and allowed for the identification of the depth position of object and removal of out-of-focus information. The way to perform the instant 3-D deconvolution was to apply it to reconstructed intensity distributions from holograms using Eq. (1):

Fig. 5

(a) and (b) Results of the instant 3-D deconvolution of the intensities reconstructed from the simulated hologram of a particle distribution. (c) and (d) Results of the iterative 3-D deconvolution of the complex fields reconstructed from the simulated hologram of continuous objects (reprinted from Ref. 71 by permission of OSA).

Iterative deconvolution could be applied on objects of continuous and extended shapes and distributions. The way to perform the iterative deconvolution was to apply it to reconstructed complex fields from holograms. The iterative loop included

3.3.

Rayleigh-Sommerfeld Backpropagation Method

Rayleigh-Sommerfeld backpropagation method is a general and fast volume reconstruction analysis tool for DHM.^55,72,73 A well-known Gouy phase anomaly^55,74 can be used to discriminate between objects lying on either side of the holographic image plane. A recorded in-line hologram was reconstructed using Rayleigh-Sommerfeld diffraction integral to refocus the scattered object at a particular position and observe the phase anomaly as a contrast inversion about the object’s position by Wilson et.al.⁷² If the complex field of scattering objects was obtained in the focal plane, it could be reconstructed at plane $z$ above the focal plane as a convolution of amplitude in the focal plane with Rayleigh-Sommerfeld propagator, as in Eq. (3):⁵⁵

Fig. 6

Example data from a single particle. Scale bars represent 2 μm in all cases. (a) Vertical slice through the center of an image stack created by physically translating the sample. (b) Image of a particle located at $z\approx 9\mu \text{m}$ (downstream of the focal plane in the illumination path). (c) Optical field reconstructed from the previous panel. The hologram plane ($z=0$) would be located below the bottom of the image. (d) $\text{Intensity}\text{gradient}<0$. The dark central spot is azimuthally symmetric about the $z$ axis and gives the particle location in all three dimensions. (e) and (f) The companion images to (b) and (c), for a particle located at $z\approx -9\mu \text{m}$ (upstream in the illumination path). (g) $\text{Intensity}\text{gradient}>0$. The particle location is specified by a maximum of intensity gradient for those scatterers with $z<0$ (reprinted from Ref. 72 by permission of OSA).

3.4.

Compressive Holographic Method

Compressive sensing is a technique to efficiently acquire the information of signal in the underdetermined linear system supported by the sparseness of the signal.⁷⁵ Hence, the higher dimensional signal can be recovered by the lower-dimensional measurements. Recently, some research groups have been studying about the compressive holography, which applied the concept of compressive sensing to holography.^75–78 In holographic recording, 3-D data of the object were recorded as a two-dimensional complex hologram. Supported by the condition that the distribution of the object in 3-D space was sufficiently sparse, the compressive sensing technique could retrieve the 3-D distribution from two-dimensional complex hologram. Brady et al.^75,76 had successfully shown the possibility of the compressive holography with the hologram recorded by in-line DHM. In-line DHM, because of its simplicity and efficiency, provided a potent tool for compressive sampling by recording a 3-D object onto a two-dimensional focal plan array, which was related to Fourier transform of the object field. Compressive holography collected fewer amounts of measurements than voxels in the reconstructions. The 3-D datacube was encoded and compressed into two-dimensional holographic measurement by holographic sampling process, and the encoding was then inverted using the compressive sampling theory.

Two seed parachutes were illuminated and placed away from the detector array at different distance separately [Fig. 7]. After reconstruction, the stem and petals representing the high-frequency features in the image were clearly shown and the distance between the two parachutes were also estimated. The final result presented a 3-D datacube of voxels reconstructed from a single two-dimensional hologram, and it demonstrated the applicability of compressive holography to generate multidimensional images from lower-dimensional data. A detailed description of compressive holography theory and experiment results can be found in Refs. 75 and 76.

Fig. 7

(a) Raw Gabor hologram for seed parachutes of taraxacum. (b) and (c) Photographs of the individual objects. (d) Transverse slices at various ranges of the backpropagated (numerical refocusing) field (reprinted from Ref. 75 by permission of OSA).

3.5.

Twin-Beams DHM Method

As previously depicted in Fig. 3, the twin-beams DHM had two plane beams passing through one objective microscope. Theoretically, this process could be sketched as two cones superposed and transmitted through the particle sample P (Fig. 8, left). Beams B1 and B2 produced projections P1 and P2 of particle P on the CCD plane, respectively. In both simulation and experiment, the 3-D coordinates of B1 and B2 points and the radius of each beam circle on the CCD plane were first determined based on a simple calibration method.⁶² Then, the corresponding coordinates of P1 and P2 could be evaluated accordingly from the simple geometric schematic and the position of real object P was determined. A real situation of various motile cells floating into a microfluidic chamber was studied using this method and the 3-D paths of cells were estimated [Fig. 8(b)].⁶² The phase-contrast maps of the cells were also obtained by numerically reconstructing the recorded holograms.^79,80

Fig. 8

Twin-beams DHM results. Sketch of interference between two beams (a) and estimated path for random motion of the three cells (b) (reprinted from Ref. 62 by permission of OSA).

4.1.

Three-Dimensional Profiling of Suspended Particles

Micro-sized particles, such as latex spheres and ferromagnetic beads, suspended in gelatin have been imaged in a 3-D volume by in-line DHM.^81–83 A hologram of objects and another background image without the object present were both recorded. The difference of these two images eliminated the artifacts of the illumination source. Reconstruction was achieved by means of Kirchhoff-Helmholtz transform⁸⁴ to generate the complex field of objects, and the combination of a stack of two-dimensional reconstructions at various planes resulted in the 3-D profile of the objects. The amplitude obtained from complex field was used to represent objects. By inspecting intensity profiles through a particle, the depth position was obtained where a sharpest intensity appeared. This allowed estimating a particle’s 3-D position coordinates within submicron. When the derivative of a second-order polynomial fitted to the intensity profiles was taken, the $X$, $Y$, and $Z$ positions of particles could be determined within 50 nm.

In Ref. 85, the distribution of dense microparticles (3.2 μm) suspended in a liquid volume with the depth of 1 mm was measured, as shown in Fig. 9. Figure 9(a) is a section of the recorded hologram and the reconstructed images at different depths are shown in Figs. 9(b) to 9(d). It was clearly observed that the in-focus particles appeared as dark circular spots. Figure 9(e) shows the combination of all the reconstructed images. The peak-searching method was then applied and each pixel was assigned the lowest intensity obtained over the entire depth. A thresholding segmentation method based on signal-to-noise ratio $[I-\overline{I}]/{\sigma }_I$ was applied on the reconstructed planes,⁸⁶ where $\overline{I}$ is the mean intensity of the particles in the volume and ${\sigma }_I$ is the standard deviation of intensity over this volume. A list of line segments after scanning through the reconstructed images were combined to two-dimensional planar blobs using a join operator, which were then united into 3-D particle traces by repeating the peak-searching procedure at different depths. The position of a particle was then determined by using the centroid of the 3-D blob and the distribution of all the particles detected within the 3-D volume was presented in Fig. 9(f).

Fig. 9

Results of 3-D profiling of particles. (a) Part of a recorded hologram using a $10\times$ objective, containing 3.189-μm-diameter particles in a 1-mm-deep solution. (b) to (d) Reconstruction of planes located 120, 580, and 800 μm from the hologram plane. In-focus particles appear as dark spots on the bright background. (e) A combined and/or compressed image containing all the particles covered by the hologram section shown in (a). (f) Location of all the particles detected within the entire $1.5\times 1.5\times 1{\mathrm{mm}}^3$ volume, totaling 5769 particles (reprinted from Ref. 85 by permission of OSA).

4.2.

Three-Dimensional Study of Microfluidics

Kreuzer’s group has studied the microfluidic phenomena by in-line DHM with the capacity of large depth of field and numerical reconstruction.⁸⁷ In a first experiment, a sphere of 150 μm diameter was attached to the top of a millimeter-wide tank, which was filled with water and seeded with small micro-sized latex beads as tracer. The water was sucked and pumped to flow by a blotting paper and the flow past a spherically shaped obstruction was recorded as a series of holograms by in-line DHM. The difference between consecutive hologram pairs was calculated pixel by pixel ($h_1-h_2,h_3-h_4,\dots ,h_{n-1}-h_n$) to eliminate background structure and retain only the information of moving beads, where $h_n$ is the $n$’th hologram of recorded frames. The resulting difference holograms were summed ($h_2-h_1+h_4-h_3\dots +h_{13}-h_{14}$) into a single hologram, which was then reconstructed with Kirchhoff-Helmholtz transform to obtain the focal planes of the stream lines of beads (Fig. 10). The flow information containing sequential positions of beads at successive recording times and the velocity field defined by vectors connecting two successive positions was measured and analyzed and the velocity field was proven to be in good agreement with Navier-Stokes’ solution. In this way, as indicators of flow patterns, the trajectories and velocities of latex microspheres were measured in a 3-D volume and time with subsecond and micron resolution.

Fig. 10

(a) 3-D flow around a fixed sphere. Ninety holograms were taken at intervals of 0.17 s to generate a difference hologram. (b) One of 60 reconstructions made to render the field velocity shown in (c). In (d) the solid blue lines represent the solution to the Navier–Stokes equation for our experiment and the arrows correspond to the measured velocity field (reprinted from Ref. 87 by permission of Optik).

The size and 3-D positions of particles in flow-through system, such as fast-moving bubbles in air-water mixture flows, were measured by Tian et al.⁸⁸ Bubbles were generated by a motor driving a small propeller in a water tank and the density was not so high as to corrupt the illumination beam. Holograms were first recorded by in-line DHM system and the Fresnel approach was used to propagate the recorded object field to various planes in the vicinity of the object.^55,56 At each propagation distance, the Fourier transform of the hologram was multiplied by the Fresnel approximated transfer function and an inverse Fourier transform was then applied to obtain the complex field of objects containing the intensity profile. A focused bubble appeared to have a minimum intensity and this was used as a focus criterion to locate bubbles. A two-dimensional projection image was generated by scanning the reconstructed images and recording the minimum intensity value for each pixel. The corresponding depth map where the minimum intensity occurred was also recorded for analysis. By applying thresholding on the minimum intensity projection image, a binary image showing only the edges of bubbles was created and then combined with the depth map to determine the depth of objects. The issue of overlapping edges in the projection image was resolved by using a Gaussian mixture model.⁸⁹ Bubble sizes were determined from the areas enclosed by each edge. In this way, the 3-D visualization of the positions and sizes of bubbles was presented, as shown in Fig. 11.

Fig. 11

Data processing results from a $1024\times 1024\text{pixel}$ hologram. (a) 3-D visualization of data processing results from a single hologram (diameters not to scale). (b) Bubble size distribution (reprinted from Ref. 88 by permission of OSA).

4.3.

Four-Dimensional Motility Tracking of Biological Cells

Motility is a major characteristic of certain types of cells, and it is essential to understand the motility of cells by measuring their 3-D motion. DHM offers a rapid and efficient method to monitor the 3-D motile behavior and dynamic process of various types of cells. The effectiveness of this method is quantified in terms of the maximum number and speed of the moving cells that can be tracked, as well as the direction of the motion (lateral and axial). The results from this work can be further used to implement an optical trap that can automatically track and capture cells with specified characteristics, such as speed, size, or shape.

4.3.1.

Red blood cells

DHM has been applied in the field of hemodynamic research.⁹⁰ A human red blood cells (RBC) sample was supplied to a circular microtube and the RBC motion in this microtube flow was recorded sequentially by in-line DHM. The angular spectrum method and quantification of image sharpness were applied to reconstruct and locate the focal planes of cells. The measurement accuracies in the lateral and axial directions were $\pm 0.3$ and $\pm 1\mu \text{m}$, respectively. In this way, the full four-dimensional trajectories of RBCs, including space and time information, and the three-dimensional velocity field were measured, and the linear movement of cells was seen in the circular microtube. The feasibilities of in-line DHM, volumetric particle tracking, and numerical reconstruction were demonstrated with RBCs.

3-D tracking of the sedimentation of RBCs has also been implemented by DH-QPM based on the phase contrast images.⁹¹ The captured holograms were reconstructed by nondiffractive spatial-phase-shifting-based reconstruction^92,93 to access to the complex field of objects, which was then numerically propagated to different planes by a convolution-based algorithm.⁹⁴ The phase profile was extracted and phase unwrapping algorithm was applied for future focusing evaluation. The autofocus criterion for a sharply focused phase object showed minimum visibility.^64,65 The time dependence of axial displacement and depth position where the focused phase happened were combined to illustrate the 3-D process of the sedimentation of RBC due to gravitation, as illustrated in Fig. 12.

Fig. 12

Time-dependent 3-D tracking of a sedimenting human RBC by DHM. (a) to (d) Unfocused phase distributions of the RBC during the sedimentation process at $t=0$, $t=22\mathrm{s}$, $t=44\mathrm{s}$, and $t=66\mathrm{s}$. (e) to (h) Quantitative phase distributions obtained of the same digital hologram as in (a) to (d) by application of digital holographic autofocus; the dotted box represents the region of interest for the $x\text{-}y$ tracking in (j). (i) Time dependence of the axial position $\mathrm{\Delta }g$ of the sample obtained by digital holographic autofocusing. (j) $x\text{-}y$ trajectory of the RBC obtained by determination of the coordinates with maximum phase contrast. (k) 3-D trajectories obtained by combination of the data in (i) and (j) (reprinted from Ref. 91 by permission of SPIE).

4.3.2.

Free-swimming cells

Chilomonas are fast-moving cells, which sense and respond to their surroundings by swimming toward or away from stimuli. We have utilized off-axis DHM to track the motility of chilomonas in a 3-D volume and time.²¹ A time-lapse hologram movie of the movement of chilomonas (average diameter 10 μm) was first recorded at 30 fps. The field of view was $90\times 90{\mu \text{m}}^2$ with $464\times 464\text{pixels}$. An excerpt of 14 frames was taken and the difference between consecutive hologram pairs was calculated pixel by pixel ($h_1-h_2,h_3-h_4,\dots h_{n-1}-h_n$) to eliminate background structure and retain only the object information, where $h_n$ is the $n$’th hologram of selected frames. The resulting seven difference holograms (from a total of 14) were then summed ($h_2-h_1+h_4-h_3\dots +h_{13}-h_{14}$) into a single hologram [Fig. 13(a)], which contained all the information on moving cells. Reconstruction by the angular spectrum method was applied on the resultant hologram, and Fig. 13(b) was the amplitude image reconstructed at the specific plane $Z=0$. The 3-D trajectory of the moving chilomonas was built up by first reconstructing the two-dimensional resultant hologram from $Z=80$ to $-120\mu \text{m}$, in $-2\text{-}\mu \text{m}$ steps. The autofocusing method based on peak searching was then employed, keeping the maximum intensity value at each pixel in the reconstructed image along $Z$ direction to obtain the all-in-focus intensity projection [Fig. 13(c)] and combining with the depth position where in-focus intensity occurred [Fig. 13(d)] to determine the focal planes for the entire trajectory in the reconstructed volume. A threshold of intensity was applied to eliminate the unnecessary background, and the center position and the mean intensity of every point cloud representing every trajectory of chilomonas were determined. The combination of depth position $Z$ and $XY$ displacement of cells allowed for quantitative 3-D motility tracking. The cell was estimated to move a total path length of 93 μm at the velocity of $198\mu \text{m}/\mathrm{s}$. Figure 13(e) demonstrates the applicability of DHM for automatic four-dimensional tracking of living cells with temporal and spatial resolution at the subsecond and micro level. Figures 13(f) to 13(h) show the $XY$, $XZ$, and $YZ$ views of the 3-D trajectory, respectively.²¹

Fig. 13

Four-dimensional tracking of a moving chilomonas cell. The field of view of (a) to (d) is $90\times 90{\mu \text{m}}^2$ with $464\times 464\text{pixels}$. (a) Difference hologram. (b) Amplitude image. (c) All-in-focus intensity profile. (d) Depth position profile. (e) 3-D profile of the trajectory of the cell ($90\times 90\times 90{\mu \text{m}}^2$) and gray-scale representation of intensity. (f) $XY$ view of 3-D trajectory ($90\times 90{\mu \text{m}}^2$). (g) $XZ$ view of 3-D trajectory ($90\times 90{\mu \text{m}}^2$). (h) $YZ$ view of 3-D trajectory ($90\times 90{\mu \text{m}}^2$).

In Ref. 95, a real-time hologram movie of free-swimming algae (5 to 10 μm in diameter) in seawater was recorded by in-line DHM. Subtracting the consecutive frames first and adding these difference frames then provided a final image containing the time evolution of algae trajectories free from background interference. A similar peak-searching reconstruction algorithm as described previously constructed the 3-D trajectories of algae in seawater approaching a concentrated salt solution. Free-swimming algae were seen to sense and respond to their surroundings by swimming toward or away from stimuli at an average velocity of $150\mu \text{m}/\mathrm{s}$.

Specimen of small-sized bacteria ($<1.5\mu \text{m}$ in diameter) inside a diatom (200 to 300 μm in diameter) was tracked in three dimensions since the diatom was almost stationary in seconds and the in-line DHM and reconstruction algorithm, including peak searching, edge detection, etc., were applicable in this weaker structure.⁸³ This demonstrated the possibility of tracking smaller and weaker scatters with high resolution by in-line DHM.

4.3.3.

Special cells with helical trajectories

Various self-propulsive microorganisms of different shapes with various width-to-length ratios exhibit helical swimming paths. Unlike spherical particles, these structures would cause nonuniform scattering, which makes it difficult to determine the depth map due to the large depth-of-focus of the specimen. In Ref. 96, cellular behaviors and interactions among microorganisms, such as the prey-induced changes in swimming behavior of predatory dinoflagellates, were statistically studied. The captured holograms by in-line DHM were numerically reconstructed by Fresnel transformation and the intensity field of the reconstructed images can be obtained. The same algorithm as described in Refs. 85 and 86 was utilized to determine the 3-D positions of cells. Besides this, information on each cell, such as cross-section area, dimensions, aspect ratio, total reconstructed volume, and in-focus snapshots, are stored for cell tracking. Each helical trajectory was taken as a vector determined by each criterion step by step, i.e., smoothness of the trajectory segment, 3-D velocity, acceleration, similarity of cell size, in-focus cross-section, 3-D segmented volume, shape based on correlation, aspect ratio, etc. This statistical analysis provided simultaneous tracking and characterization of cellular behavior and interactions among microorganisms in dense suspensions. Figure 14 presents the trajectories of P. piscicida before introduction of prey in a 3-D volume. In Ref. 97, another type of ellipsoidal shape Cochlodinium polykrikoides cell had a size of $\sim 40\mu \text{m}$ length and 25 μm width, and it could form chains consisting of several cells. A sample of these mixed C. polykrikoides cells, also exhibiting helical swimming paths in filtered seawater, were also statistically studied by in-line DHM and similar approaches.

Fig. 14

The 3-mm-deep sample volume showing trajectories of P. piscicida before introduction of prey. Upper inset shows close-up of part of the sample volume, which is marked by red dash-dot line in the bottom image [reprinted from Ref. 96, Copyright (2007) National Academy of Sciences, U.S.A].

Moreover, helical trajectories ($>1500$) of human sperm (3 to 4 μm) within a large volume ($>1\mu \text{L}$) was tracked and analyzed by a dual-view and dual-color lens-free DHM setup, which consisted of two partially coherent LEDs of different wavelengths placed at 45 deg with respect to each other (Fig. 15).⁹⁸ The recorded in-line holograms only reconstructed at correct combination of depth, angle, and wavelength could produce clear images, and the sperm head images projected at two different wavelengths and viewing angles could be isolated even if they were recorded in the same frame. Combining with iterative phase recovery method,⁹⁹ the peak-searching algorithm, and related numerical smoothing method, the 3-D trajectories of sperms and the statistics information of the swim path, pattern, speed, etc., were revealed (Figs. 15 and 16).

Fig. 15

Dual-view lensfree 3-D tracking of human sperms. (a) The schematic diagram of the imaging system. Two partially coherent light sources (red and blue LEDs at 625 and 470 nm, respectively) are butt-coupled to multimode fibers to simultaneously illuminate the sperms at two different angles (red at 0 deg and blue at 45 deg). A CMOS sensor chip re-cords the dual-view lensfree holograms that encode the position information of each sperm. The 3-D location of each sperm is determined by the centroids of its head images reconstructed in the vertical (red) and oblique (blue) channels. (b) The reconstructed 3-D sperm trajectories. At a frame rate of 92 fps, 1575 human sperms were tracked inside a volume of 7.9 μL. The time position of each track point is encoded by its color (see the color bar) (reprinted from Ref. 98 by permission of Proceedings of National Academy of Sciences, U.S.A).

Fig. 16

Four major categories of human sperm swimming patterns. (a) The typical pattern. (b) The helical pattern. (c) The hyperactivated pattern. (d) The hyperhelical pattern. The inset in each panel represents the front view of the straightened trajectory of the sperm. The arrows indicate the directions of the sperms’ forward movement. The time position of each track point is encoded by its color (see the color bar). The helices shown in (b) and (d) are both right-handed (reprinted from Ref. 98 by permission of Proceedings of National Academy of Sciences, U.S.A).

4.3.4.

Crawling cells in matrix gels

Ferraro’s group has presented a 3-D tracking of in vitro osteosarcoma cells in a gelified collagen matrix characterized by a high morphological change.¹⁰⁰ DH-QPM was utilized to capture time-lapse sequences of cell migration at 12 fph for 15 h to record any movement and morphological change of cells. Numerical reconstruction allowed for evaluating both intensity and phase profiles in different planes. The cells were first detected from phase profiles by applying a simple thresholding filter, and this type of pure phase objects showed minimum visibility when they were in focus; therefore, the estimation of focal planes was performed by numerical scanning on the stack of reconstructed amplitude images instead for the sharpest details. Then, the axial position of the center of mass of each cell was determined from phase profiles at the focal plane.¹⁰¹ Taking into account the morphological changes of cells, the concept of minimum boundary filters¹⁰⁰ was introduced to determine the positions of cells, and it was compared with other three methods, i.e., centroid, weighted centroid, and maximum phase value, as shown in Fig. 17.

Fig. 17

3-D tracking of in vitro cells in a gelified collagen matrix. Column (a) shows three different cells with the ($X$, $Y$) positions estimated using the four methods, and the in-focus distance is reported on the top of each (a). Column (b) shows the trajectories estimated of the four methods (reprinted from Ref. 100 by permission of OSA).

In Ref. 102, to reduce the coherent noise, a partially spatial coherent illumination was utilized as the illumination source of off-axis DHM setup and an accurate equalization between the object and reference beams must be reached. A semiautomated software package performed a four-dimensional tracking of unstained HT-1080 fibrosarcoma cells migration in matrix gels on the basis of phase profiles, and the 3-D locations of a central point in the cell as a function of time were recorded simultaneously. Quantitative measures on the displacement and velocity of cells were also obtained. By numerical autofocusing and evaluation of quantitative phase images, DH-QPM was demonstrated to be a label-free and noninvasive method for tracking living cells in a 3-D environment.

4.4.

Characterization of Microfibers

Kempkes et al. have applied in-line DHM to record a hologram of a needle-shaped tilted opaque carbon fiber¹⁰³ and showed the reconstruction at different planes using Fresnel-Kirchhoff diffraction integral.⁹⁴ The intensity profiles were then obtained and converted to binary images by setting a threshold, which retained only the in-focus pixels. The combination of the stack of binary images produced a 3-D profile containing point clouds resulting from the focused pixels of a fiber at different reconstructed planes. A straight line was fit onto the point clouds, and the orientation and length of the fiber were estimated. A more complex sample of fibers randomly suspended in water was studied in the same way. Seven fiber segments were identified and the corresponding orientation and length were measured (Fig. 18). In-line DHM is shown to be an effective technique to characterize nonspherical particles in 3-D volume.

Fig. 18

3-D image of a volume of carbon fibers in suspension, viewed from two different directions and showing both the fitted lines (black lines), as well as the point clouds (colored dots) (reprinted from Ref. 103 by permission of OSA).

We have recently studied the nonwoven curved and randomly oriented microfibers in suspension.²¹ Fibers were spun onto glass coverslips from co-poly ($\mathrm{L}\text{-}\text{gluatmic}{\text{acid}}_4$, $\mathrm{L}\text{-}{\text{tyrosine}}_1$) (PLEY) dissolved in water and crosslinked as described previously.⁵⁸ The diameter of fibers was 0.2 to 5.0 μm. Figure 19(a) presents a hologram of microfiber. The angular spectrum method was applied to obtain the reconstructed amplitude image in Fig. 19(b). The microfiber appeared curved and elongated, consistent with our previous work.¹⁰⁴ The field of view was $90\times 90{\mu \text{m}}^2$ with $464\times 464\text{pixels}$. The hologram was then reconstructed from $Z=100$ to $-100\mu \text{m}$, in $-1\text{-}\mu \text{m}$ steps. Adding these 201 reconstructed images of amplitude yielded the axial projection shown in Fig. 19(c). The final image had the same number of pixels as each reconstructed image. The projected image was converted to binary by threshold and segmentation, as shown in Fig. 19(d). The binary image was then axially backprojected along the path of amplitude over the set of reconstructed images. We applied the same autofocusing algorithm on the resultant images to record the maximum intensity value at each pixel along the reconstruction direction and the axial $XY$ and depth position $Z$ of where the maximum intensity occurred at the same time. The obtained all-in-focus intensity profile [Fig. 19(e)] and the corresponding depth map [Fig. 19(f)] were then combined to generate the 3-D visualization of microfiber and a threshold on intensity was chosen to highlight the area of regional interest. However, there still existed noisy point clouds from speckle noise in the 3-D profile [Fig. 19(g)]. To address this, an average and polynomial curve fitting algorithm accounting for errors was adopted. The fitted line of the point clouds (standard deviations in the $X$ and $Z$ directions were 0.21 and 0.06 μm) in the 3-D coordinate was present in Fig. 19(h). Figures 19(i) to 19(k) show the $XY$, $XZ$, and $YZ$ views of the 3-D profile, respectively. $Z$ coordinate provides the depth information of real image of microfiber in the reconstruction volume. The length of the microfiber in 3-D volume was determined from $X$, $Y$, and $Z$ coordinates of real image of microfiber, which was 126 μm.²¹

Fig. 19

3-D profiling of stationary microfibers. The field of view of (a) to (f) is $90\times 90{\mu \text{m}}^2$ with $464\times 464\text{pixels}$. (a) Hologram. (b) Amplitude image. (c) Projection image. (d) Binary image, microfiber 1 and background 0. (e) All-in-focus intensity profile. (f) Depth position profile. (g) 3-D profile ($90\times 90\times 35{\mu \text{m}}^3$) and gray-scale representation of intensity. (h) 3-D fit ($90\times 90\times 35{\mu \text{m}}^3$). (i) $XY$ view of 3-D fit ($90\times 90{\mu \text{m}}^2$). (j) $XZ$ view of 3-D fit ($90\times 35{\mu \text{m}}^2$). (k) $YZ$ view of 3-D fit ($90\times 35{\mu \text{m}}^2$).

4.5.

Cell–Environment Interaction

When cells swim in the local environment, they exhibit movement toward or away from the stimuli, as well as interaction with the stimuli. Knowledge of this interaction is essential for understanding the impact of cell’s behavior on the process of locating food, avoiding predators, finding mates, etc.

4.5.1.

Three-dimensional flow field by swimming copepod

In-line DHM was utilized to measure the 3-D trajectory of a free-swimming copepod and the complex flow around it.¹⁰⁵ A sample of Diaptomus minutus in well water was prepared and seeded with 20 μm polystyrene spheres. The in-line DHM apparatus with He-Ne laser source consisted of two inclined mirrors on the walls of the sample volume to provide two perpendicular views of the object in the same image. The series of holograms that captured the copepod with two views were reconstructed using Fresnel-Huygens principle¹⁰⁶ with a paraxial approximation to obtain the intensity distribution. Based on the location of the traces resulting from the superposition of three successive exposures, the corresponding location of mirrored views was estimated. With each of the two views, the particle displacement was measured using cross-correlation of the intensity distribution of the three exposures,^107,108 and the lateral and vertical displacements were then combined into a 3-D velocity distribution of particles, as shown in the left panel of Fig. 20. In the reconstructed images sequence movie, the copepod was seen to sink, swim upward, and then sink again. As it sinks, a recirculating flow pattern was generated due to the movement of appendages, as illustrated in the right panel of Fig. 20. Stokeslets model^109,110 was employed to estimate the excess weight ($7.2\times {10}^{-9}\mathrm{N}$) and excess density ($6.7\mathrm{kg}/{\mathrm{m}}^3$) of the copepod, and also the propulsive force generated by appendages ($1.8\times {10}^{-8}\mathrm{N}$).

Fig. 20

Left: measured instantaneous velocity near the copepod (a) in the ambient frame of reference and (b) in the copepod frame of reference. Right: particle streaks in the copepod reference frame obtained by combining 130 appropriately shifted reconstructed images. In all cases, the dorsal and lateral views are in focus (reprinted from Ref. 105 by permission of Journal of Experimental Biology).

4.5.2.

Three-dimensional displacement of microfiber by swimming paramecium

A hologram movie of paramecium moving through a microfibers mesh was recorded by off-axis DHM at 17 fps.²¹ A cell was seen to swim through fibers and cause obvious displacement of fibers. Two hologram frames showing cell approaching to fibers [Fig. 21(a)] and swimming away after pulling fibers [Fig. 21(b)] were extracted. The field of view was $200\times 200{\mu \text{m}}^2$ with $768\times 768\text{pixels}$. The difference hologram [Fig. 21(c)] clearly showed the displacement of the fiber due to the movement of cell and the trajectory of cell. The reconstructed amplitude images from angular spectrum method at $Z=0$ and $Z=-8\mu \text{m}$ are shown in Figs. 21(d) and 21(e), respectively. Different parts of the fiber appeared to be in focus at different reconstructed planes. Autofocusing algorithm at the reconstruction volume from $Z=60$ to $-70\mu \text{m}$ was applied and the all-in-focus intensity and depth position information were obtained in Figs. 21(f) and 21(g).

Fig. 21

3-D displacement of microfiber by swimming paramecium. The field of view of (a) to (h) and (j) is $200\times 200{\mu \text{m}}^2$ with $768\times 768\text{pixels}$. (a) Hologram 1. (b) Hologram 2. (c) Difference of holograms 1 and 2. (d) Amplitude reconstructed at $Z=0$. (e) Amplitude reconstructed at $Z=-8\mu \text{m}$. (f) All-in-focus intensity profile. (g) Depth position profile. (h) All-in-focus intensity of the first track of the displacement of fiber. (i) 3-D fitted line of (h) ($200\times 200\times 20{\mu \text{m}}^3$). (j) All-in-focus intensity of the second track of the displacement of fiber. (k) 3-D fitted line of (j) ($200\times 200\times 20{\mu \text{m}}^3$). (l) 3-D fitted line of the displacement of fiber ($200\times 200\times 20{\mu \text{m}}^3$). (m) $XY$ view of (l) ($200\times 200{\mu \text{m}}^2$). (n) $XZ$ view of (l) ($200\times 20{\mu \text{m}}^2$). (o) $YZ$ view of (l) ($200\times 20{\mu \text{m}}^2$). For better visualization, the scales of $X\text{-}Y$ and $Z$ coordinate are not in ratio.

To best visualize the displacement of the fiber in 3-D volume, we analyzed the two tracks of the displacement of the fiber independently. For brevity, most of the following descriptions referred to the first track. Numerical segmentation and thresholds were applied to Fig. 21(f), and the all-in-focus intensity image containing only one track of fiber was generated in Fig. 21(h). Combined with the obtained depth position profile [Fig. 21(g)] and fitting algorithm accounting for errors, the fitted line of the point clouds (standard deviations in $X$ and $Z$ directions were 1.27 and 0.27 μm) in 3-D coordinate is present in Fig. 21(i). Similar results for the second track of fiber were shown in Figs. 21(j) and 21(k), and the standard deviation of its corresponding point clouds in the $X$ and $Z$ directions were estimated to be 0.80 and 0.26 μm. The two tracks of the fiber were then plotted in one 3-D coordinate [Fig. 21(l)], and Figs. 21(m) to 21(o) show the $XY$, $XZ$, and $YZ$ views of the 3-D plot, respectively. The length and the displacement of the fiber were estimated to be 210.1 and 11.1 μm. A movie of 3-D displacement of the fiber within 0.5 s period was also presented. The 3-D displacement of an individual microfiber due to interaction with paramecium cell has been measured as a function of time at subsecond and micrometer level.²¹

To the best of our knowledge, this is the first quantitative profiling and tracking by DHM of the curvature and displacement of individual microfiber by swimming cells in three dimensions. Displacement could vary with cell type, physiological state of the cells, microfiber preparation, etc. The prospects seem excellent for DHM for particle flow analysis and 3-D imaging of randomly oriented microfibers, in cases where quantitative measurement of characteristics (size, length, orientation, speed, displacement, etc.) is of great interest.