2.1.

Multispectral Imaging System

The images in this study were acquired using an IVIS^® Spectrum Imaging System (Caliper Life Sciences/Xenogen, Alameda, California) as shown in Fig. 1 . It supports in-vivo spectral imaging for both fluorescent and bioluminescent probes. This instrument uses a $2048\times 2048\text{pixel}$ , $26\text{-}\mathrm{mm}$ -wide, high-sensitivity back-thinned charge coupled device (CCD) camera thermoelectrically cooled to $-90°\mathrm{C}$ to minimize the dark current and the associated thermal noise. The CCD has high quantum efficiency in the range of $400\text{to}950\mathrm{nm}$ and thus covers the visible (VIS) and near-infrared (NIR) range of interest of optical imaging. Light collection is accomplished with an f/1 lens with $6\times$ zoom capability. The minimum detectable radiance for the system is specified at less than $70\text{photons}∕\sec ∕{\mathrm{cm}}^2∕\mathrm{sr}$ , enabling detection of weak signals from deep tissue.

Fig. 1

Illustration of the IVIS Spectrum imaging system.

For fluorescence imaging, the instrument uses a $150\text{-}\mathrm{W}$ quartz tungsten halogen lamp with broadband emission from $400\text{to}1000\mathrm{nm}$ . The lamp output is delivered to a 12-position excitation filter wheel using a fiber optic bundle. The excitation wheel includes ten $30\text{-}\mathrm{nm}$ bandpass filters spaced every $35\mathrm{nm}$ from $430\text{to}745\mathrm{nm}$ . The emission light is focused on the CCD through the lens system with an integrated 24-position emission filter wheel. The emission wheel is installed with 18 $20\text{-}\mathrm{nm}$ bandpass filters spaced every $20\mathrm{nm}$ from $500\text{to}840\mathrm{nm}$ . The spacing and bandwidth of excitation and emission filters are chosen at a medium level to achieve a reasonable spectral resolvability without sacrificing signal per channel and spectral range. More than one hundred excitation and emission filter combinations are available in the VIS and NIR range for spectral unmixing.

For fluorescence imaging, the instrument can switch between reflectance and transillumination modes by rotating a mirror, as illustrated in Fig. 1. Reflectance mode refers to the conventional modality (epifluorescence mode) that the excitation light is projected to the top surface of the object, where the emitted fluorescence light is collected. Reflectance mode favors the reporters near the surface, since the relative strong autofluorescence background generated from the top surface can overwhelm the fluorescent signals of the deeper reporters. In transillumination mode, the excitation light is delivered to an $x\text{-}y$ translation assembly under the stage and focused to the bottom side of the object. An anodized aluminum plate with a $5\text{-}\mathrm{mm}$ grid of $2\text{-}\mathrm{mm}$ holes is used to hold the object and guide the light through. The autofluorescence contribution is often reduced, since most autofluorescence is generated from the bottom surface and is absorbed as it travels to the top surface. When imaging a fluorophore located at the center of the subject, illuminating from the bottom would create the same amount of fluorophore signal to the detector as illuminating from the top, generating much less autofluorescence contribution. Therefore, the transillumination mode offers improved access to the deep fluorophores during in-vivo imaging.

To provide a consistent measurement against different imaging conditions so that signals are traceable over time and on different instruments, the imaging system is absolutely calibrated for spectral radiance traceable to the National Institute of Standards and Technology (NIST).^{7, 21} This calibration provides the conversion of CCD electron counts to isotropic radiance on the object surface by taking into account losses through the optics and aperture (f-stop), field of view (FOV), and camera settings. For epifluorescence imaging particularly, the power and distribution of the excitation light are recorded so that measurements can be normalized to the incident excitation intensity profile to report how efficiently the target yields the fluorescence. Instrument calibration is critical to the quantification of images with different imaging conditions, as is typically encountered during longitudinal studies.

2.2.

Multivariate Curve Resolution Algorithm

2.2.1.

General Multivariate Curve Resolution Alternating Least-Square Scheme

The basic framework of MCR-ALS is shown in Fig. 2 . Initially the number of spectral components (number of fluorophores) $k$ is specified to determine the size of the bilinear problem, and an initial estimate for either $\mathbf{C}$ or $\mathbf{S}$ in Eq. 1 is made. We then employ an iterative method for solving two alternating least-square problems, i.e., minimization of Eq. 2 over $\mathbf{S}$ for given $\mathbf{C}$ , and minimization of Eq. 2 over $\mathbf{C}$ for given $\mathbf{S}$ . To help converge to a unique and realistic solution, additional constraints can be added during iterations. These constraints come from knowledge about the spectra such as nonnegativity or low and high pass filters to force the spectra to zero at certain wavelengths. Constraints reduce the solution space by enforcing some boundaries and help solve the rotation and scaling ambiguity inherent in the bilinear model. Here, the rotation and scaling ambiguity refers to the fact that any arbitrary $k$ by $k$ orthogonal matrix $\mathbf{Q}$ ( ${\mathbf{Q}}^T\mathbf{Q}=\mathbf{I}$ , where $\mathbf{I}$ is identity matrix) can result in another suitable solution $\mathbf{C}{\mathbf{Q}}^T$ and $\mathbf{Q}\mathbf{S}$ for the bilinear model. A special case of such an orthogonal matrix $\mathbf{Q}$ is a diagonal matrix often known as the intensity ambiguity. Usually this cannot be solved by setting the constraints, and thus a normalization procedure should be enforced on either $\mathbf{C}$ or $\mathbf{S}$ throughout the iteration. The convergence is achieved when the absolute change of the residual norm $e^2$ is below a threshold (typically 0.1%). After convergence, both $\mathbf{C}$ and $\mathbf{S}$ are determined.

Fig. 2

Flowchart of MCR-ALS for in-vivo fluorescence imaging

In some cases, the number of spectral components is not known. To determine the number of spectral components, one approach is based on the principal component analysis (PCA), i.e., examining the variance of data explained by a selected number of principal components.^{11, 22} Since principal components are sorted in terms of the variance they explain, when an additional principal component only affects a small margin of the total explained variance, the selected principal components have accounted for the majority signals and the rest mainly contribute to the random noise. Principal components are orthogonal to each other, and though they have no biological meanings, they imply the independent components present in the data.

3.1.

In-Vitro Experiments

The ability to separate multiple fluorophores, especially when they have significant spectral overlap, was tested in an in-vitro experiment. Two fluorescent dyes, Alexa Fluor^®(AF) 680 and 750 and two quantum dots (QD) 700 and 800 (Invitrogen, Carlsban, California), were diluted and placed in a 96-well plate. Each fluorophore occupied one well. Raw images were taken using ten emission filters from $660\text{to}840\mathrm{nm}$ , spaced every $20\mathrm{nm}$ , and excited in reflectance mode at $605\mathrm{nm}$ . The image overlays where fluorescent radiance images are superimposed on background grayscale photographs using rainbow pseudocolor are shown in Fig. 3a . From the top row to the bottom row, the fluorophores are AF680, QD700, QD800, and AF750, respectively. Four components were chosen to unmix, since the autofluorescence background was at a negligible level and signals were mostly contributed by four fluorophores. Spectral unmixing was performed with default settings that include non-negativity constraints.

Fig. 3

AF680, QD700, AF750, and Q800 in a well plate. Each fluorophore occupies one well. (a) Fluorescent images from $660\text{to}840\mathrm{nm}$ every $20\mathrm{nm}$ excited at $605\mathrm{nm}$ . (b) Unmixed distribution maps for AF680, QD700, AF750, and QD800, respectively. The last column is a composite RGB image where four fluorophores are represented in green, blue, magenta, and red color channels, respectively. (c) Unmixed spectra for AF680, QD700, AF750, and QD800.

Unmixed distribution maps are shown in Fig. 3b, where each column is displayed with its own pseudocolor scale; the unmixed images can be quantified to obtain the signal distribution of the fluorophore in the raw image. A composite image is also created to better visualize unmixed fluorophores in a single RGB color image. In this case, the distribution maps of AF680, QD700, QD800, and AF750 are combined to create a false color image shown as the last image in Fig. 3b by putting them into green, blue, magenta, and red channels, respectively. Figure 3b indicates that four fluorophores are successfully separated from the raw images where signals show a significant overlap. Improved contrast and clarity are obtained in the individual distribution maps. The unmixed spectra are shown in Fig. 3c and are similar to the published spectra for these fluorophores. Quantum dots and Alexa Fluor^® dyes can be differentiated even though they have similar peak emission wavelengths.

3.2.

Multiple Reporters in Vivo

This experiment gives an example of multispectral imaging of a mouse containing subcutaneous injections of ${10}^{14}\text{molecules}$ of two fluorescence probes, AF680 and AF750, using IVIS Spectrum. AF680 was injected near the neck and AF750 was injected at the lower body. Figure 4a shows the raw spectral images in units of efficiency acquired using ten emission filters from $660\text{to}840\mathrm{nm}$ and excited in the reflectance mode at $605\mathrm{nm}$ . Clearly tissue autofluorescence contributed substantial signals in addition to the dyes. We assume the tissue autofluorescence is a single spectral component, and therefore selected three components to unmix. The unimodality constraint was used on the AF750 to guide the unmixing. Figure 4b shows the unmixed distribution maps for three components, i.e., autofluorescence, AF680, and AF750. Figure 4c shows the unmixed in-vivo spectra for these components. The composite image is shown as the last image in Fig. 4b. Compared to the in-vitro spectra of AF680 (cyan line) and AF750 (magenta line), the in-vivo AF680 spectrum (blue line) shows more significant red shift than that of AF750, which can be explained by the steeper absorption gradient with respect to wavelength in the shorter wavelength range. To demonstrate the benefit of our approach over the conventional method using the in-vitro spectral library, we used the in-vitro spectra of AF680 and AF750 together with the autofluorescence spectrum to perform the linear decomposition. The results are shown in Fig. 4d. It is clear that directly using the in-vitro spectra leads to severe cross talk among components in this experiment.

Fig. 4

Subcutaneous injection of AF680 (top) and AF750 (bottom) dyes. (a) Raw fluorescent images from $660\text{to}840\mathrm{nm}$ every $20\mathrm{nm}$ excited at $605\mathrm{nm}$ . (b) Unmixed distribution maps for autofluorescence, AF680 and AF750, respectively, and the composite image where autofluorescence, AF680 and AF750, are shown in green, blue, and red channels respectively. (c) Unmixed spectra for autofluorescence (green), AF680 (blue), and AF750 (red). In-vitro emission spectra of AF680 (cyan) and AF750 (magenta) are plotted in comparison to demonstrate in-vivo spectral shift due to the tissue absorption and scattering. (d) Linear decomposition results of unmixed autofluorescence, AF680 and AF750, respectively, if using in-vitro emission spectra of AF680 (cyan) and AF750 (magenta).

3.3.

Colocalized Reporters

In certain animal experiments, reporters can be physically colocalized, so it is important to examine the efficacy of the unmixing algorithm in such a situation. In this experiment, two polyethylene tubes containing Alexa Fluor^® dyes (left side: a mixture of ${10}^{15}$ AF680 and ${10}^{15}$ AF750; right side: ${10}^{15}$ AF680) were implanted next to the kidneys in a mouse. Figure 5a shows the raw spectral images in units of efficiency acquired using eight emission filters from $700\text{to}840\mathrm{nm}$ using two excitation filters. The first four images were excited at $640\mathrm{nm}$ , but the last four images were excited at $710\mathrm{nm}$ to improve the AF750 signals. Since tissue autofluorescence was not significant, only two components were selected to unmix. A high pass filter at $700\mathrm{nm}$ was used for AF750 to help the unmixing. Figure 5b shows the unmixed distribution maps for AF680 and AF750. The result matched the expectation, in that AF680 was resolved on both sides of the mouse but AF750 only appeared on right side. The yellowish mixture in the composite image corresponds to the mixing of green and red channels representing AF680 and AF750, respectively. Unmixed spectra in Fig. 5c shows well separated two spectral profiles.

Fig. 5

Implants of tubes containing ${10}^{15}\text{molecules}$ of AF680 (right and left kidney) and AF750 (right kidney). (a) Raw fluorescent images from $700\text{to}840\mathrm{nm}$ every $20\mathrm{nm}$ . The first four images were excited at $640\mathrm{nm}$ and the last four images were excited at $710\mathrm{nm}$ . (b) Unmixed distribution maps for AF680 and AF750, respectively, and the composite image where AF680 and AF750 are shown in green and red channels, respectively. (c) Unmixed spectra for AF680 and AF750.

3.4.

Transillumination

This experiment gives an example of the multispectral imaging of a mouse implanted with a polyethylene tube containing ${10}^{14}$ quantum dots (QD) 805 in the transillumination mode. The tube was placed near the left kidney and the transillumination source was positioned approximately below the tube. Figure 6a shows the raw spectral images in units of radiance acquired using six emission filters from $720\text{to}820\mathrm{nm}$ and excited at $675\mathrm{nm}$ . In the first few images when QD800 has low emission, tissue autofluorescence is strong. Figure 6b shows the unmixed distribution maps for two components, i.e., autofluorescence and QD800. The unmixed autofluorescence pattern is similar to the light transmission. After unmixing, QD800 is successfully isolated. Figure 6c shows the unmixed in-vivo spectra for autofluorescence and QD800.

Fig. 6

(a) Raw fluorescent images of QD800 tube implant from $720\text{to}820\mathrm{nm}$ every $20\mathrm{nm}$ excited at $675\mathrm{nm}$ in transillumination mode. The position of the transillumination source is marked by a cross. (b) Unmixed distribution maps for autofluorescence and QD800, respectively, and the composite image where autofluorescence and QD800 are shown in green and red channels, respectively. (c) Unmixed spectra for autofluorescence and QD800.

3.5.

Weak Sources

In previous examples, fluorophores of interest are relatively strong compared to tissue autofluorescence. Figure 7 gives an example to test the situation when tissue autofluorescence dominates. $4.8\times {10}^{12}\text{molecules}$ AF680 were sealed in the tip of a $1∕8\mathrm{in.}$ -diam rod and the rod was placed about $11\mathrm{mm}$ deep underneath the low back of a phantom mouse (Caliper Life Sciences, Alameda, California). The phantom mouse was imaged using six emission filters from $640\text{to}740\mathrm{nm}$ and excited in the reflectance mode at $605\mathrm{nm}$ . The dye barely can be distinguished from the raw images shown in Fig. 7a, but is clearly separated in the unmixed maps shown in Fig. 7b. Here a high pass filter at $640\mathrm{nm}$ for AF680 was applied to guide the unmixing algorithm.

Fig. 7

${10}^{12}\text{molecules}$ of AF680 implanteed $11\mathrm{mm}$ deep. (a) Raw fluorescence images from $640\text{to}740\mathrm{nm}$ every $20\mathrm{nm}$ excited at $605\mathrm{nm}$ . (b) The unmixed distribution maps for autofluorescence and AF680, and the composite image where autofluorescence and AF680 are shown in green and red channels, respectively. (c) Unmixed spectra for autofluorescence and AF680.

4.1.

Quantification

Spectral unmixing outputs the distribution maps and the component spectra. To use the unmixing results to quantify the fluorophores, one should first understand the meaning of those outputs. Unmixing mathematically is a decomposition process as

It also should be recognized that the signal distribution map bears a complex relationship to the fluorophore concentration map, not just because the fluorescence yield varies with excitation and emission wavelength, but also because the light transport is determined by the surrounding tissue. It is possible to correct the quantum yield with calibration, but accounting for the light transport is much more challenging and usually requires solving a complex 3-D model for photon diffusion. Therefore, the distribution map should not be regarded as a measure of absolute fluorophore concentration, but as a tool to compare fluorescence sources at similar imaging conditions and comparable depth.

4.2.

Scanning Excitation Filters

Most typically, emission filters are used to scan through the emission peaks of multiple fluorophores when performing spectral unmixing. All the previous examples shown in this work utilize emission filter scans primarily. However, the choice of scanning emission filters is relatively arbitrary. There may be situations where scanning excitation filters, or scanning both excitation and emission filters, is desirable. One example would be separating quantum dots from an organic dye with a similar emission spectrum. Since the quantum dots and organic dyes have very different excitation spectra, it would be preferable to scan excitation filters over emission filters. An example of unmixing using excitation filters is shown in Fig. 8, where we unmix the same target used in Fig. 7 and achieve similar results.

Fig. 8

Phantom model from Fig. 7 with an excitation filter scan. (a) Raw fluorescence images at $720\mathrm{nm}$ excited from $570\text{to}675\mathrm{nm}$ every $35\mathrm{nm}$ . (b) The unmixed distribution maps for autofluorescence and AF680, and the composite image where autofluorescence and AF680 are shown in green and red channels, respectively. (c) Unmixed excitation spectra for autofluorescence and AF680.