2.1.

Hyperspectral Multiphoton Microscopy System

The system is a custom-built multiphoton microscope, as shown in Fig. 1 and discussed in the Supplemental Material in greater detail. Briefly, a femtosecond laser with a center frequency at 785 nm and a pulse length of 15 fs is used for illumination. After passing a dispersion precompensation unit, the beam is sent through a spatial filter before being redirected by a galvanometer scanner and two relay lenses. The beam passes a longpass filter, is reflected by a dichroic mirror, and is focused with a water dipping objective. Emitted light is collected through the same objective, filtered by the dichroic mirror and a 720-nm shortpass filter, before being focused into a fiber with a $600\text{-}\mu \mathrm{m}$ core. The signals can then either be detected by a photomultiplier tube (PMT) for quick imaging or a spectrometer (QE Pro, Ocean Insight, Orlando, FL) for hyperspectral imaging. The setup is controlled by a custom LabVIEW program. For hyperspectral imaging, an average power of 12 mW on the samples was used with a pixel dwell time of 8 ms, whereas imaging with the PMT was conducted at 2.3 mW with 100-ns pixel dwell time to avoid photobleaching. The spectral shape was not significantly influenced by photobleaching, as shown in Fig. S5(a) in the Supplementary Material.

Fig. 1

Setup of the custom-built two-photon fluorescence microscope. As excitation source, we use a mode-locked Ti:sapphire laser with a center wavelength at 785 nm for illumination. The emitted fluorescence light is collected in epi-detection by the same objective and then focused into a fiber. The signals can be detected by a PMT for quick imaging or a spectrometer for hyperspectral imaging.

2.2.

TPEF Spectra Processing

High-resolution images of the samples are recorded for navigation using the PMT. Hyperspectral TPEF images are then recorded with reduced resolution ($64\times 64{\mu \mathrm{m}}^2$, $64\times 64{\mathrm{pixels}}^2$). The spectra of the entire TPEF images are averaged over all pixels. Given by the selected spectrometer slit, the spectral resolution of the system is 10 nm, while the spectrometer has a pixel size covering $<0.5\mathrm{nm}$ per pixel. Therefore, the recorded spectra were smoothed with a sliding window of 10 nm width. All spectra are background subtracted and compensated for a temperature offset using Spectragryph,²² as described in Fig. S1 in the Supplementary Material. The investigated spectral range is limited from 425 to 650 nm in which a linear system response function has been confirmed and laser excitation is excluded, as shown in Figs. S2 and S3 in the Supplementary Material, respectively.

Three samples of each cancer model and cell line (detailed below) are investigated, and their mean spectra are calculated and used as an input for the algorithm in the presented analysis. Additionally, spectra from the literature are investigated. The TPEF spectra of NAD(P)H and FAD as reported by Huang et al.¹⁷ are extracted using WebPlotDigitizer.²³

2.3.

Algorithm for Optimal Filter Determination

The presented algorithm takes two spectra as input and quantifies, for all possible spectral band combinations, the ability to separate and thus classify the two spectra, corresponding to the achievable contrast of an imaging system, and the resulting signal-to-noise ratio (SNR) normalized to the maximum possible SNR of the system. It is expected that a trade-off between these two parameters will have to be made, as for many known endogenous fluorophores, fluorescence emission spectra are overlapping.

Figure 2 shows an overview of the algorithm. To calculate the separation of a given filter setup, ideal bandpass filters are assumed, and the probability mass function (PMF) of each spectrum is summed in the spectral bands, from ${\lambda }_{\mathrm{SB}1,\mathrm{start}}$ to ${\lambda }_{\mathrm{SB}1,\mathrm{stop}}$, as shown for band 1 in spectrum 1 in the following equation:

Fig. 2

Infographic visualizing the underlying algorithm. (a) Two input spectra (orange and dark green) are separated into spectral bands. (b) The ratios of the power falling into these spectral bands (yellow and light green) are calculated separately for each spectrum, and the two ratios are then divided to quantify the separation. (c) The separation is compared against the geometric mean of the four band’s powers, which serves as a measure for the expected SNR of the entire system.

Subsequently, the ratio of the two spectral bands is created within each spectrum to represent the relative change within the signal independent from its absolute intensity. This reduces the influences of variations in the system performance.

The ratios of the spectral bands in the two spectra are then divided to calculate the separation.

The inverse is taken if the ratio of ratios is smaller than 1, as in this case, the bands only need to be “swapped” to provide a high separation.

To quantify the relative signal collection efficiency using a given filter setup, the geometric mean of the power in the four spectral bands is calculated. Using the geometric mean of the four spectral bands (two in each spectrum), the algorithm prefers spectral bands with balanced signal strengths in both spectra, avoiding small (in bandwidth) or weak (in signal) bands that would have very low SNRs. The resulting relative signal collection efficiency will range from 0 to 0.5, as in the most balanced case each spectral band would contain half of the total signal in each spectrum. To predict the SNR achievable with the determined spectral bands, the SNR of the entire recorded spectrum can be scaled by this factor.

Both the separation and the relative signal collection efficiency are calculated for all permutations of the four wavelengths ${\lambda }_{\mathrm{band}1:\mathrm{start}},{\lambda }_{\mathrm{band}1:\mathrm{end}},{\lambda }_{\mathrm{band}2:\mathrm{start}},\mathrm{and}{\lambda }_{\mathrm{band}2:\mathrm{end}}$ for which the two spectral bands do not overlap, i.e., where ${\lambda }_{\mathrm{band}1:\mathrm{end}}<{\lambda }_{\mathrm{band}2:\mathrm{start}}$. The resulting values for all possible spectral bands can then be visualized in a scatter plot, with the relative signal collection efficiency on the $x$ axis and the separation on the $y$ axis (see Fig. 5 and subsequent figures).

TPEF emission spectra (and, in fact, any type of spectra) can be imported and all possible filter setups within a given detection range are calculated. Custom filter setups, such as filters suggested in literature or readily available filters, can be highlighted in the graph. By selecting spots on the graph, their filter combinations are revealed.

The MATLAB code is available at https://gitlab.gbar.dtu.dk/biophotonics/optimizeSpectralChannels/.

2.4.

Cell Models

Cancerous and normal epithelial colon cells, namely human epithelial colon adenocarcinoma cells (HT29) and human fetal epithelial colon cells (FHC), were cultivated at Bioneer A/S, Denmark, as 2D cultures and spheroids. Spheroids are known to mimic an in vivo cell behavior concerning spatial configuration and signaling pathways.²⁴ For the formation of spheroids, 25,000 cells (subcultured for 7 days before trypsinization) and $100\mu \mathrm{l}$ of cell medium were used per well in an ultralow attachment round bottom 96-well plate (Corning), which was spun at 500 rpm for 10 min. The 2D cultures were seeded with 25,000 cells and $2\mu \mathrm{l}$ per petri dish. The cell medium used was Dulbecco’s Modified Eagle’s Medium with 4.5 g l-glucose per liter, 10% fetal bovine serum (20% FBS for FHC cells), 1% of penicillin/streptomycin solution and 1% of L-glutamine. 2D cultures and spheroids for biomarker detection were kept in a normal oxygen incubator (37°C, 5% ${\mathrm{CO}}_2$, 20% ${\mathrm{O}}_2$) for 9 days and then transferred to a low oxygen incubator (37°C, 5% ${\mathrm{CO}}_2$, 5% ${\mathrm{O}}_2$) to create a cancer characteristic tissue environment and harvested after another 3 days to avoid excess buildup of metabolic waste. The development of spheroids was monitored using optical coherence tomography (OCT) to optimize the sample consistency and time of harvesting.

For the imaging of 2D cultures, three locations for TPEFM acquisition were chosen randomly in one petri dish for each cell type. For spheroids, three samples of each cell type were imaged, and locations were chosen centrally, about two cell layers deep to yield a homogenous layer of cells while avoiding debris on the surface and necrotic regions deeper inside the spheroid. Approximate imaging areas are shown in Fig. 3. A further set of 2D cultures and a 2D co-culture using both cell lines have been grown for 7 days in a normal oxygen and 4 days in a low-oxygen incubator to demonstrate the application.

Fig. 3

(a) Cancerous (HT29) and (b) normal (FHC) epithelial colon cells are grown as 3D spheroids. Spheroid formation is observed using a commercial OCT system (TELESTO-II, THORLABS), and approximate TPEFM imaging locations are marked (arrow).

3.1.

Two-Photon Excited Fluorescence Emission Spectra of Live Cell Cultures

The obtained TPEF spectra from 2D cell cultures are shown in Fig. 4. TPEFM images of cancerous (HT29) and normal (FHC) cells in 2D cultures (PMT detection, entire visible spectrum) are shown in Fig. 4(a). Figure 4(b) shows the corresponding TPEF spectra summed over the full respective images, normalized to the area underneath the curve for better visual comparison. Figure 4(a) shows the cancerous (HT29) or normal (FHC) cells that are normalized using the PMF for a better visual comparison and plotted in Fig. 4(b) along with the mean spectra. Autofluorescence spectra for cancerous cells are blueshifted when compared to their normal counterparts, which is in agreement with findings in cell cultures²⁵ and clinical studies.²⁶ To ensure the applicability of the presented method, variations within a sample group need to be smaller than the difference between the sample groups. The shift between FHC and HT29 spectra has been observed despite small changes in culturing parameters for 2D cultures and spheroids as shown in Figs. S5(d) and S5(b) in the Supplementary Material, respectively. A relative variability is defined as described in the Supplemental Material and calculated measurements on 2D cultures are shown in Fig. 4 and measurements on two additional samples are included in Fig. S5(d) in the Supplementary Material. Briefly, the center of gravity varies around 12% for FHC cell cultures and 10% for HT29 cell cultures, compared to the general shift between the cell types. For the spheroid cancer models, the measured spectra follow a similar spectral shift, as shown in Fig. 6(c). Individual measurements are shown in Fig. S5(c) in the Supplementary Material and their relative variabilities calculate to 29% for HT29 and 3% for FHC cells.

Fig. 4

(a) TPEFM images of cancerous (HT29) and normal (FHC) cells in 2D cultures (PMT detection and entire visible spectrum). (b) Corresponding TPEF spectra summed over the full respective images, normalized to the area underneath the curve for better visual comparison.

3.2.

NAD(P)H and FAD Autofluorescence

The algorithm presented above is first verified using TPEF emission spectra of NAD(P)H and FAD data as an example of a possible biomarker from the literature,¹⁷ as shown in Fig. 5(a). The gray area in Fig. 5(b) represents the different spectral band combinations. For every relative signal collection efficiency, the respective maximal achievable separation lies on the green line. The combination of a 410- to 490-nm and 510- to 650-nm spectral band as used in Ref. 17, marked with a circle, yields a relative signal collection efficiency of 0.26 and a separation of about 65.6. A filter setup resulting in the same relative signal collection efficiency (within 0.1% tolerance) and 23% increased separation is determined and marked with a circle: the corresponding spectral bands are 401 to 483 nm and 489 to 641 nm. The spectral bands of both setups are marked in Fig. 5(a). Due to the relatively small overlap of spectra in Fig. 5(a) (as compared to real autofluorescence spectra), a much stronger separation could potentially be reached when choosing the narrow spectral bands 422 to 464 nm and 515 to 635 nm outside of the overlapping parts of the spectra. A fourfold increase in separation of 320 as opposed to 81 is calculated in this case for an SNR reduced by a factor of 2.

Fig. 5

Simulations of spectral bands on data from literature.¹⁷ (a) Spectra of NAD(P)H and FAD are normalized using the PMF and shown as used for the simulation. Spectral bands of bandpass filters (410 to 490 nm and 510 to 650 nm, circle) and improved spectral bands (401 to 483 nm and 489 to 641 nm, triangle) are marked. Additionally, spectral bands with a high separation and reduced relative signal collection efficiency are marked (square). (b) Simulated combinations of spectral bands are plotted with their separation (scaling the contrast of a system) and their relative signal collection efficiency (scaling the SNR of a system). The combinations of spectral bands shown in (a) are marked and favorable combinations yielding a maximal achievable separation are highlighted.

3.3.

Cell Model Autofluorescence

Measured autofluorescence spectra of live 2D cell cultures and spheroids, shown with their average spectrum in Figs. 6(a) and 6(c), respectively, are used in the following as inputs for the algorithm to determine optimal detection bands. Evaluating the simulated combinations of spectral bands on 2D cultures in Fig. 6(b), the 410- to 490-nm and 510- to 650-nm bands suggested in Ref. 17 already yield a separation of 1.97, very close to the best possible value of 2 with optimized spectral bands at the same relative signal collection efficiency of 0.41 given by said bands. The same is true for the 3D cultures. The separation of the spheroid cell models can neither be improved by more than 0.07, from 1.59 to 1.66, when keeping the relative signal collection efficiency the same at 0.4, as shown by the inset in Fig. 6(d). It can be seen that, for both cell models and for measurements of NAD(P)H and FAD in solution (above), the determined optimal spectral bands at the same relative signal collection efficiency are shifted (band 1) or expanded (band 2) toward the blue when compared to the 410- to 490-nm to 510- to 650-nm bands suggested by the literature. The expansion of band 2 is considerably stronger in the spheroid cell models than in the 2D cell cultures.

Fig. 6

TPEF spectra from different cancer models are investigated and spectral bands with optimized separation are visualized. Autofluorescence spectra of cancerous (HT29) and noncancerous (FHC) cell cultures grown in (a) 2D and (c) spheroid cancer models are shown normalized using the PMF as used for the simulation. Spectral bands of bandpass filters (410 to 490 nm and 510 to 650 nm, circle) and improved spectral bands (triangle) are marked for the investigation of cancer models. Simulated combinations of spectral bands are plotted for (b) 2D and (d) spheroid cancer models with their separation (scaling the contrast of a system) and their relative signal collection efficiency (scaling the SNR of a system). The combinations of spectral bands shown in (a) and (c) are marked and favorable combinations of spectral bands yielding a maximal achievable separation between the measured spectra are highlighted.

In both cancer models, a significantly improved separation can be achieved by compromising in relative signal collection efficiency. Combinations of spectral bands with increased separation and halved relative signal collection efficiency are marked (square) in Fig. 6(b) for 2D cell autofluorescence and Fig. 6(d) for spheroid autofluorescence. In the case of 2D cell cultures, a separation of 2.7 as opposed to 2 can be achieved compared to the literature bands. Correspondingly, a halved relative signal collection efficiency increases the separation of spheroid autofluorescence from 1.7 to 2.3. In 2D cell cultures and spheroids alike, a better separation requires a particularly narrow spectral band in the blue part of the spectrum and a broader band in the red part of the spectrum.

3.4.

Imaging Application

To demonstrate the capabilities of the presented method, a mosaic composed of 12 hyperspectral images from an FHC and HT29 2D co-culture is acquired with a resolution of 256 pixel and a pixel dwell time of 20 ms for each frame. Figure 7(a) shows an image composed of two channels generated by integrating over the initial¹⁷ spectral bands (cyan: 410 to 490 nm and red: 510 to 650 nm). In Fig. 7(b), which is composed by integrating over the optimized spectral bands calculated above (cyan: 407 to 450 nm and red: 535 to 650 nm), an increased noise is visible due to the narrower spectral bands. Despite this, a stronger color contrast is visible, making the FHC cells appear redder and thus more detectable. In both images, both channels are scaled to the same thresholds relative to their histogram (lower limit: mean $-2\sigma$, upper limit: mean $+5\sigma$, $\sigma$: standard deviation). To verify these findings numerically and to provide comparability to the commonly used redox ratio, we divide the red channel by the sum of both channels to create ratiometric images of the initial spectral bands as shown in Fig. 7(c) and the optimized spectral bands are shown in Fig. 7(d). A $2\times 2$ binning is performed to increase the SNR of these images. The SNR of all images is calculated as described by Nylk et al.²⁷ in the frequency domain using the 2D fast Fourier transform, by considering features with a spatial frequency higher than four times the theoretical system resolution as noise, while features at lower frequencies were considered to be signal. The SNR is then calculated as the ratio of the integrals over the respective power spectra. While the SNR of the individual image channels decreases by 2.6% (red) and 42.6% (cyan), the calculated SNR of the ratiometric image is increased by 13.1%. Furthermore, a statistical analysis in the form of a Student’s $t$-test is performed on 12 hyperspectral images of 2D FHC and HT29 monocultures to quantify an improved confidence in differentiating these specimen populations. Aforementioned spectral bands are used to calculate a redox ratio for ease of comparison as detailed in Table S1 and Fig. S6 in the Supplementary Material. Although the difference of the mean redox values between cell cultures only increased by 3.7%, the lower standard deviation of 31.9% (FHC) and 8.3% (HT29) leads to an improved separation when using optimized spectral bands, as shown by combining these values in a $T$-ratio, which increases by 31.5%.

Fig. 7

(a) and (b) Color-coded images are calculated from hyperspectral images of a 2D co-culture (HT29 and FHC cells). Images are composed by integrating over (a) initial spectral bands: 410 to 490 nm (cyan), 510 to 650 nm (red) and (b) optimized spectral bands: 407 to 450 nm (cyan), 535 to 650 nm (red). (c) and (d) Ratiometric images are calculated by dividing the red channel by the sum of both channels for (c) the initial and (d) optimized bands. To improve SNR, a $2\times 2$ binning is applied.