Marcus T.
Cicerone†
* and
Charles H.
Camp
Jr.
100 Bureau Drive, Mail Stop 8543, Gaithersburg, MD 20899, USA. E-mail: cicerone@nist.gov; Fax: +1 301 975 4977; Tel: +1 301 975 8104
First published on 10th October 2017
Histopathology plays a central role in diagnosis of many diseases including solid cancers. Efforts are underway to transform this subjective art to an objective and quantitative science. Coherent Raman imaging (CRI), a label-free imaging modality with sub-cellular spatial resolution and molecule-specific contrast possesses characteristics which could support the qualitative-to-quantitative transition of histopathology. In this work we briefly survey major themes related to modernization of histopathology, review applications of CRI to histopathology and, finally, discuss potential roles for CRI in the transformation of histopathology that is already underway.
Over the past several decades, a number of approaches have been pursued to increase the amount of quantitative information extracted from histology samples, and thus support improved analytical performance of histopathology approaches. Immunohistochemistry9 (IHC), introduced in the early 1940s, enhances image information content through antibody-based contrast agents for diagnostically valuable targets, such as cell surface markers and structural proteins. This approach can yield information on the spatial distribution of these diagnostic targets and has proven very useful in some cases,10 but also has important limitations.11–13 Other efforts have focused on quantifying image information and presenting it to physicians in the form of computer aided diagnostic (CAD) tools. A requirement for widespread use of CAD methods is that clinical tissue images be digitized. Although histopathology analysis is generally performed directly through the microscope, with no digital image ever acquired, digital recording of whole slide images (WSI) is becoming increasingly common, particularly for training purposes.14
Several label-free contrast modalities are being explored with regards to their utility in histopathology. Among these, vibrational spectroscopies such as Raman scattering and infrared absorption (IR) offer significant potential to increase the information content of histology images. Like IHC, these modalities provide chemically specific and diagnostically relevant information, but without the need of labeling. IR spectroscopy was applied as early as 1952 for qualitative characterization of normal and neoplastic tissue.15 Several IR and Raman histopathological studies in the 1990s through the 2000s utilized multivariate analysis methods to provide probabilities of disease within regions of interest.16,17 Some studies also provided relative abundances of major tissue components having diagnostic value.18 While studies such as these showed impressive sensitivity and specificity, they often did not provide image information that could be easily integrated into a diagnostic decision. IR imaging was not of sufficient spatial resolution, and Raman spectral acquisition is sufficiently slow as to preclude imaging at the desired spatial resolution. These have been important limitations for a field where image interpretation is central.
The recent advent of coherent Raman microscopies19 have facilitated rapid label-free imaging, and bright infrared (IR) sources now available allow for IR imaging with higher spatial resolution.20,21 When spatially resolved, vibrational spectroscopy can provide familiar and diagnostically important information such as cell arrangement, phenotype, cellular and subcellular morphologies, structural proteins, and intracellular lipids. Because they require little or no sample preparation, these modalities can provide this data in situ or with minimized latency between excision and availability of actionable information. Further, the intrinsically digital image format and potential for rich chemical contrast makes vibrational histopathology a strong candidate for strengthening CAD.
Here we review current applications of coherent Raman methods to histopathology, and consider its potential for meeting the needs of this field in the future. To better evaluate this potential, we first provide brief overviews of histopathology workflow, diagnostic criteria & uncertainty, and technology-related efforts underway to improve histopathology, including application of infrared and spontaneous Raman spectroscopy.
Pre- and post-operative histopathology studies are typically done in a batch mode, and usually take a day or two to wend their way through the process, requiring approximately 13 hours of actual processing time. In this process, excised tissue is first grossly examined (by eye) for diagnostic information, and prepared for microscopy. Tissue samples are typically fixed and embedded within paraffin wax, and some of the tissue is sliced in 10 to 50 νm thick sections. These formalin-fixed paraffin-embedded (FFPE) sections are then stained (typically with hematoxylin and eosin [H&E]) and mounted for inspection. In a large hospital, a pathologist may inspect 500 histology slides in a day. Diseased regions are frequently small, so the clinician may spend a significant amount of time combing through normal tissue. In order to search more efficiently, pathologists will often identify landmark species such as blood cells and vessels, lymphocytes and ducts, which are frequently found in vicinity of cancerous lesions. Preliminary microscopic analysis and diagnosis are often carried out by a single pathologist. In difficult cases, a panel of experts may review the slides, and additional staining may be requested to identify diagnostically important species that H&E staining does not highlight. Ultimately, analysis and diagnosis are generally reviewed and signed-out by an attending physician.
One type of study involves mandatory or voluntary review of a wide range of tissues by “peer” physicians (i.e., not necessarily subspecialty experts). In this class, Kronz et al.26 and Raab et al.27 report on inter-institution or institution-wide studies where large numbers of cases (6171 and 6162 respectively) covering many tumor types were reviewed. Kronz et al.26 found 1.5% discordant diagnosis that resulted in a major modification of therapy or prognosis was observed. The majority of these cases involved a change between benign and malignant or a major change in tumor classification, and changes involving only a modification of tumor grade or stage were not included. That study found significant variability in diagnostic discordance with type of tissue. Serosal surfaces and the female reproductive tract tissues had 9.5% and 5.1% diagnostic discordance respectively, the highest found in the study. Raab et al.27 estimated an error rate 6.7% from self-reported discrepancies upon second review at 72 institutions. In that study, only 1.1% of discordant diagnosis resulted in a major modification in therapy or prognosis.
Addressing these types of studies, Ho et al.28 argue that statistics based on second review likely reflect an underrepresentation of errors since the reviewer often has knowledge of the original diagnosis and the sign-out pathologist. They argue that such a priori knowledge has led to biased review in similar situations.29
In another class of study, previously diagnosed cases for a single tissue type are analyzed by a panel of sub-specialists. Higher discordant diagnosis rates are typically found in these types of studies. Lurkin et al.30 report on a review of all sarcoma cases (366) in the Rhone-Alpes region of France over a 12 month period. They found that 19% of cases resulted in change of type or invalidation of diagnosis, and 27% of cases resulted in change of grade or subtype of diagnosed cancer. Bruner et al.31 found similar results for 500 neuropathology cases submitted to a specialist review committee. They found that 9% of reviews resulted in immediate significance for therapy or intervention, and 19% resulted in a change in type or grade of glioma. Similarly, specialist review of 602 prostate adenocarcinoma cases32 led to a change in the Gleason score by at least 1 point in 44% of cases, and patients’ risk category was increased in 11% of cases. Likewise, of 340 patients presenting for second opinions regarding breast cancer, 80% resulted in some diagnostic change, with 8% of reviews leading to altered surgical therapy.33 In another study, 131 bladder carcinoma cases underwent secondary review, with 18% exhibiting significant discrepancies.34 In a gynecological oncology35 study of 295 referred patients, 5%, resulted in diagnostic changes that had major therapeutic or prognostic implications.
Arriving at definitive numbers for diagnostic uncertainty is outside the scope of this review. However, it does seem clear from the studies cited above that the uncertainty is significant for some cancer types. This widely recognized fact was reflected in a recent survey of pathologists,36 which showed that physicians on average expect 10% diagnostic uncertainty from histopathology for cancer, and that lack of sub-specialty expertise is seen as the most important factor contributing to misdiagnoses. Based on these facts, one may be inclined to favor the sub-specialist review literature, and the slightly higher discordant diagnosis numbers found therein. The overall picture is that difficulty in diagnosis varies significantly with tissue and tumor type, and many human factors figure in precision and accuracy of diagnostics.36,37
Type | New cases39 | Diagnostic uncertainty | Initial cost40 | Unnecessary treatment costa | Raman uncertainty |
---|---|---|---|---|---|
a These values are estimates assuming diagnostic uncertainty is evenly distributed between precision and accuracy. | |||||
Breast | 250k | 20%7 | $23k | $575 M | 5%18 |
Lung | 225k | 10%3 | $61k | $680 M | 10%41 |
Prostate | 180k | 30%5 | $20k | $540 M | 5%42 |
Colon & Rectal | 135k | 10%6 | $52k | $350 M | 5%42 |
Bladder | 77k | 40%4 | $21k | $320 M | 7%43 |
Melanoma | 76k | 25%44 | $5.5k | $52 M | 5%44 |
The magnitude and gravity of this problem is not lost on physicians and technologists, and the field of histopathology has been slowly evolving to provide better informed and precise diagnoses. Several innovations have been in process of adoption over the past decades.
Improved diagnostic precision has been achieved by multiplexing classical and IHC stains, where the various contrast agents are imaged together.50 Alternatively, cyclic IHC labeling24,51–53 of samples facilitates registered imaging of many proteins. Such approaches can lead to improved specificity for cancer characterization, but can add considerable complexity and, in the case of serial IHC, require up to a day for each labeling cycle.53
Fig. 1 shows an example of a prostate adenocarcinoma display from a pedagogical WSI application, allowing simultaneous view of an image thumbnail, and a detailed sub-image section.14 This sort of data presentation has obvious pedagogical and practical benefits, but large, high-resolution images such as this may occupy ≈1 Gb of disk space. Thus, the work of a single pathologist may occupy 0.5 Tb day−1, and storage space requirements are often cited as a potential barrier to widespread acceptance of WSI.56 Another barrier is slightly reduced image quality compared to direct microscopic observation.55 There are further concerns about variation in sample preparation and staining practices, and that image capture and coloration can vary over an important range among WSI instruments. Consideration of these issues is currently underway at the U.S. Food and Drug Administration.57
Fig. 1 WSI image of prostate adenocarcinoma tissue from set of Genitourinary Block training in the preclinical curriculum at the University of Arizona College of Medicine, in Tucson, AZ.14 |
While radiological CAD is now well-integrated into the diagnostic process, histological CAD is not used routinely. This dichotomy may be due in part to the fact that the former simply provides evidence for the presence of tumors, while the latter must address the more complex questions of tumor type and grade, and deals with images of higher complexity. Numerous algorithms have been proposed for diagnosis of specific cancers,60–62 but there has not been significant clinical adoption. Histological CAD, however, seems to be emerging from the research-only phase, as a number of companies are becoming involved in developing these tools.63
Histological CAD systems may exploit a large range of image features to derive clinically significant information,60 including object size and shape, intensity, color, texture, and inter-object organization. Approaches for extracting these inputs from images have been reviewed recently.64,65 The image processing associate with CAD will generally include some or all of the following steps: (1) preprocessing, such as color and intensity normalization and de-noising. (2) Feature identification using thresholding or segmentation approaches. (3) Quantification of relevant feature characteristics, such as shape, size, spatial arrangement, texture, or color. (4) Reduction of data dimensionality using methods such as principle component analysis. (5) Classification based on the reduced-dimension dataset. Steps 2 through 5 (and sometimes step 1) are collectively referred to as “machine learning”. We briefly discuss these steps below.
Use of blank images66 or reference materials can reduce inter-image variability, but this is often insufficient. In order to minimize these variations, preprocessing steps such as color and intensity normalization are often employed. Color convolution67 quantifies staining densities of a small number of contrast agents based on the fact that light absorptivity of individual stains is linear in log space. This approach continues to be developed, and can be used to map image color and intensities from many samples onto common range,68 assuming each image contains similar ranges of features. However, color and intensity normalization approaches can lead to significant artifacts when based on assumptions that the proportion of pixels or range of stain intensities from each image in a set should be identical.69 In cases where a given stain vector may not be appropriate for all samples, individual vectors can also be normalized to a single image.69
Another approach uses active contours, which are deformable splines that encompass image segments. They are constructed to minimize some energy function that may have contributions dictating the stiffness of the spline, the placement of the spline on the image (e.g., lowest energy at a minimum image gradient location), and a term that encapsulates a priori knowledge of image segment shape or location.65 These splines can be parameterized (SNAKES)76 or unparameterized (level set methods).77 Each approach has strengths and weaknesses, with SNAKES being more robust to noise, and level set methods having greater flexibility to accommodate variable topology. In cases where salient aspects of the image segment shape is known, Bayesian analysis of template comparison78 has been successful, and found robust to image variation. Each of these segmenting approaches may have trouble with overlapping features of interest, and active contours are prone to inclusion of unwanted background objects.64 Also, Bayesian methods are, in general, computationally expensive.
Of course, pixel-level and active contour approaches are not mutually exclusive. It is possible to leverage the strengths of each in a single algorithmic approach.79
Linear discriminant analysis (LDA) is a commonly used supervised feature selection approach that assumes independent variables are normally distributed. It is similar to PCA, except that the feature types are specified by the user. A comparison of PCA and LDA can be found in Martínez et al.,83 who prefer PCA when the training dataset is small, other factors being equal.
Similar to methods mentioned above, manifold learning (ML) approaches project an M dimensional dataset to a space with fewer (N) dimensions, where preferably, N ≪ M. Both supervised and unsupervised ML approaches exist, and these differ from those above in that there is no assumption of linearity in the relationship between the features of interest and the original data.64
Deep learning is another route to classification. These approaches are an evolution of neural networks in which the data vectors are iteratively transformed using nonlinear functions in such a way as to emphasize image aspects having predictive power. Using these approaches, segmentation, feature selection, and classification are all done at once, and all without user intervention. However, the entire process is opaque to the user in that there is no way to know what features the model is using for classification. Thus, there is thus no way to know when the model is likely to fail. In fact, only to the extent that the dataset on which the deep learning model was trained is comprehensive, can the characterization be reliable. By way of apparent counterexample, however, Cirean et al. were able to use a relatively small sampling of mitotic events to create a highly effective deep learning model for detecting mitoses in H&E images.87 Nonetheless, building a comprehensive dataset for deep-learning image recognition seems to be the intent of IBM in acquiring a company with access to 30 billion radiological and histology images.59
Even the most basic level of CAD assistance – that of providing image feature identification, such as of nuclei or regions of potential metastases86 may be quite valuable. In many cases, a pathologist may search for regions of interest (ROIs) that cover only a small fraction of the sample. Identifying such regions up front, or even identifying landmark species that typically signify such regions could provide a significant time savings.53
Feature quantification might be considered the next level of sophistication where CAD approaches could be of high value. We find evidence for this by considering the work of Fuchs and Buhmann,88 who found 10% disagreement on location and size of nuclei in a renal cell carcinoma sample between two sub-specialists, and inter- and intrareader disagreement of 42% and 21% respectively for counts of normal and atypical nuclei among five pathologists who were not sub-specialists. This level of variability is comparable to the overall diagnostic uncertainty discussed in Section 2.3, and suggests that correct identification of relevant features may play a significant role in diagnostic uncertainty for this and other cancers. Image analysis approaches discussed in the previous sections can provide quantitative characterization of key features, such as cell density61 or mitotic event counts87 and nuclear shape metrics78,84 at precision levels similar to that of sub-specialist analysis in the Fuchs and Buhmann work.88
While CAD work is largely done using H&E,87 improved approaches to image contrast would likely lead to increased reliability of CAD outputs.63 Naturally, an expanded contrast pallet serves to put image-derived metrics on a more solid footing.53 Even the simplest pixel-level thresholding methods can be quite reliable when pixel intensity values are related to specific chemical or functional information. For example, pixel-level analysis of vibrational spectra resulted in accuracies of 94%–100% for classification of ten disparate histologic classes,89 98% for positive prostate nuclei from IHC images with contrast for androgen receptor protein,90 and 95% for micro-metastasis from cytokeratin-stained lymph node sections.91
Refractive index maps and light scattering patterns from tissues both provide information useful for diagnosing aspects of some cancers. In light scattering experiments, light is refracted, or scattered, as passes through materials with discontinuities inrefractive index. Optical coherence tomography is now well developed as a clinical imaging tool, and can be used to determine spatial location of scattering sites.92 Additionally, polarization, wavelength, and scattering angle may be measured directly in an imaging mode to deduce relevant tissue properties. Tumor growth modes produce tissues that scatter light strongly93 and anisotropically,94 and it appears that light scattering may be sufficiently sensitive for very early tumor detection of some epithelial cancers.94 Elastic interactions of light and matter may also be characterized by directly mapping out the refractive index of a sample through quantitative phase imaging.95 QPI reports on refractive index variation from the nano-scale to the micro-scale, and as with qualitative phase imaging, it makes boundaries between tissue regions and cellular compartments readily visible. QPI can also discriminate some tissue and cell types,93,96 locate micro-calcifications,93 and even indicate degree of malignant transformation97 based on values of refractive index. Light scattering pattern maps and phase imaging maps are essentially interchangeable as the former can be calculated from the latter and vice versa, provided the light scatting is spectrally resolved.98,99
Light absorption100 and intrinsic fluorescence101 from ultraviolet102 or visible and infrared light103 can be used to characterize tissues. When specific excitation and emission wavelengths are used, signals deriving primarily from species such as nicotinamide adenine dinucleotide (NAD) and flavin adenine dinucleotide (FAD),100 elastin and collagen,104 tryptophan,105 and possibly porphryns103 can be isolated. Accumulation or deficit of these species is not specific to tumors, but does seem to be a sensitive indicator for the presence of neoplastic masses. In fact, autofluorescence has been used as method to rapidly locate candidate tissue regions for a slower, but more specific mode of characterization, vibrational spectroscopy.106
It is common to describe vibrations of molecules with N constituent atoms in terms of oscillatory motions along vectors with up to 3N-6 dimensions called “normal modes” (Q). These modes constitute collective intramolecular motions that preserve the overall symmetry of the molecule. The modes have discrete allowed states with energies approximately equal to (ν + 1/2)ħωm, where ν are integer values, and ωm is the ground vibration frequency. The ground state vibrational frequencies and energies vary depending on the strengths of bonds and masses of atoms involved in the normal mode motion. Transitions with Δν = ±1 are most often measured, and those with energies in the range (500 to 3100) cm−1 are typically of interest in biological systems. Signatures in the higher energy range (2800 to 3100) cm−1 arise from transitions between states of modes involving symmetric or asymmetric stretching of C–H bonds. The range (1800 to 2800) cm−1 is referred to as the quiescent region because transitions of natural biological molecules typically do not appear in this range. The greatest variety of vibrational transitions in biological molecules occur in the fingerprint range (500 to 1800) cm−1.
Transitions between these states are usually detected through absorption, emission, or inelastic scattering of light. The probability for photon absorption or emission is significant when ∂μ/∂Q ≠ 0 at Q = 0, where μ is a mode's permanent dipole moment. By contrast, inelastic scattering is important when ∂α/∂Q ≠ 0 at Q = 0, where α is the molecular polarizability. Infrared and Raman spectroscopies are respectively based on absorption and inelastic scattering.
Fig. 2, panels a and b show IR and Raman spectra for major classes of tissue constituents. Since these have distinct signatures in both spectroscopies, they can be identified by either method. On the other hand, IR absorption linesare slightly broader. This is of consequence in the fingerprint region, where spectral lines are quite congested. Panels c and d of Fig. 2 show fingerprint regions of IR and Raman spectra from cells, illustrating that the width of the individual IR peaks makes it difficult to separate them into more than just a few distinct contributions without peak fitting. On the other hand, Raman peaks are more narrow, so that spectra from similar systems contain a larger number of easily resolvable peaks, providing a much higher level of chemical specificity. The differences can be quantified in terms of information content, or entropy, Si = ln(Ωi), where Ωi is the maximum number of distinct spectral states one can discriminate. Here, Ωi = SNRM where M is the number of spectral peaks and SNR is the signal to noise ratio in each peak. We can estimate this value by noting approximately 15 peaks in the IR spectrum, 5 peaks in the Raman CH region, and 45 peaks in the Raman fingerprint region. Assuming SNR = 15 for IR peaks and for CH stretch Raman peaks, and SNR = 3 for Raman fingerprint, we obtain Ωi = 1017, 106, and 1021 for IR, Raman CH, and Raman fingerprint spectra respectively (1027 for the entire Raman spectrum). Partly for this reason, Raman is found to have better specificity for classifying cancer.110
Fig. 2 a & b: Respectively, IR spectra and Raman spectra of major cell components.107 From top to bottom on both panels, spectra are from albumin, globulins, collagen, glycogen, DNA, RNA, and lipid. c: IR fingerprint spectrum of whole cells,108 with ≈12 resolvable peaks. d: Raman fingerprint spectrum of whole cells,109 with ≈45 resolvable peaks. |
Given their capacity for encoding chemical information, it is not surprising that these spectroscopies have been applied to many general aspects of disease diagnoses.111,112 They have also been used for guidance in tissue sampling,113 augmented bases for stratified diagnosis,114 and intraoperative guidance during tissue-conserving resection.115,116 Of particular present interest is the considerable body of literature showing that vibrational spectroscopy provides diagnostically significant information for cancer diagnosis in conjunction with histopathology. There have been many excellent reviews on this topic in recent years,107,111,117–121 so here we will only summarize relevant aspects.
Histology-related studies using vibrational spectroscopy began to appear in the 1960s122 for IR, and in the 1980s for Raman,123 and to date, there have been thousands of publications. However, it appears that only in the past decade or so that IR and Raman technology have been sufficiently developed to produce meaningful studies.107 Recent work provides rather convincing evidence that vibrational spectroscopy can detect diagnostically important changes in tissues, routinely yielding objective sensitivity and specificity figures generally in the 85% to 95% range for detection and classification of cancers, even when spectra on which discrimination is based are usually taken at just a few spots from normal and diseased portions of tissue.
In one example, Pence et al.120 discuss 26 large studies of epithelial cancers (>50 patients), among which, the average sensitivity and specificity obtained using Raman spectroscopy were 89 ± 7% and 88 ± 8% respectively. Such precision generally exceeds that possible by subjective morphological analysis. Similar examples are listed in the final column of Table 1, indicating a reduction in diagnostic uncertainty by more than a factor of two in most cases. Table 1 also provides prevalence, histology-based diagnostic uncertainty and initial treatment costs for the six most common cancers in the US. Assuming naively that diagnostic uncertainty is evenly distributed between precision and accuracy, we estimate that using Raman spectroscopy to supplement histology could save >$2B per year in unneeded treatments, and reduce the burden on patients and families associated with missed opportunities for earlier treatment.
Table 2 gives a partial list of cancer biomarkers that have been used in Raman studies to classify neoplastic and cancerous tissues, and lists spectral peaks used to identify those markers. Most markers have several peaks, facilitating increased robustness in detection, to the extent that noise in separate peaks is uncorrelated, but even correlated noise can be suppressed if peak ratios can be used. The fact that most of the peaks used in these Raman studies are found in the fingerprint region is due to the high spectral density and low peak widths found there, and underscores the importance of this spectral range for chemical specificity.
Marker | Relevant cancers | Spectral lines [cm−1] |
---|---|---|
ν-stretch, δ-scissoring, ρ-rocking, ω-wag, τ-twist. | ||
Metabolic markers | ||
DNA (relative abundance) | Breast18 larynx124 | 668, 678, 728, 750, 785νO–P–O 825νO–P–O, 1336, 1488, 1580 |
Lipid (structure and relative abundance) | Breast18 colon42,125 bladder43 skin44,126 | 1309τCH2 1445δCH2 1654νC–C 2850νCH2,sym 2880νCH2,asym 2920νCH3,sym 2960νCH3,asym |
Cholesterol ester | Breast18 brain127 | 430, 702, 1302, 1442, 1740νCO |
Glycogen | Bladder43 prostate42 brain127 | 472, 481, 846, 932 |
Creatinine | Breast128 | 600νN–CH3 678νCO 685τC–S 692δCO2 840δN–CH2 903νC–C–N |
Structural molecules, etc. | ||
Protein structure | Skin44 melanoma44 | 1660νC–O (amide I) 2850νCH2,sym 2880νCH2,asym 2920νCH3,sym 2960νCH3,asym |
Keratin | Epithelial cancers129 | 623νCS, 643νCS, 850δCCHaromatic, 885ρCH2 933, 1002νC–C (ring breathing), 1031 (phenylalanine), 1200 1350 (amide III) 1450τCH 1650νCO (amide I), 2940νCH3 |
Collagen | Breast18 prostate42 colon42 lung41,130 skin131 | 855, 938, 1004νC–C1260νCN, δNH1314ωCH3CH21445δCH21660νC–O 2940 |
Elastin | Lung41 | 1442δCH2 1660νC–O |
Carotenoids | Bladder43 lung41 brain127 | 1159, 1523 |
Polysaccharides | Brain132 | 856 |
Others | ||
Phospholipids | Lung41 brain127 | 719 1442δCH2 |
CaC2O4, Ca5(PO4)3 | Breast18 brain132 | 912νC–C 960νPO 1477νC–O |
Phosphatidylcholine | Adenocarcenoma107 | 719, 1666νC–O |
Trypotphan | Lung41 prostate42 | 650τC–C 1260νCN, δNH |
Proline | Prostate42 | 939νC–C |
Tyrosine | Prostate42 | 1176νC–H 1217νC–C6H5 |
Many of the markers listed in Table 2 are commonly visualized in pathology through IHC or classical stains, but, of course those methods require labeling. Not only is vibrational contrast label-free, it can also be semi-quantitative when peak amplitude ratios are used. Accordingly, peak pairs that provide information on protein-to-DNA or protein-to-lipid ratios, or on degree of unsaturation in fatty acids frequently yield highly reliable metrics of disease type and grade.41,124,132,134–137
Macromolecular structure and material phase information are also available from vibrational spectroscopy, and can be useful in cancer detection and diagnosis. For example, degree of calcification is a known marker for many cancers.8 Using classical stains, it is difficult to distinguish between types of calcification, but such discrimination turns out to be important. Haka et al.18 showed that Raman spectroscopy can be used to easily distinguish CaC2O4 from Ca5(PO4)3, and that their relative abundances correlate with malignancies in breast cancer. Raman spectroscopy also contains structural and hydrogen bonding information for lipid, protein, and nucleic acids, which has diagnostic value for many cancers.18,44,126,128
Spectral changes between healthy and diseased tissue can be striking,41,138 or subtle,107 but generally appear in the context of highly complex spectra from the tissue. Thus, while it is sometimes possible to determine identity of diagnostically relevant species though differences between spectra from normal and diseased tissue,127 this is sometimes not possible even when the primary marker species is known.129 For this reason researchers often derive diagnostic information from tissue spectra using spectral pattern recognition approaches.
These unsupervised feature extraction methods frequently serve as the input to unsupervised and supervised classification methods. Unsupervised methods, such as K-means clustering139,146 (KMC), divisive correlation cluster analysis (DCCA), and agglomerative hierarchical cluster analysis (AHCA), link together related spectra based on spectral proximity metrics.139 Supervised methods, such as linear discriminant analysis111 (LDA), partial least squares147,148 (PLS), support vector machines149,150 (SVM), and random forest classifiers,146,151,152 rely on training data to generate a classification model. These models may be conservative or relaxed according to clinical need. For applications such as tissue-conserving resection,115 one may choose a conservative model to minimize false positives. Alternatively, if the aim is to reduce the burden on pathologists, one could design a model that may include false positives, but automatically classifies the most clear-cut cases, leaving the rest for inspection.153 Training these models may be arduous, requiring many datasets and extensive validation; but post-training classification may be rapid.146,148 There are, additionally, ‘semi-supervised’, methods that aim to incorporate known and unknown information into classification problems, such as hybrid linear analysis154 (HLA) semi-supervised PCA-linear discriminant analysis155 (PCA-LDA).
Deep learning156 (DL) and artificial neural networks157,158 (ANN) constitute an important family of methods that have received significant attention recently. These methods rely on layers of interconnected decision units or ‘neurons’; typically 1–2 layers for ANN approaches, and potentially many layers for DL approaches. Whether employed as supervised or unsupervised learning, the weights and the interconnection strengths of the neurons are iteratively modified to optimize a merit function. One of the primary strengths of ANNs and DL is that nonlinear and extremely complex interconnections can be created within the network that may not be possible with traditional linear or nonlinear methods such as those mentioned above. A significant challenge to ANNs and DL, as with all supervised techniques, is the requirement that the training set be comprehensive in that it is representative of all future data. Another challenge is that these approaches are ‘black box’ in that it is not currently clear how to understand what data features are found to be important, and how they are used in the evolved ANN/DL neural connection architecture. Interpretation of ANNs and DL is an active area of research.159
Thanks to recent commercial availability of quantum cascade lasers, approaches20 for IR imaging at spatial resolution as high as 1 μm are now feasible, making it possible to generate gross tissue maps based on IR absorption. Since the chemical specificity of IR spectroscopy is sufficient to easily discriminate among tissue regions such as epithelium and stroma,160 and between benign and malignant tissue,89 IR maps can be merged with H&E images to indicate tissue regions of interest,133 as shown schematically in Fig. 3a. Morphological features extracted directly from such maps have been used to classify tissue samples as cancer or non-cancer, with high accuracy.161 This automated ability could provide considerable value, as mentioned in Section 2.7.6.
Fig. 3 IR and Raman histopathology: (a) Infrared absorption spectra contain sufficient information for classification of tissue, and provide value through indicating regions of interest.161 (b) A: H&E stained section of lymph node germinal and mantle zone, and B: a sub-image with contrast from Raman spectroscopy showing activated lymphocytes in yellow.107 (c) Spatially multiplexed Raman spectroscopy with 30 ms effective spectral acquisition time using 500 mW of 532 nm light. The cytochrome-c distribution is indicated in yellow.164 |
From a spatial resolution perspective, Raman imaging has an important advantage over IR in that it uses visible or near infrared wavelengths, so can reach spatial resolution 2× to 3× higher than with IR, making cellular and sub-cellular imaging possible. The combination of high resolution and rich fingerprint spectra allows for significant information in addition to providing classification of tissue regions. For example, Raman imaging can provide label-free identification of cell type,162 distribution139,163 and content of organelles.164 Important functional information is also available, such as DNA to protein ratio of nuclei,165 the cell cycle state,163,166,167 and whether immune cells such as leukocytes are activated.107,168 Examples of cellular function and organelle distribution maps generated with Raman spectroscopy are shown in Fig. 3b and c.
Although IR and spontaneous Raman imaging can provide useful diagnostic information, both have practical drawbacks. For example, conventional glass slides can be problematic for both methods as they emit fluorescent light when excited with visible and near IR light, and absorb IR light at wavelengths of 3.5 μm and longer, which constitute the fingerprint region for IR. Consequently, both approaches often require special substrates – quartz or CaF substrates for Raman studies, and CaF or BaF2 substrates are used in IR. On the other hand, glass slides are transmissive to IR wavelengths in the range λ = (2.8 to 3.5) μm, allowing only the CH stretch spectral region to be used.160 Another consideration is that the intrinsically low Raman scattering levels can be easily masked by fluorescence; thus, in addition to use of specialty substrates, it is often necessary to photobleach samples before acquiring Raman spectra.
The benefits of spectral image acquisition should be considered in context of required imaging time. These considerations differ significantly for IR and Raman spectral imaging. IR absorption cross sections for fingerprint and CH stretch modes are typically on the order of 10−18 cm2, so most of the light will be absorbed for a high density of absorber with (5 to 10) μm path length. Having such a large effect on transmitted light, signal can be acquired very quickly. Yeh et al.21 describe work wherein 128 × 128 pixel wide field IR absorption images were acquired serially on a fixed region of the tissue while the laser wavelength was scanned. In total, 282 frames were acquired, yielding IR absorption images over a 1128 cm−1 range in the fingerprint region, with an effective pixel rate of 1.6 MHz, and spectral acquisition rate of 3 kHz. In cases where fewer spectral points are required, some imaging speed increase can be obtained by using only several discrete frequencies. For example, Tiwari et al.169 found that they could reliably identify diseased cardiovascular tissue regions with on-the-order of 10 discrete spectral points. Allowing for imaging of ∼10 discrete contrast frequencies and a 5 μm pixel size, tissue could be mapped at 0.5 cm2 min−1, similar to WSI instruments currently in use. Further, if the discrete frequencies are exclusively in the CH stretch region, glass slides could be used, and there would be minimal impact on the histology work flow, as IR imaging can be performed on H&E stained slides.160
In contrast to IR absorption, the Raman signal is quite small, and imaging with this contrast mechanism is proportionally slow. The differential scattering cross-section of Raman-scattered light collected of over solid angle (Ω) is given by dσ/dΩ ∝ |∂α/∂Q|2, and is on the order of 10−30 cm2 sr−1 for most modes. Accordingly, spectral acquisition rates typically range from 5 to 0.01 Hz.129,170 Using spontaneous Raman scattering, one cannot excite only selected vibrational frequencies, so there is no benefit to discrete spectral imaging as in IR. On the other hand, some increase in acquisition speed can be achieved through spatial multiplexing. Fig. 3c displays a spontaneous Raman image acquired at a 90 Hz effective spectral acquisition rate164 through line-focused (rather than spot-focused) excitation light. Because practical detector arrays are presently limited to 2 dimensions, and one of those is devoted to spectral variations, spontaneous Raman signal is limited to 1 dimensional multiplexing such as line excitation. Of course, spatially multiplexed signal generation requires light sources with proportionally scaled power.
Given the much slower acquisition speeds for spontaneous Raman scattering, it is difficult to find an obvious application in the histopathology work flow. For example, pixel sizes of 27 μm and a 90 Hz spectral acquisition rate achieved through spatial multiplexing would provide appropriate sample throughput, but with such course resolution, one would give up significant benefit of sub-cellular chemical detail otherwise available from Raman scattering. Spontaneous Raman imaging could be useful in a guided subsampling mode, where only regions of special interest were imaged. On the other hand, even with spatial multiplexing and 90 Hz spectral acquisition, only 0.003% of the total tissue area could be imaged at high resolution in the 2 minutes time frame required to obtain a 1 cm2 whole slide brightfield image.
At this point it is appropriate to consider that the 2 minutes per slide imaging time is the standard for single-contrast image formation, whereas Raman spectral images could contain most or all of the potentially desired contrasts in a single, label-free image. Nonetheless, it seems clear that some trade-off between coverage and resolution would be necessary if spontaneous Raman scattering is used. The severity of such a trade-off could be reduced by use of coherent Raman techniques, which provide equivalent spectroscopic information with imaging throughput that is closer to that of WSI.
We can gain insight into coherent Raman scattering by considering that an oscillating electric field of light induces a polarization in a substance that can be described as:
P(t) = Nα(t)Esin(ωt) | (1) |
(2) |
In coherent scattering, vibrational modes are driven by external light fields. When the difference in energy between a pump (p) and Stokes (S) field is equal to the energy difference (Ωv) between two adjacent quantum levels of the vibrational mode, i.e., when Ωm = Δω = ωp − ωS, energy can be transferred to that vibrational mode. While the primary fields, E(t) = Eje−iωjt, (j = p,S), ωp > ωS ≫ Ωm, are at much too high a frequency to directly induce significant molecular response, electrons are able to follow the fields adiabatically. Through its nonlinearity, the electron response will contain some amplitude at the difference frequency, Δω. This component of the response will in turn exert an oscillatory force F(t) on the molecule:179
(3) |
(4) |
(5) |
Substituting eqn (5) and a first-order truncation of eqn (2) into eqn (1), we obtain an expression for the polarizability of a given mode in presence of a pump and Stokes field:
(6) |
(7) |
χ(3)(Δω) = χ(3)NR(Δω) + χ(3)R(Δω) | (8) |
In third order coherent Raman processes, a pump and Stokes field interact with a medium to create a vibrational coherence, given as:
C(Δω) = χ(3)(Δω)[ES ★ Ep](Δω) | (9) |
(10) |
Another important difference is that CARS signal contains a nonresonant background (NRB), arising from χ(3)NR. It is termed “nonresonant” because it arises from the electronic response, and is not sensitive to vibrational resonances. Since the electrons adiabatically follow the driving field of the laser, the NRB is always in phase with the driving field, and retains a fixed phase relationship with the resonant response. The overall CARS signal has a component that is a coherent mixing between these two terms:
(11) |
Owing to the three terms in eqn (11), the analyte concentration dependence of the raw CARS signal depends on the ratio of resonant to nonresonant response. χ(3)NR is of similar magnitude for most biological materials but χ(3)R may vary widely owing to the density of molecular oscillators, Ñ (fraction of N0 within the focal volume) and the Raman scattering cross-section of individual vibrational modes. Typically, χ(3)NR ≤ 0.1Ñχ(3)R for CH stretch resonances and ICARS ∝ Ñ2. However, for many fingerprint peaks, χ(3)NR ≪ Ñχ(3)R and the resonant contribution to the ICARS is linear in Ñ, but NRB dominates the signal. This is why narrowband CARS is performed almost exclusively with CH stretch resonances as contrast. In fact, considering only the term |Ñχ(3)R|2, Cui et al.181 demonstrated that the resonant CARS signal amplitude is greater than spontaneous Raman only when oscillators are very concentrated, such as in bulk phase, and samples rich in lipid or structural protein.
Another important effect of the coherent interaction between resonant and non-resonant signal components arises from the Gouy phase shift, wherein a phase shift accrues with field propagation beginning at the focus of a Gaussian beam compared to propagation of a plane wave. This becomes a problem at sharp interfaces of features smaller than the Rayleigh range when resonant signal is generated only by light on one side of the focus while the non-resonant signal is generated at positions on both sides of the focus. When these mix, there is a spatial position-dependent spectral phase error182 causing spectral distortion. However, the Gouy phase is not a problem when χ(3)NR ≪ Ñχ(3)R, or when there are no sharp spatial discontinuities in resonant signal contribution.183
The CH stretch vibrations are sensitive to their environment by direct coupling and through Fermi resonances with C–H bending modes near 1500 cm−1. Together, these influences can introduce significant shifts and broadening of the CH-stretch peaks. Spectroscopic184–187 and spectral focusing188 CARS approaches are best positioned to leverage the information contained in peak shape and position changes. Narrowband approaches19 return vibrational contrast over a narrow spectral band. In the CH stretch region, excitation is tuned to report primarily on a single resonance, but the signal generally includes lesser contributions from multiple resonances due to spectral overlap. Signal from CH2 and CH3 are frequently used to determine relative abundance of protein and lipid because the ratio of CH2 and CH3 moieties differs significantly in these chemical families. The information content in (symmetric, asymmetric) peak pairs is largely redundant, so only one peak from each pair is typically used for this purpose.
In narrowband CARS, a pair of picosecond pulses is used to excite coherence of a single selected vibrational band, and the higher frequency field is used to probe the coherence. Meyer et al.189 found that, for strong resonant signals (χ(3)R ≥ 10Nχ(3)NR), simple subtraction of NRB is sufficient to render a quantitative signal with no significant distortions to the CARS peak positions and amplitudes. Wu et al.190 have shown quantitative correspondence between νCH2,sym signals and lipid content in liver tissue. Further, large resonant signals in the CH-stretch enables rapid image acquisition, with pixel rates as high as 6 MHz,191 but more typically range from 200 kHz (ref. 192) to 1 MHz (ref. 189) for tissue samples. Repeated images can be acquired at several discrete Raman shifts to map scenes with some chemical complexity. For discriminating species that are spectrally distinct, like water and protein or lipid, a single additional scan is generally sufficient for each component.193 A larger number of spectral contrast points may be necessary to discriminate species with only subtle differences.189 Given the high pixel acquisition rates, the necessity of acquiring several scans may not itself be onerous, but latency between scans for wavelength tuning can add significant time to image acquisition for most CARS systems. Rapid spectral scan approaches have been demonstrated for narrowband CARS with only 100 μs required to scan from one frequency to another in the CH stretch region, making inter-scan latency a non-issue for such systems.194
Spectral focusing is another approach that facilitates spectroscopic scanning with narrowband CARS. In spectral focusing CARS,188 a pair of pulses that are spectrally broad and centered at distinct wavelengths are temporally broadened with nominally identical chirp. Although the absolute frequencies of the two light fields impinging on the sample changes significantly with time, variation in the difference of frequencies is minimized so a single vibrational state can be excited. Using this approach, it is feasible to collect single-frequency image pixels at 250 kHz.195 This approach to CARS can be performed with sub-picosecond pulses and spectral tuning over a limited range (typically 500 cm−1) can be accomplished by simply changing the delay of one pulse relative to the other. When spectral focusing or another CARS approach is performed at a series of Raman shifts, the effects of the NRB (distortion and nonlinear concentration response) can be separated through spectral phase retrieval approaches.196–198
Spectroscopic CARS differs from narrowband or spectral focusing CARS in that a broad swath of spectrum is obtained in each laser shot. Such an approach in the CH-stretch region has been demonstrated with picosecond/femtosecond CARS184,186,187,199 and with nonlinear interferometric vibrational imaging (NIVI).200 Histological studies focused on CH-stretch using spectroscopic CARS have been performed primarily with NIVI. From tissues, NIVI typically yields CH-stretch spectra with a bandwidth of ≈250 cm−1 at a 20 Hz spectral acquisition rate (1 kHz effective rate for a single frequency assuming 5 cm−1 resolution). These are undistorted Raman spectra, linear in N.200
Several studies have evaluated the utility of CARS in the CH stretch region for generating contrast equivalent to H&E stain. Evans et al.201 compared CARS images to H&E stained images, finding that CARS could provide similar image information when νCH2,sym and νCH3,sym or νCH3,asym were used. Similar “pseudo H&E” obtained through intravital CARS imaging202 enabled inspection of tumor environments and margins without staining, and no obvious evidence of photodamage even after 300 frames.
Quantitative tissue morphology metrics can be obtained from CARS contrast, similar to what is done with H&E-contrast images. Meyer et al.203 compare CARS with spontaneous Raman in brain, demonstrating ability to extract nucleus-to-cytoplasm ratio, cell density, nucleus size and shape. They found, however, that since nuclei lack νCH2,sym contrast, nuclear density and shape metrics may not be as reliable as H&E. On the other hand, the Wong group have used primarily νCH2,sym CARS contrast and image segmentation204–206 to determine nuclear size, cell volume, and cell–cell distance and other metrics to classify cancer subtypes in lung and breast tissue, producing encouraging results for differential diagnosis of cancer in these tissues. Uckermann et al.207 also compared metrics from CARS and H&E, finding that CARS provided additional relevant information via the signal amplitude.
In addition to conveying morphological information through imitating H&E stain, CARS νCH2 and νCH3 contrast has been used to study lipid metabolism in cancer,208 and to discriminate cell type based on lipid content. Excess lipids were found as a potential marker for circulating tumor cells,209 and Evans et al.201 found contrast in νCH2,sym to evince replacement of normal white matter with lipid deficient astrocytic glioma tissue. Chowdary et al.210 found spectral differences arising from variation in lipid to protein ratio in a rat breast cancer model, and were able to use SVD to automatically segment normal and diseased tissue regions at a 99% confidence level. Similarly, Uckermann et al.207 also found a νCH2,sym contrast gradient in the infiltrative zone around glioblastoma, with rather distinct contrast levels in the normal region. They also found that levels of νCH2,sym contrast could be used to distinguish between glioblastoma, and metasticized breast tumor in the brain. In a similar finding, Meyer et al.119 showed that CARS νCH2,sym contrast in the brain could highlight metastatic tumor regions of lung origin as the metastases retain a chemical profile similar to the originating tissue. Their finding was supported by Raman and IR imaging.
While narrowband CARS contrast from νCH2 and νCH3 alone is useful in many circumstances, it can also be combined synergistically with contrast from other methods, as exemplified in Fig. 5a and b. In other examples, Yue et al. used CARS signal from νCH3,sym to visualize mammary acini, providing guidance for acquisition of spontaneous Raman spectra. From subsequent spectral analysis, they observe changes in spatial distribution of lipids concomitant with a loss of basoapical polarity in mammary acuinus, a transformation linked to early stages of carcinogens.177 The Popp group have taken similar approaches.119,170,189,211 In comparing CARS to spontaneous Raman imaging of colon sections, Krafft et al.212 found that CARS images were consistent with expectations from spontaneous Raman scattering in the same spatial sample regions, and that with just a few CARS scans in the range 3000 to 1000 cm−1, there was sufficient information to discriminate a couple of cell types and identify regions of normal and diseased tissue. The CARS images were acquired 106 times faster than spontaneous Raman (10 μs vs. 30 s per pixel). Subsequently, they have used additional CARS peaks, in conjunction with other contrast mechanisms to characterize tissues. The same pulses that generate the CARS signal can also give rise to second harmonic generation (SHG) in collagen fibrils, and to two-photon excited fluorescence (TPEF) from intrinsic fluorophores such as NADPH, flavins, collagen, and elastin. Meyer et al.119,189 used CARS contrast at multiple Raman shifts along with intrinsic contrast from TPEF and SHG to detect white matter, the granule layer, elastin, ordered collagen, and NAD(P)H in brain, and elastic fibers, triglycerides, collagen, myelin, cellular cytoplasm, and lipid droplets in perivascular tissue.
Fig. 5 a & b:221 (a) Immunohistochemical stain for Sox-10 in mouse ear with haematoxylin counterstain, revealing melanocytes in red. (b) CARS image of adjacent slice to (a), showing pigmented regions, consistent with positive staining in (a). c & d:218 (c) Pseudocolour BCARS image of tumor and normal brain tissue, with nuclei highlighted in blue, lipid content in red and red blood cells in green. (d) Raman spectra retrieved from (c), with each spectrum taken from one pixel, and acquired in 3.5 ms. e & f:222 SRH image (e) is compared to a similar section of tumor imaged after formalin-fixation, paraffin-embedding and H&E staining (f). Comparison was made using a lookup table to match SRH “staining” with H&E. g & h:223 (g) Spectroscopic SRS images of pancreas and liver (left and right respectively). (h) Independent component spectra retrieved from (g) with 99% data reduction. |
Because the fingerprint peaks are typically much weaker than the nonresonant signal, it is not possible to obtain quantitative spectral information by simply subtracting the NRB. Because typically χ(3)R ≪ χ(3)NR for fingerprint resonances, the third (cross) term in eqn (11) dominates the resonant contribution to the CARS signal, and the overall spectral phase must be retrieved in order to extract a Raman spectrum. CARS phase retrieval is possible with an approach based on maximum entropy methods196 or one based on time-domain Kramers–Kronig.197 Both provide functionally equivalent results, but the computations for the Kramers–Kronig approach require considerably less time.213 It has recently been appreciated that the NRB shape estimate required by both of these methods can have a profound impact on extracted Raman peak amplitudes. Camp et al.198 showed that symmetry relations between the NRB and resonant signal can be used to unambiguously determine the NRB shape, and thus the relative peak amplitudes of the retrieved Raman spectrum. Additionally, the retrieved signal is linear with analyte concentration. In this way, the NRB acts as an internal calibration for spectroscopic CARS, and using this internal calibration, retrieved peak ratios are absolute.
Spectral acquisition of CARS signal is required for recovery of Raman fingerprint signals through phase retrieval, and also facilitates system noise removal via singular value decomposition.214 Spectrally focused CARS195 can be delay-scanned used to generate spectroscopic images that are amenable to phase retrieval215 Additionally, several inherently spectroscopic CARS microscopy approaches have been developed, but most are too inefficient for biological imaging. Here we will exclusively consider approaches that have been used to generate images from cells and tissues. The first of these was based on a combination of spectrally broad but uncompressed Stokes pulses, and spectrally narrow probe pulses.186,187 This approach generates broadband CARS spectra with up to 3000 cm−1 bandwidth at ≈20 Hz spectral acquisition rate (12 kHz single spectral element, assuming 5 cm−1 resolution), with strong CH stretch but quite weak fingerprint spectra.216 The quality of fingerprint spectra was improved significantly in one case by implementation of a narrow-bandwidth sub-nanosecond probe,217 and in another by using a temporally compressed Stokes source.218 In the latter work, a nearly transform-limited broadband pulse acts as both pump and Stokes, exciting vibrational coherence much more efficiently than separate pulses. This approach yielded CARS spectra in both CH and fingerprint regions with uniformly high SNR across the spectrum, and 280 Hz spectral acquisition rate (corresponding to 170 kHz single spectral element acquisition, assuming 5 cm−1 resolution).218
Perhaps because it has only recently become possible to acquire high quality fingerprint spectra in an imaging modality, there is very little work on histological samples using fingerprint CARS. With relatively weak fingerprint signal, Pohling219 was able to use principal component analysis to discriminate between gray and white matter, and identify layers of granule and Purkinje cells in mouse brain tissue. Their structural assignments agreed with those obtained from H&E staining. Also in murine brain tissue, but with significantly better SNR in the fingerprint due to impulsive coherence generation, Camp et al.198,218 were able to identify many diagnostically important species and features directly from spectral peaks, without multivariate analysis techniques (Fig. 5c and d). These included red blood cells through the 1548 cm−1 and 1565 cm−1 hemoglobin peaks (C–C stretch), as well as collagen, elastin, and nucleotides through peaks listed in Table 2 for each of these species. They were able to identify tumor margins, as there were spectral differences between normal and invasive glioblastoma image regions, such as at 785 cm−1 (nucleotide), 1004 cm−1 (phenylalanine ring breathing), and 2956 cm−1 (CH3 stretch). In both the Pohling and Camp work, a tumor-associated reduction in νCH2 was observed, as in CARS studies that focused on CH-stretch.
In the mouse glioblastoma work, Camp et al.218 obtained SHG and TPEF signals simultaneously with fingerprint CARS spectra. Although SHG is typically used to identify collagen,119,220 the multi-peak spectral identification of this protein did not demonstrate identical results. They noted that collagen generates an SHG response only when the triple-helical structure is in-tact and only when the optical polarization and fibril axes are aligned.
(12) |
This expression differs from eqn (10) by addition of a pump field term. The pump field interacts coherently with the nonlinear polarization in the sample, generating a heterodyne term in the overall signal. Due to the relative phases of Ep and the field arising from the induced coherence, the heterodyne signal amplitude is proportional to only the imaginary part of χ(3). Furthermore, only this heterodyne component is retrieved if the SRL is detected through modulation techniques, and one directly obtains a signal proportional to the spontaneous Raman scattering cross section (see eqn (6)). Because χ(3)NR is real, the nonresonant component that is problematic for narrowband CARS is not detected in SRS. In spite of this, the theoretically achievable signal-to-noise ratio for SRS is the same as that of CARS,224 and a related value, the sensitivity limit, seems to be similar for both approaches.218 The heterodyne component of the SRL signal can be expressed as:
(13) |
For field strengths permissible in tissue microscopy, the relative change in pump or Stokes intensity is on the order of 10−6 to 10−8, even for the strong peaks in the CH stretch region. These small signals cannot be spectrally separated from the intense excitation light as in CARS, so must be discriminated by other means, such as lock-in or balanced detection methods. Under favorable conditions, pixel acquisition rates up to 7 MHz have been demonstrated with contrast from Raman peaks in the CH stretch region.225 However, 30–40 kHz pixel rates are more common for single spectral bands in tissue samples.222,226 Furthermore, even the strongest fingerprint peaks, such as found at 1440 & 1660 cm−1 (CH2 deformation & CC stretch) are challenging to use for image contrast with SRS. Ozeki et al.227 were unable to directly use the 1660 cm−1 peak for contrast, even when pixel dwell times were 10× longer than necessary for good SNR in the CH-stretch region.
While the NRB does not interfere with the SRS signal, other background signals can. These include two photon absorption (TPA), cross phase modulation and thermal lensing. Signals from these sources can have amplitude comparable to or greater than even the largest SRS signals. For example, in skin, the TPA signal exceeds that of CH stretch SRS by almost 100-fold.221 Not surprisingly, these background signals can easily mask weak contrast from the fingerprint region, and careful measures may be taken to suppress their effects and recover some of the stronger fingerprint signals.228,229
Because the SRS signal must be extracted directly from the laser field, lasers with very low residual intensity noise (RIN) are preferred. These are typically solid state lasers. However, fiber lasers have a number of qualities that make them desirable in a clinical setting, including being robust to physical movement, vibration, and temperature shifts, and requiring very little maintenance. A low-noise fiber source has been developed specifically for SRS230 although even for this laser, it is necessary to use both balanced detection and lock-in approaches to recover the SRS signal.
Using only contrast from the νCH3,sym band and morphological clues, Mittal et al.236 were able to identify cell membranes, nuclei, collagen and keratin in tissues containing squamous cell carcinoma. In that study, the authors found that SRS images from this single band provided similar information to that found in H&E images.
Lu et al.237 compared images from H&E staining to those generated with SRS contrast from νCH2,sym and νCH3,sym. They found that cell density counts from SRS and H&E images were indistinguishable. However, using the lipid/protein ratio from the SRS signal, and confirming this with luxol fast blue staining for the myelin sheath, they found that the two SRS bands provided a better distinction between white and gray matter than could be achieved with H&E staining. They were also able to identify red blood cells and vessels through morphological features of images with contrast at 2800 cm−1, in the nominally quiescent region. The contrast there may have arisen from TPA of heme. They also found that necrotic regions were characterized by lack of nuclei and lower protein signal, and banded regions of high protein signal appeared to be collagen.
Ji et al.238 used SRS contrast for νCH2,symm and νCH3,symm in fresh surgical brain tissues from 22 patients to create a classifier based on cellularity, axnoal density, and protein to lipid ratios, which provided tumor infiltration detection with 97.5% sensitivity and 98.5% specificity. They found that SRS images were superior to H&E images for detecting infiltrating glioma, and that SRS microscopy could detect invasive tumor cells in peri-tumor brain tissue that appears grossly normal.
Orringer et al.222 used νCH2,symm and νCH3,symm bands from SRS to recapitulate image contrast from H&E, but on unprocessed tissue, with an eye towards improving intrasurgical tissue sampling. Example images from that study are shown in Fig. 5e and f. Pathologist diagnosis based on proximally located SRS and H&E tissue samples from 30 patients was compared and found to agree 92% of the time. They also used computer algorithm-derived image attributes (based on SRS contrast) to successfully distinguish between diagnostic classes of tumors with 90% accuracy.
Cui et al.239 used SRS spectra in the region (2800 to 3000) cm−1 and multivariate curve resolution to find a correlation between cholesterol ester content in intracellular lipid droplets and Gleason scores of prostate tissue. Using acalibrated surrogate mixture for cellular lipid content, they found that the ratio between a band assigned to cholesterol at 2870 cm−1 and the νCH2,sym band was proportional to the fraction of total lipids due to cholesterol ester.
While full spectral information in the CH-stretch region is sometimes useful, a small number of peaks often provide sufficient discrimination. Otsuka et al.223 examined tissues from a tumor-graft mouse model using spectrally scanned SRS spectrally scanned over the range (2800 to 3100) cm−1. They found that through independent component analysis, they could reduce their spectral dataset by >95%, and still obtain images in which nucleus, zymogen granules, lipid droplets, and red blood cells could be visualized, and irregular glandular cell alignments detected (see Fig. 5g and h). They found that images from the reduced spectral data was qualitatively equivalent to H&E stained images. Similarly, Egawa et al.240 used SRS spectroscopy in the range (2800 to 3100) cm−1 to map spatially resolved keratinocyte differentiation, and distinguish these from Langerhans cells in the spinous layer of the epidermis. The key differences were primarily morphological, as both cell shape and nuclear prominence differ between the cell types, so the important contrast came at νCH2,symm and νCH3,symm bands.
In other studies, vibrational signal from SRS has been used in concert with other nonlinear contrast mechanisms. Freudiger et al.226 combined νCH2,sym and νCH3,sym bands with hemoglobin TPA to derive contrast of roughly equal quality to H&E-stained sections, recapitulating major histologic features in tissue from several brain regions. Galli et al.241 used νCH2 in conjunction with TPEF of an extrinsic label, and SHG from structured collagen to observe significant differences in nuclear area of normal and cancerous regions in human tumors and murine tumor models. Yue et al.242 used SRS νCH2,sym contrast for morphological imaging in liver and prostate tissue, and for guidance in identifying lipid droplets (LDs). They then used spontaneous Raman spectroscopy at LD positions to determine that the cholesterol ester content of the LDs was correlated with the ratio of 702 and 1442 cm−1 peak intensity ratios, and with the aggressiveness of the cancer.
Colon tissues from 9 patients were imaged by spectrally scanned SRS in conjunction with SHG, THG and TPEF.244 They found that an increase in protein (νCH3,sym and Amide I), and water (3250 cm−1), and a decrease in lipid (νCH2,sym, 1445 cm−1, 1745 cm−1) all correlated with presence of adenocarcinoma. They reported similarity in TPEF of NADH and 1665 cm−1 contrast images, suggesting that increased protein could be due to increased metabolic activity. They also suggested that increased water in cancerous regions could be due to increased aquaporin protein production.
Tissue involved in and surrounding a tumor is typically sampled and analyzed by histology or for biomarkers at various time points to guide therapy by establishing prognosis and following disease progression. Accordingly, inappropriate tumor sampling is thought to be a leading problem in both prognosis and evaluating treatment response.245
Tissue sampling by needle biopsy or exploratory surgery is often guided by palpitation, preparative imaging, or visual inspection, and is notoriously inefficient.245 Neoplasms may be difficult to properly sample at early stages, and the ability to categorize tissue as normal or neoplastic during the biopsy process would go a long way towards ensuring high-quality samples for subsequent analysis. Spontaneous Raman was demonstrated decades ago as an potential guidance tool for needle biopsy,246 and progress has been made toward clinical adaptation, but challenges remain with regards to low signal levels, and typically, Raman needle probes must be coupled with other detection modalities.113 Coherent Raman methods could easily provide sufficient signal levels for this application. Narrowband CRI approaches have been implemented in fiber,247–249 so could be adapted to needle biopsy, but are probably not suited for this application since disease detection would be based strictly on spectra, and thus benefit greatly from sensitivity to multiple peaks in the fingerprint spectral region. Current broadband spectroscopic CRI approaches based on impulsive excitation or spectral focusing will be difficult to implement in a fiber delivery, but other approaches may be possible in the future.
While narrowband CRI may not be appropriate for purely spectroscopic disease detection, in needle biopsy, it is effective in a microscopy modality for in vivo detection and analysis of skin carcinomas, which are present in the optically accessible epidermis layer of the skin.211,221,236,250 New approaches to signal detection may push the useful range of in vivo CRI to slightly deeper tissues,251 and to detection of fingerprint signals.229
Another widely recognized clinical problem is the need for intraoperative analysis of tissue at a resection margin. Strong correlations exist between presence of cancer cells in surgical margins and recurrence rates of cancer,252,253 and incomplete margin resection is held as a major cause for recurrence of cancers that call for tissue conservation, such as in brain and breast.
A hand-held probe is required for in situ margin analysis, and appropriate equipment has already been described for narrowband CRI247–249 although, to our knowledge, clinical studies using these probes have not yet been published. Use of a hand-held Raman probe for identification of tumor tissue in surgical margins was clinically demonstrated for brain surgery by Jermyn et al.254 and proof of principle for breast was demonstrated by Kong et al.255 As with Raman needle probes, tumor detection in these reports was based on spectroscopy alone, thus suffering somewhat from low signal and precluding imaging. This difficulty was avoided in the breast application by using autofluorescence levels to identify regions of interest, and those regions were imaged sparsely with Raman. In the brain application, the surgeon visually selected regions of interest.
Although CRI technology has not yet progressed to in situ margin analysis, it has been demonstrated as an effective tool for characterizing freshly excised, unprocessed tissue.222 Typically, margin tissue must be frozen and stained for imaging. This process can add 20 minutes to a surgery each time a margin is to be investigated. The label-free nature of CRI, and the ability to image in a backscattering mode makes it possible to obtain histology-quality images from surface regions of thick, unprocessed tissue, significantly reducing the delay between tissue removal and analysis.222
Another area in which speed of information availability would be useful is in initial evaluation of histology slides. The majority of tissue in most slides is normal, so a pathologist may spend a significant portion of their time searching for potential diseased regions. One particularly strong benefit of chemical imaging is that it can provide clear indications of tissue type and regions of normal and potentially abnormal tissue. This regional tissue typing has already been demonstrated with IR on slides from which high-resolution H&E images have been obtained, and the two images overlaid.161 CRI methods can provide both the regional tissue typing201,207,210 and the high-resolution H&E mimic without the need for staining or imaging in multiple contrast modes.201,202,222
An important example of how CAD can bridge this gap is through providing basic quantification of image features. As mentioned previously, non-specialist pathologists were found to be much less precise in cell counts and classification than sub-specialists,88 no doubt contributing significantly to overall diagnostic uncertainty. Many of the same factors contributing to non-specialist variability, including the fact that cells are often overlaid in H&E images, with no clear delineation between nuclei,88 are also challenges to the families of morphological image analysis on which CAD is based. Further, factors such as variability in staining protocols, and in image field illumination, and issues with nonspecific staining contribute to overall image variability, adversely impacting reliability in thresholding and feature identification for CAD. These, and related issues are expected to be significant barriers to widespread adoption of CAD. In recognition of this, the US Food and Drug Administration has issued guidelines for performance of WSI instrumentation.57
CRI methods can ameliorate or resolve many of the reproducibility and reliability issues in image formation. For example, obtaining reliable cell counts will be easier because CRI has intrinsically high axial resolution, making it feasible to resolve stacked nuclei. Additionally, where imaged nuclei are spatially close, there will be a nuclear and cellular membrane signal between them, strengthening the ability of CRI-based analysis to arrive at accurate cell counts. Yang et al.206 have already demonstrated 90% accuracy and reproducibility in automated cell counting using coherent Raman imaging with only νCH2,sym, and thus poor nuclear contrast.203 This is an important near-term opportunity for coherent Raman methods.
In addition to intrinsic z-sectioning, CRI can provide a large number of contrasts through various vibrational bands. Since increased image dimensionality generally leads to higher precision segmentation and feature identification,89–91 one could expect improved performance in these aspects with these multi-contrast images. The Wong group have derived morphological parameters familiar to pathologists, such as cell shape and density, from CARS images and based on these were able to identify with 80 to 95% accuracy breast tumor types.206
Although CRI requires little or no sample preparation, it seems likely that most early implementations will include fixation and paraffin embedding of samples, simply for consistency with protocols for staining. In fact, both of these steps have potential to introduce variability in CRI signals. Paraffin has a strong CH stretch signal, and fixing agents can liberate diagnostically important molecules, particularly lipids.256 For example, compared to fresh frozen tissue, formalin-fixed brain showed modification in νCH2,assym and νCH3,assym amplitudes from CARS, but not in νCH2,sym. In the fingerprint, only choline groups of phospholipids at 717 cm−1 dropped slightly after formalin fixing. However, methanol or acetone fixation severely reduced all of these peaks and others.257
Fingerprint CRI provides chemical specificity identical to that of spontaneous Raman spectroscopy,218 and therefore seems likely to be a potential substitute for most or all stains, and perhaps many IHC agents. Raman spectra can be used to recapitulate many image contrast agents used commonly in biology.258 Also, as shown by the partial listing in Table 2, fingerprint Raman has already been used to detect many diagnostic markers. With fingerprint CRI, these will be available as label-free image contrasts. An important added benefit of spectroscopic CARS is that it provides an internal reference based on NRB,198 so that spectroscopic CARS images can serve as quantitative, rather than qualitative maps of markers. This level of quantification cannot generally be achieved with stains, fluorescence, or even spontaneous Raman. The quantitative imaging and absence of non-specific staining will greatly simplify image analysis and increase confidence in image metrics and associated diagnostic decisions. A few specific examples of potential important contributions are given below.
The number density of mitotic cells plays an important role in diagnosis and grading of many cancer types,8 and is usually determined by IHC or cell morphology analysis. The best morphologically-derived automated detection of mitoses achieve F-scores (the geometric mean of precision and accuracy) between 75% and 80%.87 It seems likely that fingerprint CRI imaging could yield significantly better results. By mapping the intranuclear distribution of DNA, RNA, and chromatin, Raman spectroscopic imaging can not only positively identify whether a cell is mitotic, but which stage of mitosis it is in.163,166,167 The chemical distribution information is essentially orthogonal to morphological analysis, so can only serve to increase precision and accuracy in determining mitosis counts.
Accurate characterization of pleomorphism (atypical cell shape) is also important to proper diagnosis of many cancers.8 Pleomorphic cells of one type may be mistakenly identified as normal cells of a different type, and this was part of a 42% cell count and classification disagreement among non-specialists found by Fuchs et al.88 It is well established that Raman spectroscopy can be used to positively identify cell type.162 Applying spectral cell identification as an orthogonal metric will facilitate much better recognition of pleomorphism.
Invasion of immune cells into a tumor site, and their state of activation is an key factor in diagnosis of some cancers.8 As mentioned above, morphology-based cell identification can be difficult, and determination of immune cell activation is currently done by assessing the presence of cell surface markers, typically by IHC. For this important problem, fingerprint spectroscopic imaging will not only allow immune cell identification, but also whether they are activated.107,168
Identifying the tissue from which of CUP (cancer unknown primary) metastases originate is useful for identifying appropriate therapeutic strategies, but presently requires involved analysis.259–261 Fingerprint spectroscopic CRI may be able to positively identify metastatic tumor origins simply from chemical content of the tumor cells. Even narrowband CRI studies were able to discriminate primary (metastasized) cancer cells from cells belonging to the target tissue simply on basis of contrast at νCH2,symm and νCH3,symm.119,207
In recent years it has become clear that genomic subclasses of cancers can have distinct clinical outcomes,262 and thus require varied interventions. While spectroscopic Raman imaging has not been evaluated as a tool for identifying these subclasses, it seems reasonable that it may have power to do so in some cases. CRI will not directly detection genome changes, but it is likely to detect downstream consequences of those changes. For example, cell phenotype162 and function107,162 can be discriminated using Raman spectra. These spectral changes come not from DNA sequence or methylation modifications directly, but from the intra- and extra-cellular changes in bimolecular profiles they induce. Furthermore, microenvironment properties determined through histology have recently been found to correlate with the genomic cancer subtypes, and these microenvironment changes should be detectable with CRI in most cases.263
Broadband CRI tissue imaging using CH stretch has been demonstrated at 20 Hz for CARS185 and at 4 kHz for SRS223,240 over 300 cm−1. Fingerprint CARS spectro-microscopy has been demonstrated at 300 Hz over 3000 cm−1 in tissue.218 Overall, these range in speed from 105 to 103 times slower than WSI. This difference may be difficult to surmount. Assuming the 0.5 cm2 min−1 imaging speed is indeed an appropriate mark, it seems that present spectral CRI methods are not good candidates, although some newer spectroscopic CRI approaches may hold promise. A spectral focusing approach has been demonstrated at 77 kHz for a 200 cm−1 bandwidth,251 although not yet in tissue, and a family of approaches featuring ≈25 kHz spectroscopic CARS acquisition has been demonstrated,264,265 but with laser pulses incompatible with tissue imaging. Broadband SRS266,267 has been demonstrated with potentially fast imaging, but seem to suffer from insufficient modulation sensitivity.
Assuming that signal generation and collection efficiency of current CRI instruments is nearly optimal, and there is no way to intrinsically speed up image acquisition, there are at least two non-exclusive options for applying spectroscopic CRI to histopathology. One is to use a faster imaging modality (such as narrowband CRI) to generate a guide image from which regions of interest can be determined, and use spectroscopic CRI over ROIs covering a small fraction of the specimen. The other approach would be to use spatial multiplexing to effectively increase the rate of spectroscopic signal acquisition.
Provided sufficient excitation laser power, spatial multiplexing approaches can yield speed increases proportional to the number of detection elements. For spectroscopically detected signals, such as in spontaneous Raman scattering, 1-D (line) spatial multiplexing can realistically lead to 20×–100× imaging speed increase.164 This approach could also be used for spectroscopic CARS. 2-D multiplexing has been demonstrated with narrowband CARS268–272 though not in ways suited for clinical applications. Spectroscopic approaches using single element detectors, such as time-domain264 or spectrally scanned195 CARS could also be 2-D multiplexed, so considerable speed increases are conceivable, provided laser sources of sufficient pulse energy can be found to spread light over a larger area and still obtain signal. SRS will be more challenging to implement in spatially multiplexed modes since it requires very high sensitivity to small amplitude modulations. 2-D array detectors, such as CMOS cameras can be used in differential or lock-in mode, but these have yet to demonstrate sensitivity that would be needed for tissue imaging.266,267 Slipchenko et al.273 introduced an array of 32 LC circuits for simultaneously detecting SRS signal in as many channels, which has shown sufficient sensitivity for tissue imaging.
Finally, we note that routine use of spectroscopic CRI in histopathology could present significant data storage issues. A single-contrast slide image from a WSI instrument may occupy 1–2 Gb of storage. At a rate of 500 images a day, storing digital histopathology data is already a challenge. If all such images were acquired in full spectral mode, the data storage load would increase by 103 times – an untenable proposition. Such a scenario is neither likely or necessary; there are at least two factors that would mitigate the need for such excessive storage. One is that spectroscopic imaging would be particularly beneficial for only small areas of most tissues. The other is that, while spectral images could be stored temporarily, for processing, it quite unlikely that saving all spectral dimensions will be necessary in most cases.169 Data reduction methods such as those discussed in Sections 2.7.3 and 3.1.1 could be applied to determine the most important spectroscopic information, allowing to discard the rest and reduce the size of images for long-term storage.
Intra-surgical microscopic analysis of excised tissue has already been demonstrated. In situ use will likely require a hand-held probe with fiber-delivered excitation light and signal collection. Whether in situ tissue analysis will be best done by narrowband coherent Raman imaging or by coherent Raman spectroscopic analysis is unclear at this time.
Another significant opportunity for CRI is facilitating improvements in histology-based diagnoses. The art of histopathology emerged near the end of the 19th century in nearly its present form. Significant new knowledge of disease mechanisms has improved interpretation of histological features, but, apart from the introduction of IHC in the 1940s, histopathology tools in use today are largely the same as they were in the early 20th century. Efforts to modernize histopathology practice are making headway, however, fully transforming it from a subjective art to a quantitative and objective science remains a challenge. Most of these challenges involve variability and contrast reliability in image formation. It appears that many of these issues, could be largely resolved with CRI. Furthermore, spectroscopic CRI may provide a significantly enhanced contrast space that will open new options for quickly stratifying cancer and other diseases, leading to a more personalized diagnosis.
While opportunities for CRI to improve clinical practice are clear, important barriers remain for widespread adoption in histology practice. An increase in imaging speed for narrowband CARS and a slightly larger increase for SRS would be helpful. For spectroscopic CRI, substantial imaging speed increases will be necessary if they are to be used to a significant extent in an imaging mode. Fortunately, the CRI field is still vibrant with new innovations for increased imaging speed and chemical selectivity and sensitivity. Much work remains to demonstrate the utility and reliability of these methods to the medical community. Finally, potential regulatory issues must be considered and resolved to the extent that they are found.
Footnote |
† Official contribution of the National Institute of Standards and Technology; not subject to copyright in the United States. |
This journal is © The Royal Society of Chemistry 2018 |