1.

Introduction

Heart failure (HF) can result from any structural or functional abnormality that impairs the ability of the ventricle to eject blood, referred to as systolic HF, or to fill with blood, termed diastolic HF. Prevalence of HF is widespread in the USA with 5.7 million people diagnosed; projections estimate an increase past 8 million by 2030.¹ The economic cost of HF was estimated to be $31 billion in 2012.¹ More than 40% of the cost is attributed to hospitalization, with volume retention and overload being the most common reason for HF hospitalization. According to a study of more than 1.3 million HF patients, 24.8% of patients were readmitted within 30 days of discharge.² The American Heart Association has launched a program called Rise Above Heart Failure that assists healthcare professionals in educating patients concerning heart failure.³ In addition to patient education, there is a need to better understand the disease state and monitor patients both inside and outside a clinical setting in order to reduce hospital length of stay and patient readmission rate, improve quality of life for HF patients, and reduce the economic burden of HF.

HF is diagnosed based on a patient’s signs and symptoms, past medical history, family history, and test results, including blood tests and imaging. Fluid retention is one of the cardinal manifestations of HF and may accumulate in the lungs, abdomen, and extremities. Peripheral edema describes interstitial fluid volume overload in the lower extremities and is a sign of hypervolemia and progression of disease. Edema occurs when an excessive volume of fluid accumulates in the tissues, either within cells (cellular edema) or within the collagen-mucopolysaccharide matrix distributed in the interstitial spaces (interstitial edema).

The traditional clinical assessment of peripheral edema in HF patients, which is still widely used, is subjective. In this examination, a clinician applies pressure with his or her index finger to an anatomical point, usually the patient’s ankle or mid-tibia. The anatomical point is subjectively chosen to represent the location of greatest edema. This technique captures pit depth and the time needed for the skin to return to its original appearance (recovery time) as a single edema score ranging from 0 to $4+$.⁴ This measurement will vary upon the experience of the clinician, time constraints, and individual patient factors.⁵ Alternative, non-invasive measurements have been investigated such as ankle circumference, but these methods lack reliability, feasibility, or correlation with the classic clinical assessment of pitting edema.⁵ Assessment of edema in the upper extremities (forearm) is not practiced clinically. Measurement of daily weight as a surrogate for volume retention is fraught with inaccuracies and is clinically unreliable.⁶

Additional methods to track patient progress including thoracic impedance measurements, pulmonary lung fluid tracking, pulse oximetry, and blood testing for natriuretic peptides are in various stages of commercial development.^7–10 A non-invasive and non-contact monitoring option, such as imaging, would lead to reduced costs, reduced hospital readmission rate, and greater patient compliance. Invasive implantable devices, such as CardioMEMS, have shown a reduction in hospitalization rate compared to control groups.¹¹ The implantable devices routinely monitor pulmonary artery pressure and edema but not peripheral edema. There is currently no commercially available imaging modality that could accurately measure and monitor peripheral edema in HF patients.

This study explores the use of short-wave infrared (SWIR) molecular chemical imaging (MCI) to non-invasively and objectively measure the level of edema in a patient’s limb. MCI has been explored for several medical applications due to the non-contact, non-destructive nature of the technique and use of non-damaging illumination that can be used to obtain multiple measurements from a given patient as needed.¹² These characteristics are useful in medical applications in which invasive tests can lead to patient discomfort or non-compliance with physician testing schedules. SWIR MCI is potentially suitable for quantitatively measuring subcutaneous interstitial fluid due to water being a strong absorber in the SWIR spectral range. We expect to see evidence of higher water absorption in SWIR spectra from subjects with edema than from subjects without edema. Additionally, the SWIR wavelength range exhibits decreased scattering when compared with the more conventionally exploited near-infrared (NIR) range. Furthermore, the longer wavelengths in the SWIR allow deeper penetration of light into tissues.^13,14 The penetration depth with SWIR will allow the measurement of patients with varying skin thickness, such as obese patients, and has relative insensitivity to skin pigmentation.¹⁵ The epidermis layer varies by body location, averaging 0.1 to 0.15 mm in thickness; the dermis layer can be 1.5 to 4 mm; and the subcutaneous layer is thicker than the dermis.¹⁶ The study described here serves as an opportunity to establish technical feasibility and inform future directions for the potential development of a clinical instrument. The objectives were to investigate a primary endpoint of discrimination between healthy and HF patients with edema and a second primary endpoint of whether a SWIR-based index [the CardioVerification Index (CVI)] could be conceived and used to predict the pitting edema level using the ground truth provided by the traditional pitting edema test.

2.

Methods

2.2.

Instrument

A tower-based sensor was used to facilitate measurements of patients with limited or no mobility. Figure 1 shows the imaging system and major components. The sensor head unit (SHU) used in this study has been previously described in a benchtop configuration that was used to detect illegal drugs in correctional facility mail.¹⁷ The benchtop configuration was impractical for imaging limbs in the present study, therefore, the sensor was incorporated into the tower configuration to allow for movement of the instrument between data collection locations. Briefly, this sensor collects hyperspectral images via a SWIR multiconjugate filter (Gen 5 SWIR multiconjugate filter, ChemImage Corp., Pittsburgh, PA) that filters light, which is subsequently captured by the detector, a SWIR InGaAs focal plane array (TAU SWIR camera, FLIR, Santa Barbara, CA). This SHU is attached to an articulating arm, which enables it to be positioned over patient limbs, normal to each limb, at an approximate working distance of 70 cm (Fig. 2). A light housing has been attached to the SHU and is equipped with six 20 W halogen light bulbs (MR-16 EuroStar-Refleckto #1000010, Ushio, Tokyo, Japan) for illumination of the $163\mathrm{mm}\times 124\mathrm{mm}$ field of view. Motorized focus optics were used by the instrument operator to maintain focus between limbs and patients. Once positioned, the operator controls data collection via Spectral Kitchen^® software (ChemImage Corp., Pittsburgh, PA), which runs on the workstation that has been integrated onto the tower.

Fig. 1

(a) Tower-based SWIR MCI instrument with main components. (b) Close-up view of the SHU attached to the articulating arm. (c) View of the lighting and optical components on the SHU.

Fig. 2

SWIR MCI of a healthy volunteer shin. Future form factors are envisioned to not require a tower and will be handheld.

Daily operational qualification testing was completed to confirm that the spectral and imaging quality of the instrument was suitable for collection. Instrumentation used for this study will be useful in future studies; however, a smaller, handheld sensor will allow for increased ease of use among intended use demographics. The utility of SWIR sensors is highlighted by effective use in biological and non-biological applications.

2.4.

Data Collection

Protocols and consent forms were developed by the investigators with review and approval by the Western Institutional Review Board. Prospective patients were screened against criteria defined in Table 1. Enrolled patients were assigned a deidentified study ID and metadata was collected. Pitting edema level in each shin was measured by clinical staff $\sim 10\min$ prior to measurements. Compression stockings, if applicable, were removed followed by a minimum 5-min recumbent period before data collection. Data collection involved an RGB image and a series of SWIR images covering 1000 to 1700 nm. For a given field of view, individual wavelength frames were captured at 5 nm intervals across this spectral range, with an exposure time of 32 ms. The set of wavelength frames is referred to as a hypercube, a three dimensional stack of images, where $X$ and $Y$ are spatial dimensions and the $Z$ dimension represents the spectral dimension. Therefore, each pixel location in a hypercube has an individual spectrum. Collection of each hypercube takes about 50 s.

Table 1

Criteria used to screen patients for enrollment.

No.	Item	Criteria status
1	Males or females age 18 to 90 years	Inclusion (all cohorts)
2	Evidence from history, physical examination, imaging studies, and laboratory tests that support the diagnosis of HF	• Inclusion for HF cohort • Exclusion for healthy volunteers
3	Subjects who are incapable of providing consent may be non-compliant or whose care could be compromised	Exclusion (all cohorts)
4	Subjects who have a confirmed or suspected pregnancy	Exclusion (all cohorts)
5	Subjects with overt skin lesions	Exclusion (all cohorts)
6	Patients who have peripheral edema caused by conditions unrelated to HF	Exclusion (all cohorts)
7	Patients with a diagnosis of diabetes (type I or type II)	• Exclusion for healthy volunteers• Not an exclusion for HF cohort

Each patient had both shins and forearms imaged, and all data that met quality standards were included in the analysis. A typical data collection for all four patient limbs was 15 min or less. Data were collected using ChemImage-developed Spectral Kitchen^® software. Periodic reviews of enrollment were conducted by clinical staff, and de-identified metadata were shared with the sponsor. MCI technology is used for diffuse reflectance measurements where reflected light is measured from a patient’s limb, and data are recorded at specified wavelengths in a hypercube as described in Fig. 3.

Fig. 3

Overview of MCI data collection and processing.

3.

Results

Figure 4(a) displays example shin and forearm RGB images from a patient with edema and a volunteer without edema, whereas Fig. 4(b) shows a spectral image at 1200 nm indicating ROIs from which spectra were sampled. The images in Fig. 4(b) are displayed at a consistent wavelength between limbs using 1200 nm; however, any of the evaluated SWIR wavelengths could be used to illustrate the data. Figure 4(c) shows example processed images that may reveal features, such as bone or vascular locations not obvious in RGB or raw spectral images. Spectra were extracted from soft tissue regions to eliminate influence from the tibia, which is more reflective. The extracted regions represented several areas of the shin to account for local edema changes and variability of molecular species. Figure 4(d) displays representative reflectance spectra from the ROIs for both shins and forearms.

Fig. 4

Example data from a healthy volunteer and an edema patient. (a) RGBs of a healthy $\mathrm{PE}=0$ volunteer shin (left), $2+$ edema shin (middle left), healthy $\mathrm{PE}=0$ volunteer forearm (middle right) and a $2+$ edema forearm (right). Areas sampled for SWIR analysis indicated by blue boundaries. (b) Spectral images at 1200 nm and ROIs were used for spectral extraction. (c) Processed images displaying bone and soft tissue regions in shin and vasculature in forearm images. (d) Average spectra from the ROIs.

The principal macromolecules represented in the SWIR spectra are water, lipids, and collagen.²⁵ Lipids, and to a lesser extent, collagen and water, absorb between $\sim 1120$ and 1230 nm.^25,26 The peaks in this spectral region are associated with the second overtone of $\mathrm{C}─\mathrm{H}$ stretching in lipids and collagen and a vibrational overtone of the O–H bond in water. The larger envelope from 1300 to 1650 nm encompasses mostly collagen and water, peaking at $\sim 1480\mathrm{nm}$, and some lipid, absorbing closer to 1400 nm.^25,26 The presence of collagen is reflected in the absorption peaking at $\sim 1500\mathrm{nm}$, representing a combination of ${\mathrm{CH}}_2$ stretching and non-stretching modes. Water and lipids are represented by the first overtone of OH stretching, closer to 1430 nm.

A portion of collected data was not suitable for analysis due to incorrect wavelength step size (wavelength distance between collected images in a hypercube), file save failures, file corruptions, or protocol issues such as the absence of the flatfielding correction standard (99% reflectance standard disc) or incorrect standoff distance. In total, 27 of 94 shins were removed from the HF population, corresponding to 11 subjects in which both shins were excluded, and 5 subjects in which one shin was excluded. Similarly, 12 forearm data sets were found to be not evaluable. The removal of data due to quality included one volunteer in the healthy population who had an edema level other than 0 and another with a diagnosis of diabetes.

The primary analysis endpoints for the study were to (1) resolve if SWIR MCI can accurately discriminate between healthy and edema patients and (2) investigate if SWIR MCI can accurately discriminate amongst all pitting edema levels. Acceptance criteria for each endpoint were established as an accuracy of 80% or greater. Two-class PLS-DA models were built to discriminate between healthy patients without pitting edema and HF patients with all levels of pitting edema. The distribution of edema in evaluable limbs is summarized in Table 2. Pitting edema ground truth level was measured in the shin area as part of the routine care of HF patients. Forearm pitting edema was not collected as part of this pilot study, and the applicable shin ground truth was applied to the forearm data. The forearm population was selected using corresponding forearm data from the final shin population. Some data sets from the forearm population did not meet the data quality requirements outlined above and were removed from the evaluated data.

Table 2

Distribution of edema in patients.

The discrimination plots for the shin and forearm models are shown in Figs. 5(a) and 5(b). The ROC curves, shown in Fig. 5(c), correspond to an AUC of 0.993, accuracy of 97.1%, sensitivity of 97.0%, and specificity of 97.2% for the shin population. The test for forearm edema had an AUC of 0.993, accuracy of 96.7%, sensitivity of 98.2%, and specificity of 94.4%. Figure 5(d) displays the average spectra from the edema and healthy population shins and forearms. These results are encouraging, and suggested additional analysis be undertaken to understand if discrimination between levels of pitting edema is feasible.

Fig. 5

Differentiation of healthy and edema populations using PLS-DA. (a) Discrimination plot for shin data. Shin data are classified as edema for values above the dotted threshold. (b) Discrimination plot for forearm data. Forearm data are classified as edema for values above the dotted threshold. (c) ROC curves for two class PLS-DA models discriminating healthy patients from HF patients with peripheral edema. (d) Average reflectance spectra for edema and healthy shins and forearms used in models.

The second primary endpoint was investigated using hierarchical PLSR to build a model using extracted spectra and the ground truth edema level as the dependent variable. The shin model generated CVI scores that predict healthy and HF patient edema levels with an accuracy of 81.6% when compared to the ground truth. This model was then applied to shin hypercubes to generate score images that enabled visualization of edema in participant shins. Figure 6(a) shows box plots of predicted pitting edema level versus the provided ground truth with the standard deviation represented as a box. The red dot indicates the median, the center line in the box is the mean, and the bottom and top edges of the box represent the 25th and 75th percentiles, respectively. The “whiskers” around the box plot represent the extremes of the distribution that are not considered outliers and the outliers are represented by open and closed circles, where open circles are outliers and closed circles are extreme outliers. Figure 6(b) comprises the PLSR score images generated when this model is applied to subject hypercubes. Each pixel in the image is generated by multiplying the spectrum for that pixel by the regression vector that is the output of the PLSR model building exercise. By processing every pixel, a score image for that model is assembled. The average intensity of the pixels, in general, increases as the predicted edema level increases. The same methodology was applied to the forearm population as shown in Fig. 7. A hierarchical PLSR model using forearm data discriminated edema levels with an accuracy of 63.8%.

Fig. 6

(a) Box plot for hierarchical models comparing the predicted shin edema level using CVI scores and ground truth provided by clinical staff. (b) Output images for the hierarchical PLSR model. Shins are displayed in row-order of increasing ground truth edema level. Shins 1 to 36 starting in upper left corner ($\mathrm{PE}=0$ ground truth; red box); shins 37 to 46 ($\mathrm{PE}=1+$ ground truth; green box); shins 47 to 60 ($\mathrm{PE}=2+$ ground truth; blue box); shins 61 to 78 ($\mathrm{PE}=3+$ ground truth; yellow box); and shins 79 to 80 ($\mathrm{PE}=4+$ ground truth; purple box). The overall classification accuracy for this model is 86%.

Fig. 7

(a) Box plots for hierarchical models comparing the predicted edema level using CVI scores and ground truth provided by clinical staff. (b) Example output images for the hierarchical PLSR model. Forearms are displayed in row-order of increasing ground truth edema level. Forearms 1 to 36 starting in upper left corner ($\mathrm{PE}=0$ ground truth; red box); forearms 37 to 46 ($\mathrm{PE}=1+$ ground truth; green box); forearms 47 to 58 ($\mathrm{PE}=2+$ ground truth; blue box); forearms 59 to 76 ($\mathrm{PE}=3+$ ground truth; yellow box). The overall classification accuracy for this model is 71.1%.

4.

Discussion

In this pilot study, the primary endpoint of a non-invasive and non-contact SWIR MCI-based approach to discriminate between limbs with and without edema was achieved as judged by the high accuracy of the two-class PLS-DA model. The second primary analysis endpoint of predicting edema level in patients gave encouraging results, but a larger study is needed to obtain and train on greater populations for all edema levels. The majority of biological tissue-based spectroscopic studies have focused on the measurement of chromophores such as hemoglobin, which absorb in the visible wavelength range.^27–29 However, the SWIR region offers benefits that have not been exploited in high volume until more recently, due to recent developments in SWIR-based optics, e.g., InGaAs sensors.¹⁵ First, the SWIR spectral region is highly favorable for thicker samples due to the longer wavelengths and transparency of the region to blood. Additionally, the absorption coefficients of key chromophores: water, lipids, and collagen are higher in the SWIR region compared with the visible and NIR regions. For example, the absorption coefficient of water is more than 60 times higher at 1440 nm than at the 975-nm NIR peak.³⁰ In a similar trend, lipid absorbs many magnitudes more at 1210 nm than at 930 nm.^28,29

The basis of discrimination among the populations can be explored by comparing spectra from the healthy and edema populations. Figures 8(a) and 8(b) show average normal and edema reflectance spectra representing the shin and forearm data, respectively, used to generate the two PLS-DA models. In the key region of 1120 to 1230 nm, it is clear that absorption is higher in spectra from subjects with edema in both shins and forearms. We can exploit spectral analysis further by evaluating the variable importance in projection (VIP) scores from the model. Shin and forearm VIP scores are shown alongside average model spectra in Figs. 8(a) and 8(b), respectively. A VIP score is a measure of a variable’s importance in a PLS model. VIP scores can be used for variable selection and for identifying the spectral regions of importance in a model.³¹ A variable (in this case, wavelength) with a VIP score $>1$ can be considered important in a given model. The VIP scores associated with the PLS-DA shin model indicate the spectral regions that play a significant part in discriminating between healthy subjects and those with edema, regardless of grade. Several spectral locations with VIP scores $>1$ are evident. As expected, these locations represent all three macromolecules—lipids, collagen, and water. VIP scores higher than 1 at 1140 and 1150 nm reflect water content; 1200 and 1235 nm correspond to lipid and collagen absorption. However, the highest VIP scores are at 1385, 1465, and 1530 nm. This broad spectral envelope corresponds to the absorption of mostly water and collagen, and lipids to a lesser extent. At 1385 and 1465 nm, OH stretching likely causes the observed absorption, and at 1530 nm, we expect there to be water influence as well as contribution from ${\mathrm{CH}}_2$ stretching and non-stretching vibrations in collagen. The forearm VIP scores are very similar to the shin VIP scores.

Fig. 8

(a) Average shin reflectance spectra and VIP scores calculated from the PLS-DA model for edema in shins. (b) Average forearm reflectance spectra and VIP scores calculated from the PLS-DA model for edema in forearms.

What these analyses tell us is that not only are edema and normal shins and forearms distinguishable using SWIR MCI, the basis of discrimination is water content in addition to collagen and lipid differences. Because subjects with high levels of edema can have as high as a 70% increase in water content,^19,20 we estimate that water differences impart a high bearing on differentiating between edema and normal tissue spectra.

Among other collected metadata, the influence of diet and medications will be monitored to determine effects on results during treatment. Alternate prediction strategies, such as artificial intelligence (AI), may be employed when a larger data set is available with greater patient populations. Currently, PLSR is a good choice for model building, given the limited and the spectral nature of the data. Deep neural networks are often chosen for machine learning and AI but are a better choice when the data consists of large training sets of images. Certain AI tools, such as data augmentation, will be investigated via spectral additions of water to current samples to increase the training set coverage and provide more robust models. Greatly increased data collections for an individual patient when used in a monitoring mode could allow for adaptive background estimation which could better model the spectral background. Next generation handheld sensors will incorporate more advanced computer vision techniques for standoff distance and angle calculations as well as for motion correction.

The prediction accuracy of the shin model is greater than that of the forearm model. The ground truth for a given shin was used as ground truth for the corresponding forearm. This may have influenced the degradation of forearm accuracy, as shin edema may be more severe due to gravity. The effect of sampling location will continue to be investigated, as flexibility in this area will increase clinical utility for patients with physical impairments, morbidly obese populations, and limited mobility patients. Evaluating the forearm, for example, which is not currently used to measure pitting edema, allows us to assess the upper extremity as an option for edema measurement, which would give us flexibility when designing future form factors.

Subsequent clinical trials will, when possible, include a more diverse population in terms of ethnicity, pitting edema level, and age to better capture biological variability and increase the robustness of models.

In an additional experiment, individual two-class PLS-DA models were built from subjects’ right and left shin measurements. Using individual shin pitting edema ground truth, the accuracy of the right shin model was higher than that of the left side shin model (100% versus 96.1%, respectively; data not shown). Although not currently statistically different, we will continue to monitor this trend in future clinical trials to understand if sampling from multiple limbs is necessary, and to understand optimal sampling location. Any metadata in a patient’s history that may influence the results, such as a heart bypass operation or which veins were harvested from a patient’s leg for bypass surgery, will be monitored.

The feasibility of monitoring dynamic peripheral edema levels in patients over time, including patients who are receiving treatment, will be investigated in upcoming clinical trials. Ultimately, these trials will help understand if MCI technology used consistently on a periodic basis can reduce readmission rates for HF patients. Efforts are underway to develop a hand-held form factor that will automate features where possible to reduce the training burden. These efforts are being conducted by CardioVere, a spinout of ChemImage. The eventual goal is to develop strategies to automatically integrate MCI results into the electronic health record to alert clinicians when a patient has a condition change and requires clinical intervention.

We acknowledge some of the limitations of the study, including underrepresentation of non-Caucasian ethnicities and patients with 4+ pitting edema. Data sets from four enrolled non-Caucasian patients met data quality standards and were included in the reported models. Although we currently do not have evidence suggesting that a change in performance would occur when including a larger number of non-Caucasian patients, this will be investigated in future studies with a larger patient population. Though we only included HF patients with pitting edema, it is possible that some of the patients had non-pitting edema, which may have influenced the results. The cohort without HF had a lower average age than the cohort with HF. Skin thickness may decrease with increased age and this will be considered in future protocols. Finally, this study included forearm assessment for edema, which is typically not part of routine evaluation of HF patients. Understanding the body locations useful in the evaluation of edema will be explored in additional studies. Although this study only obtained ground truth from the shin area, the protocols for future studies are anticipated to collect ground truth from each location sampled. Importantly, the evaluation of sampling locations is not limited only to the extremities used in this study.

5.

Clinical Perspectives

Tracking volume status in HF patient care is vital to prevent the progression of the disease. Accurate tracking helps to manage symptoms better by making simple lifestyle changes and modifying therapeutics. This technology may provide a reliable, objective, non-invasive, and non-contact methodology for quantitative peripheral edema measurement. As the technology matures, it is envisioned that patient self-monitoring, with wireless transmission of edema levels to physicians from home, can aid clinicians in monitoring HF patients for improved patient outcomes and to improve the future of individualized medicine. This increases the chance of modifying treatment plans appropriately based on accurate monitoring of edema level. This ultimately may reduce HF hospital readmission rates and unscheduled, emergent visits to the clinic or emergency room.