2.1.

Subjects

Ten volunteers [$\text{age}=37.8\pm 15.3\text{years}$ ($\text{mean}\pm \mathrm{S.D.}$), range 21 to 65 years, $\text{male}/\text{female}=7/3$, $\text{height}=1.74\pm 0.08\mathrm{m}$, $\text{weight}=71.3\pm 10.3\mathrm{kg}$, $\mathrm{BMI}=23.8\pm 4.0\mathrm{kg}/{\mathrm{m}}^2$] enrolled from The Blizard Institute, Barts & The London School of Medicine & Dentistry, participated in this study. None of them had any known cardiovascular disease, and none were diabetic. The investigation conformed to the principles outlined in the Declaration of Helsinki (1989) of the World Medical Association, and it was approved by the local Research Ethics Committee. The nature of the research was explained to the subjects before the recordings, and their informed consent was obtained. All subjects were asked to refrain from consuming caffeine or alcohol, and they were asked not to smoke or undertake strenuous exercise for the 2 h preceding the study.

2.2.

Instrument Setup

The iPPG system is schematically presented in Fig. 1(a). A monochrome CMOS camera (model: EoSens, MC 1363-63, Mikrotron GmbH, Unterschleissheim, Germany) with a spectral range of 500 to 1000 nm was focused on the palm of the participant’s hand using a standard F-mount lens (model: Nikkor 20 mm, $\mathrm{f}/2.8\mathrm{D}$, Nikon, Japan). Two commercial infrared ($\lambda =880\mathrm{nm}$, $\mathrm{\Delta }\lambda =20\mathrm{nm}$, 10 W) light sources (model: ABUS TV 6818, ABUS Security-Centre GmbH, German) were mounted on either side of the camera to provide a uniform illumination of the target area: the palm of the left hand. Images were captured for 4 min at a rate of 200 fps and an exposure time of 4 ms, producing a raw image size of $256\times 384\text{pixels}$. The pixels were encoded in 10-bit gray scale, which allowed the camera to detect the weak pulsations of the microvascular tissue bed. During the image recording, a reference contact PPG signal was obtained simultaneously from a reference LED-PPG circuit, as shown in Fig. 1(b). Specifically, the reference PPG signal was initially obtained by means of a standard commercial contact PPG sensor (model: SA30014C, Shanghai Berry Electronic Tech. Co. Ltd., China) placed on the middle finger of the left hand and a DISCO4 data acquisition system (Dialog Devices, UK). This plethysmographic signal was then transferred to a custom-written LabVIEW (National Instruments Co., USA) software platform and served as the power intensity signal to trigger a red LED ($\lambda =650\mathrm{nm}$, $\mathrm{\Delta }\lambda =15\mathrm{nm}$, RS Components, UK) through a DAQ (model: USB-6008, National Instruments Co., USA). The red LED probe was attached to the target area with double-sided adhesive tape during image acquisition, providing a clear and reliable reference signal for easy comparison with iPPG signals during postprocessing.

Fig. 1

(a) A schematic showing the experimental setup of the noncontact iPPG system, with the dashed box indicating the reference LED-PPG system, and (b) a picture of the reference LED-PPG system.

2.3.

Experimental Protocol

As mentioned above, the purpose of this study was to assess the practicability and the variability of the iPPG in assessing PRV and to investigate and quantify the influence of different sample rates on PRV analysis. To accomplish these objectives, 10 sets of recordings were taken from 10 healthy subjects enrolled during four successive days. The image sequences were taken with the subject at rest to minimize physiological variations and motion. A soft cushion was placed under the target hand to further minimize the motion. The palm of each subject was exposed to the infrared illumination source, and the distance between the camera lens and the skin was about 400 mm. All measurements were taken in a temperature-controlled (20°C to 22°C) darkroom by a trained operator. The activity of the ANS governing the PRV clearly reveals circadian rhythms.²² Hence, the recordings for this study were obtained within the period of 9:30 to 17:30 ($\text{median}=12:30$) for consistency.

2.4.

Image Processing

Motion artifacts were minimized by recording images under resting conditions. Hence, a simple yet efficient spatial averaging approach¹⁵ was adopted and was found to be adequate to attenuate relatively small motion artifacts.^11,14 Once a set of recordings was acquired, the raw images were divided into discrete subwindows to produce a new set of reduced frames, as shown in Fig. 2(b). The value of each pixel in the reduced frame was set as the average of all the pixel values within each subwindow. Though it compromises the spatial resolution, such a procedure has been shown to significantly improve the signal-to-noise ratio.¹⁵ Subwindow size was set at $8\times 8\text{pixels}$, resulting in a reduced frame size of $36\times 48\text{pixels}$. PPG signals were then derived from each pixel position across a sequence of frames. A fifth-order Butterworth filter with cutoff frequencies set at [0.05, 4] Hz was employed to further attenuate noise.

Fig. 2

A representative figure showing (a) an original image (frame 11999, $t=60\mathrm{s}$) and (b) the reduced image, each pixel of which was the average of an $8\times 8\text{pixel}$ subwindow from the original; (c) contact PPG signal with TFR results; and (d) iPPG signals with TFR plot. The inserts in (c) and (d), in which the time scale has been expanded, show the plethysmographic waveform in which RR and HR can be clearly seen. The upper TFR trace is from the cPPG, and the lower one is from iPPG, with a color bar indicating the power intensity. The position at which the iPPG signal was obtained is shown by the black box ($1\times 1\text{pixel}$) and an arrow in (b), while the region where the cPPG reference signal was obtained is indicated with a larger black box ($3\times 3\text{pixel}$). The signal is from Subject #1 (male, $\text{age}=28\text{years}$).

2.5.

Heart/Respiration Rate Extraction from iPPG

In our previous studies, a time-frequency analysis [time-frequency representation (TFR)] was used to assess the time-varying heart rate (HR).¹² The HR was first obtained through averaging the fundamental HR frequency from the TFR trace calculated from the reference contact PPG signal, and this HR was then treated as a reference. Successive calculations were then performed on the reduced frames, where if the difference between the current HR estimation and the reference value exceeded the threshold of 9 bpm, the algorithm isolated these regions as corrupt and rejected the invalid HR. The ${\mathrm{HR}}_{\mathrm{iPPG}}$ could then be obtained by averaging HRs within all the valid subwindows. For each set of recordings, a 1-min moving window (12,000 frames) was employed for this HR calculation with 0.5-min overlap (6,000 frames). This yielded 7 HR readings for each subject. A similar approach was employed to obtain the respiration rate (${\mathrm{RR}}_{\mathrm{iPPG}}$).

2.6.

Trough Detection

To analyze the PRV, the first step is to detect the pulse-pulse intervals. A revised technique based on wavelet transforms was used for detecting trough positions, where the PPG signal was flat and contained little disturbance, thus facilitating PRV analysis. After bandpass filtering, the wavelet transform was performed on the obtained PPG signals, where each peak and trough pair in the PPG signals corresponds to a positive maximum and negative minimum pair with two zero-crossing points indicating the peak (maximum value) and trough (minimum value) positions. The locations of these zero-crossing points ($n$), where the minimum values were revealed in the original PPG signals, served as the raw trough positions. A semi-automated procedure was then employed to further eliminate the false positive and negative trough detections and to improve the accuracy. A manual validation was initially conducted to verify $n_1$ as the first trough position. HR was then calculated within a moving window $W_{t1,t2}$ ($t2-t1=10\mathrm{s}$), and then 0.9 to 1.1 of ${\mathrm{HR}}^{-1}$ was adopted as the PPI threshold. If $0.9{\mathrm{HR}}^{-1}<|n_{i1}-n_{i2}|<1.1{\mathrm{HR}}^{-1}$, where $t1\le n_{i1}$, $n_{i2}\le t2$, the algorithm considers $n_{i2}$ a valid trough position. If no valid trough position was found, a search back approach was performed to find the minimum value within the threshold range, and a warning index was saved for postvalidation.

2.7.

Pulse Rate Variability Analysis

A series of straightforward and efficient time domain and frequency domain measurements was employed to extract the PRV information. The mean values of pulse-pulse intervals (MPP) and standard deviation of the pulse-pulse intervals (SDPP) were employed in the time domain analysis. As a discrete event series, PPI is an unevenly sampled time series, hence an interpolation and resample approach was conducted before the frequency domain analysis. In practice, it is recommended that the duration of the recording be about 10 times the wavelength of the lowest-frequency band of the spectral component being investigated.¹ Therefore, only the low frequency (LF: 0.04 to 0.15 Hz), the high frequency (HF: 0.15 to 0.40 Hz), and the LF/HF ratios were calculated in this work. The HF component, which has a peak at respiratory frequency, corresponds to respiratory sinus arrhythmia (RSA) and reflects parasympathetic influence on the heart through efferent vagal activity. The LF component, consisting of fluctuations below 0.15 Hz and usually centered at about 0.1 Hz, is mediated by both cardiac vagal and sympathetic nerves.^23,24 Hence, the ratio of the LF and HF represents the sympatho-vagal interaction. LF and HF were also expressed in normalized units to account for inter-individual differences, denoted as ${\mathrm{LF}}_n=\mathrm{LF}/[\text{Total power}-\mathrm{VLF}(\text{very low frequency},0.003-0.04\mathrm{Hz})]$ and ${\mathrm{HF}}_n=\mathrm{HF}/(\text{Total power}-\mathrm{VLF})$, respectively.¹ Postprocessing and analysis were performed with custom software in Matlab 2008a (MathWorks, Natick, Massachusetts, USA).

2.8.

Influence of Frame Rate on PRV Analysis

A down-sampling approach has been adopted (Fig. 3) to investigate the influence of sample rates on the PRV analysis. Once a set of reduced frames was acquired, three new frame sequences with sample frequencies of 100, 50, and 20 fps were obtained by down-sampling the original reduced frames. These new frame sequences used the iPPG recordings with the same image configuration yet different sample frequencies, which could therefore be treated independently. The PPG signal extraction and preprocessing procedures were repeated. To refine the trough fiducial point, especially for the low-sample-rate iPPG recordings, the PPG signals were interpolated with a spline function¹³ to reach an effective frequency of 200 Hz. These interpolated PPG signals were then ready for the trough detection and PRV analysis as described above.

Fig. 3

A representative figure showing the process and effects of investigating the influence of the frame rate on the PRV analysis. The PPG signal shown is from Subject #1 (male, $\text{age}=28\text{years}$).

2.9.

Statistical Analysis

To evaluate the reliability of the iPPG system for remote physiological assessments, Bland-Altman analysis²⁵ was performed to compare iPPG and contact PPG (cPPG). The difference between iPPG and cPPG was plotted against their average, as were the mean and the standard deviation (S.D.) of the differences and 95% limits of agreement ($\pm 1.96\mathrm{S.D.}$). The Pearson’s correlation coefficients and the corresponding $p$-value were also calculated to assess the physiological variables (HR, RR, and PRV) from both systems. Differences between the PRV measurements at different sample rates were assessed with one-way analysis of variance (ANOVA, $\text{factor}=\text{sample rate}$) to demonstrate the influence of sample frequency on the PRV analysis. The group differences were assessed by post hoc analysis using Fisher’s least significant difference (LSD), where groups represent the different sample frequency. All analyses were performed with $\alpha$ (Type I error) set at 0.05 using the statistical software program SPSS for Windows, version 17.0 (IBM, Armonk, New York).

3.1.

Heart/Respiration Rate Measurement

Figure 2 shows an example of the PPG signals and their corresponding TFR traces. The reference cPPG signal was obtained by calculating the mean intensity value within the region of the reference LED. The variations in the signal due to heartbeat and respiration are clearly visible for both PPG signals in the inserts of Fig. 2(c) and 2(d). It can be seen in the TFR traces that the oscillations of RR, HR, and the first harmonic component derived from the iPPG signals were in agreement with those obtained from the commercial pulse oximeter sensor readings. Bland-Altman analysis was used to evaluate the agreement of HR and RR between the imaging method and the contact readings. The results of the Bland Altman analysis are shown in Fig. 4. A close agreement between these two techniques was revealed. Specifically, in no case did the mean difference of the physiological measurements (HR/RR) between the two methods differ significantly from zero. The mean bias of RR is $0.03\text{breaths}/\min$ with 95% limits of agreement $-1.29$ to $+1.35\text{breaths}/\min$; while the mean bias for the heart rate is $-0.04\mathrm{bpm}$ with 95% confidence interval $-3.00$ to $+2.93\mathrm{bpm}$. Significant correlations between both techniques were also revealed (Pearson’s correlation for RR and HR, $r^2>0.95$, $p<0.001$).

Fig. 4

Bland-Altman plots showing the average of (a) the respiration rate and (b) the heart rate measured by the imaging PPG and contact PPG, plotted against the difference between them.

3.2.

Pulse Rate Variability Measurement

A typical example of the imaging PPG signals is presented in Fig. 5(a) and 5(b) along with the contact PPG signals recorded with the reference LED-PPG system. The boxes in Fig. 5(a) and 5(b) show data at a magnified time scale (60 to 80 s), where clear plethysmographic waveforms with detected trough positions are shown in both PPG signals [cPPG (black), iPPG (green)]. The obtained pulse-pulse intervals are shown in the right panel of Fig. 5(a) and 5(b). It is evident that the two PPI sequences are in close agreement. The resulting power spectral density maps are shown in Fig. 5(c), where both spectra exhibit comparable functional characteristics and a dominant LF component. Table 1 summarizes the time and frequency domain results of the PRV analysis [$\text{values are mean}\pm \text{standard error of the mean}$ (SEM)]. Significant correlations between cPPG and iPPG were found in all estimated results (Pearson’s correlation, $r>0.85$, $p<0.001$).

Fig. 5

A representative figure showing (a) contact and (b) imaging PPG signals with detected troughs (red circles) and the corresponding pulse-pulse intervals and (c) the power spectral density map of the PPIs. This demonstration signal is from Subject #5 (male, $\text{age}=30\text{years}$).

Table 1

Overall results of the PRV analysis.

To further evaluate the performance of the imaging PPG in assessing PRV information, Bland-Altman analysis was employed to assess the agreement between contact and imaging readings of the estimated PRV results. Figure 6 shows that the mean biases for MPP and SDPP are $-1.00$ and $-0.49\mathrm{ms}$, respectively. The corresponding 95% limits of agreement are from $-9.94$ to 7.94 ms and $-11.54$ to 10.56 ms, which shows that they do not differ significantly from zero. Similar statistically comparable results are revealed in the frequency analysis. The mean biases between cPPG and iPPG for the measurement of ${\mathrm{LF}}_n$, ${\mathrm{HF}}_n$, and LF/HF are $-0.008$, 0.005, and 0.046. The corresponding 95% confidence intervals are $-0.093$ to 0.078, $-0.076$ to 0.086, and $-0.381$ to 0.473. Again, this shows that none of these biases differs significantly from zero.

Fig. 6

Bland-Altman plots of the comparison between contact and imaging readings of PRV. (a) Mean value and (b) standard deviation of pulse-pulse intervals, (c) low and (d) high frequency components, and (e) ratio of low and high frequency components.

3.3.

Effects of Frame Rate on PRV Analysis

Three additional down-sampled iPPG sequences were obtained as described previously (i.e., 100, 50, and 20 fps). Table 2 summarizes the PRV results for different sample frequencies. Further statistical analysis showed no significant influence of the sample rate on any of the PRV measurements: MPP ($F=0.03$, $p=0.998$), SDPP ($F=0.07$, $p=0.990$), ${\mathrm{LF}}_n$ ($F=0.03$, $p=0.998$), ${\mathrm{HF}}_n$ ($F=0.03$, $p=0.997$), or LF/HF ($F=0.01$, $p=0.999$). Treating the PRV measurements calculated from cPPG readings as the reference, significant correlations between cPPG and the three down-sampled iPPG sequences were also uncovered (Pearson’s correlation, $r>0.81$, $p<0.001$). We note that MPP increases monotonically with increasing sample frequency (${\mathrm{MPP}}_{\_20\mathrm{fps}}<{\mathrm{MPP}}_{\_50\mathrm{fps}}<{\mathrm{MPP}}_{\_100\mathrm{fps}}<{\mathrm{MPP}}_{\_200\mathrm{fps}}$), whereas no such consistent trend is seen with the other related variables. The variations of PRV measurements in the down-sampled sequences indicate a negligible loss of data for reductions in sample frequency down to 20 fps.

Table 2

Effects of sample rate on PRV measurements (values are mean±SEM).