Noninvasive, three-dimensional full-field body sensor for surface deformation monitoring of human body in vivo

Zhenning Chen; Xinxing Shao; Xiaoyuan He; Jialin Wu; Xiangyang Xu; Jinlin Zhang

doi:10.1117/1.JBO.22.9.095001

12 September 2017 Noninvasive, three-dimensional full-field body sensor for surface deformation monitoring of human body in vivo

Zhenning Chen, Xinxing Shao, Xiaoyuan He, Jialin Wu, Xiangyang Xu, Jinlin Zhang

Author Affiliations +

Journal of Biomedical Optics, Vol. 22, Issue 9, 095001 (September 2017). https://doi.org/10.1117/1.JBO.22.9.095001

Abstract

Noninvasive, three-dimensional (3-D), full-field surface deformation measurements of the human body are important for biomedical investigations. We proposed a 3-D noninvasive, full-field body sensor based on stereo digital image correlation (stereo-DIC) for surface deformation monitoring of the human body in vivo. First, by applying an improved water-transfer printing (WTP) technique to transfer optimized speckle patterns onto the skin, the body sensor was conveniently and harmlessly fabricated directly onto the human body. Then, stereo-DIC was used to achieve 3-D noncontact and noninvasive surface deformation measurements. The accuracy and efficiency of the proposed body sensor were verified and discussed by considering different complexions. Moreover, the fabrication of speckle patterns on human skin, which has always been considered a challenging problem, was shown to be feasible, effective, and harmless as a result of the improved WTP technique. An application of the proposed stereo-DIC-based body sensor was demonstrated by measuring the pulse wave velocity of human carotid artery.

1. Introduction

Surface deformation measurements of the human body in vivo are important for biomechanical analysis and medical diagnosis. Numerous biomedical investigations are based on surface deformation, such as understanding the human tactile sensory system,¹ testing biomechanical properties (viscoelastic, nonlinear elastic, and anisotropic) of skin,² diagnosing the epidermis and dermis,³ and characterizing skin stretch for cosmetic surgery.⁴ Despite these studies, investigations into surface deformation using a noninvasive, three-dimensional (3-D) full-field technique still remain challenging since both accuracy and efficiency are critical to in vivo measurements, which further increase the difficulty of developing this technique.

To measure surface deformations and determine biomechanical properties of a human body, many optical techniques have been developed, including interferometry-based techniques [laser speckle imaging (LSI),⁵^,⁶ digital moiré interferometry (DMI),⁷^,⁸ electronic speckle pattern interferometry (ESPI),⁹ and digital holographic interferometry (DHI),¹⁰ etc.], noninterferometry-based techniques [3-D motion capture,²^,¹¹ fringe projection (FP),¹² digital image correlation (DIC),¹³^–¹⁵ etc.], and vision-based techniques (stereo-DIC,¹⁶^–²⁰ etc.). Most optical techniques have full-field and noncontact superiorities, as well as some disadvantages. For example, although most interferometry-based techniques (ESPI, DHI, DMI, etc.) are very sensitive to small motions, the correlation of laser speckles and interferometry of fringe patterns degrade with time and large deformation. LSI generates random speckles by utilizing laser to illuminate diffuse objects; however, these speckles are uncontrollable and depend on the surface roughness. The 3-D motion capture method, which is generally used for face recognition and motion tracking, has limited spatial resolution, while FP, which is widely used for profile measurements, is not suitable for deformation measurements. Conventional DIC, which uses a single-camera, two-dimensional (2-D) imaging system to monitor in-plane deformations of a perfectly flat specimen, is usually accompanied by strain errors because of its inability to isolate out-of-plane displacements. Based on the theory of DIC, vision-based stereo-DIC was developed to measure 3-D deformations (profile, in-plane displacements, out-of-plane displacement, in-plane strain, etc.) of flat or curved objects with full-field resolution. Although stereo-DIC is not as sensitive as laser-based techniques, the advantages of its simple setup, low environmental vulnerability, and ability to measure large deformations make stereo-DIC very promising for measuring surface deformations of the human body. Recent applications, e.g., in vitro testing on biomedical components,¹⁶ in vivo testing on the human heart,¹⁷ fingers,¹⁸ pulse frequency,¹⁹ and breast,²⁰ have also demonstrated its practicality and potential. However, there are three limitations of in vivo applications of stereo-DIC in biomedical studies:²¹^–²⁷ troublesome calibration, poor efficiency, and prefabricated speckle patterns. Many studies have worked on improving these limitations. For example, Shao et al.²³ proposed a practical speckle-based calibration to make the extrinsic parameters of the sensor cameras calibration-free, and the use of a single-lens system²⁴^–²⁶ makes the calibration board unnecessary.²⁵ Zhang et al.²⁷ and Shao et al.¹⁹^,²⁸ proposed parallel computing (graphical-processing-unit-based, seed-point-based) and fast stereo-DIC methods to increase computation efficiency. However, few studies have been conducted on the prefabricated speckle patterns used on human body.

Speckle patterns are usually prefabricated on sample surfaces with even and random distributions. These speckle patterns, which function as the deformation “sensing element” for stereo-DIC, have been demonstrated to be essential for obtaining accurate and efficient deformations.²⁹^–³¹ References 29 30.–31 also showed that different distributions of speckle patterns resulted in different theoretic accuracies of stereo-DIC. However, most traditional techniques (e.g., spray paint, airbrush gun, grinding, and markers) are artificial, time consuming, and experience dependent; thus, the precision of such patterns fluctuates, and different speckle patterns are generated. Some recent techniques [e.g., stencil,³²^,³³ toner transfer printing,³⁴ UV photolithographic,³⁵ and water-transfer printing (WTP)]³⁶ were developed to fabricate the same computer-designed speckle patterns, i.e., the same “sensing element,” on different samples. However, among the existing techniques, only WTP is suitable for use on the human body. However, the existing WTP method cannot be applied on different complexions. Therefore, improving the WTP application on the human body needs to be comprehensively investigated to further improve biomechanical analysis and medical diagnosis using stereo-DIC.

In this paper, a noninvasive, 3-D full-field body sensor based on stereo-DIC is proposed and carefully examined for surface deformation monitoring of human body in vivo. First, we describe the improved WTP method and body sensor based on stereo-DIC. Then, we present the superiorities of the improved WTP method by comparing the accuracy and efficiency of applications to varying complexions. Moreover, we validate this body sensor by measuring the pulse wave velocity (PWV) of human carotid artery,³⁷^,³⁸ and further suggest promising applications in the discussion and conclusion.

2. Noninvasive, Three-Dimensional Full-Field Body Sensor

2.1.

Stereo Digital Image Correlation

The stereo-DIC method is a noninvasive, 3-D full-field imaging method that utilizes prefabricated surface speckle patterns as deformation “sensing elements,” and stereo-DIC has been demonstrated to be a powerful approach for quantifying surface deformations through cross-correlating speckle patterns between “reference” and “deformed” images.²¹^,²² Unlike laser speckles, which are a random scattering phenomenon that occurs when coherent/laser light illuminates a diffuse object or rough surface, speckle patterns of stereo-DIC are artificially manufactured in order to improve the resolution of surfaces for deformation tracking. A schematic diagram of the stereo-DIC system is shown in Fig. 1(a). To monitor surface deformations, two cameras were synchronized to a personal computer at a proper stereo angle [ $θ$ , see Fig. 1(b)] between 10 deg and 30 deg to provide good sensitivity to out-of-plane motion while maintaining in-plane accuracy.²¹ To ensure sufficient image contrast, the specimen was illuminated with blue light-emitting diode sources that are insensitive to ambient light.³² Before the experiment, various images of a translated, rotated, and tilted board (e.g., chessboard) were used to the calibrate intrinsic parameters (principle points, focal length, skew coefficient, and distortions of lens) and extrinsic parameters (orientation and position of optical center) of the two cameras based on Zhang’s principle,³⁹ which was used for 3-D reconstruction. The relationship between image coordinates $M_{i}$ ( $u_{i}$ , $v_{i}$ , $i = 0$ , 1) and the unknown 3-D global coordinates $M$ $(x_{w}, y_{w}, z_{w})$ of interested point $M$ [see Fig. 1(b)] can be expressed using the following equation:

Eq. (1)

[\begin{matrix} a_{11}^{i} - u_{i} a_{31}^{i} & a_{12}^{i} - u_{i} a_{32}^{i} & a_{13}^{i} - u_{i} a_{33}^{i} \\ a_{21}^{i} - v_{i} a_{31}^{i} & a_{22}^{i} - v_{i} a_{33}^{i} & a_{23}^{i} - v_{i} a_{33}^{i} \end{matrix}] [\begin{array}{l} x_{w} \\ y_{w} \\ z_{w} \end{array}] = [\begin{array}{l} u_{i} a_{34}^{i} - a_{14}^{i} \\ v_{i} a_{34}^{i} - a_{24}^{i} \end{array}], (i = 0, 1),

where

a_{m m}^{i}

is the projection matrix that involves the intrinsic and extrinsic parameters of the stereo-DIC system obtained from calibration. Therefore, the global coordinates

M

(x_{w}, y_{w}, z_{w})

can be calculated from four equations of Eq. (1) with matched

(u_{0}, v_{0})

and

(u_{1}, v_{1})

. Here,

(u_{0}, v_{0})

and

(u_{1}, v_{1})

are the 2-D image coordinates of camera 0 and camera 1, respectively, which can be matched through the following zero-normalized sum of squared differences correlation criterion

Eq. (2)

C_{ZNSSD} = \sum_{u = - N}^{N} \sum_{v = - N}^{N} {\frac{f (u, v) - f_{m}}{\sqrt{\sum_{u = - N}^{N} \sum_{v = - N}^{N} {[f (u, v) - f_{m}]}^{2}}} - \frac{g (u^{'}, v^{'}) - g_{m}}{\sqrt{\sum_{u = - N}^{N} \sum_{v = - N}^{N} {[g (u^{'}, v^{'}) - g_{m}]}^{2}}}}^{2},

where

N

is the half width of the subset defined in the reference image and searched for in target image,

f (u, v)

and

g (u^{'}, v^{'})

are gray intensities of image points

(u, v)

and

(u^{'}, v^{'})

in the reference and target subsets, respectively, and

f_{m}

and

g_{m}

are the mean gray intensities of reference and target subsets, respectively. DIC assumes that the intensity variations of reference subset and target subsets remain unchanged, i.e., Eq. (2) equals 0 when the two subsets are perfectly matched [see Fig. 1(c)]. To achieve accurate and efficient calculation, nonlinear iterative algorithms [e.g., inverse compositional Gauss–Newton (IC-GN) algorithm²⁸] can be used to obtain optimal 2-D image coordinates

(u_{1}, v_{1})

corresponding to

(u_{0}, v_{0})

for Eq. (1).

Fig. 1

Schematic diagrams of stereo-DIC: (a) setup and basic procedures, (b) stereovision theory, and (c) correlation principles.

During experiments, speckle patterns were imaged at both camera 0 and camera 1. A correlation algorithm [Eq. (2)] was used to match image 0 and image 1 at time $t = t_{0}$ in order to determine the global coordinates $M$ (reference, $t = t_{0}$ ). After deformation (deformed, $t = t_{1}$ ), Eq. (2) was used twice to match image 0 and image 1, as well as reference image 0 and deformed image 0, in order to determine the deformed global coordinates $M$ . Therefore, the displacements (in-plane, out-of-plane) were calculated by subtracting the global coordinates of reference $M$ from the deformed $M$ . Calculations of the full-field displacements, strain, and other variables were all based on the procedures described above.

2.2.

Improved Water Transfer Printing

The correlation algorithm [Eq. (2)], which is based on the gray intensity of the speckle pattern, is crucial to stereo-DIC because the matched image points between each camera and each deformed state are used to estimate the global coordinate $M$ $(x_{w}, y_{w}, z_{w})$ for further deformation calculations. Thus, the accuracy of stereo-DIC, which is the difference between measured and real deformations, as well as the efficiency of stereo-DIC, which is the computational speed of the deformation, are both affected by speckle patterns. Here, we use “quality” to evaluate speckle patterns in which a good quality speckle pattern represents high accuracy and high efficiency of stereo-DIC.

To achieve high-quality speckle patterns, the patterns were predesigned, well controlled by a computer, and fabricated onto sample surfaces. These speckle patterns are herein called digital speckle patterns (DSP), and they have been previously demonstrated to improve the stability and repeatability of stereo-DIC.³²^–³⁶ Characteristics of the various techniques used to fabricate DSP are listed in Table 1. A brief overview of these techniques showed that WTP is the most promising and powerful technique for human body applications. Moreover, the improved WTP technique proposed in this paper was shown to function better for different complexions with the advantages of harmless, pollution-free, user-friendly application.

Table 1

DSP fabrication techniques.

Techniques	Characteristics
Techniques	Harmless and pollution-free	User-friendly	Robust to complexions
Stencil	×	√	×
Toner transfer printing	×	×	√
Screen printing	×	√	×
UV photolithographic	×	×	×
Improved WTP	√	√	√
Speckle pattern kit	√	√	×

In particular, WTP can transfer images to the 3-D surfaces. A simple method reported the use of water-soluble paper as the image carrier to achieve convenient speckle pattern fabrication for DIC.³⁶ The transferred DSP, which is a basic “sensing element” for stereo-DIC similar to a strain gauge for electric measurement, was used as the body sensor for surface deformation monitoring of a human body.

There are three main sections of the transfer paper [see Fig. 2(a)]: prefabricated decal, printed DSP, and protective coating. The protective coating is a transparent cover to protect the printed DSP from dust. The prefabricated decal section is made up of a paper base layer and a water-soluble layer, which is crucial for separating the DSP from the base layer. The printed DSP section has three layers, including a transparent elastic layer, printed DSP layer, and adhesive layer. The transparent elastic layer is used to protect the DSP from cracking. Traditionally, on the printed DPS layer, only the black dots of DSP are printed on a transparent background surface. Therefore, to increase the contrast of black dots, white matte paint should be sprayed on the surface prior to transferring the DSP. Otherwise, the DSP will be transferred onto testing surfaces with different natural textures, i.e., the speckle pattern recorded by the stereo-DIC system will be different from DSP. The common practice of spraying matte white paint is not suitable applications to the human body. However, maintaining the natural skin surface will generate speckle patterns that vary with different complexions. Therefore, an easy, harmless improvement in the printed DSP could be achieved by employing a white background surface, instead of a transparent background, under the black dots on transfer paper. In this way, high-quality speckle patterns (black dots and white background) are possible.

Fig. 2

Comparisons of DSP transferred by the existing and improved WTP methods: (a) compositions of the transfer paper, (b) DSP transferred onto different backgrounds (top) by the improved (middle) and existing (bottom) WTP methods, and (c) corresponding gray histograms of the speckle patterns for the bottom two rows of (b).

The following procedure was used to fabricate DSP on tested surfaces using the improved transfer paper: remove the protective cover to expose the adhesive layer; fix the transfer paper onto skin using the exposed adhesive; and uniformly moisten the base layer to gently separate the prefabricated section from the printed DSP section. Thus, the DSP is retained on skin and ready for surface deformation measurements.

2.3.

High-Accuracy and High-Efficiency Speckle Patterns

Comparisons of DSP transferred by existing WTP methods and the improved WTP method were performed. Examples of 10 speckle patterns transferred from the same optimized DSP [see Fig. 3(a)] onto different backgrounds (with gray values of 191, 126, 52.6, 12.1, and 1.3) are shown in Fig. 2(b), and the corresponding gray histograms of these DSP are given in Fig. 2(c). Note that a gray value of 0 represents black and 255 represents white. Briefly, an overview of the gray distributions of the transferred DSP showed that the percentage of black was more stable in improved WTP and maintained more information of the original DSP. Thus, the improved WTP with white background was shown to be less sensitive to the backgrounds.

Fig. 3

Quality comparisons under different backgrounds: (a) typical DSP, (b) full-field accuracy assessment, and (c) full-field efficiency assessment.

Further comparisons of DSP quality using the existing and improved WTP methods are shown in Fig. 3. Two aspects were considered for the quality assessments: accuracy and efficiency. To verify the accuracy, the standard deviation (STD, $σ$ ) error of one calculated point was used to predict the displacement accuracy measured at that point:²⁹^,⁴⁰

Eq. (3)

σ \approx {\frac{D (η)}{\sum_{u = - N}^{N} \sum_{v = - N}^{N} {[f_{i} {(u, v)}^{'}]}^{2}}}^{1 / 2},

where

D (η)

is the variance of image noise,

f_{i} {(u, v)}^{'}

is the intensity gradient of DSP in the direction of displacement component (

u

or

v

), and

\sum_{v = - N}^{N} \sum_{u = - N}^{N} {[f_{i} {(u, v)}^{'}]}^{2}

is the sum of square of subset intensity gradients.⁴⁰ Here, zero-mean Gaussian white noise with constant standard deviation 8 was applied on Fig. 2(b) to obtain noisy image, using MATLAB function imnoise. The accuracy prediction was performed on every calculated point, and the full-field results are shown in Fig. 3(b). To verify the efficiency, iteration times of the principle IC-GN algorithm, calculated between the noisy image and original image using Eq. (2), were used to evaluate the computational speed. Here, the convergence condition

‖ Δ p ‖ \leq 0.001

and bicubic B-spline interpolation were selected.¹⁹^,²⁸ Full-field assessments of efficiency are shown in Fig. 3(c). In these comparisons of DSP quality, we found that improved WTP was superior to the existing WTP with both lower STD and less iteration, i.e., DSP transferred by the improved WTP method achieved higher accuracy and higher efficiency in stereo-DIC application, regardless of the backgrounds.

2.4.

Stereo-Digital Image Correlation-Based Body Sensor

The ultimate objective of using stereo-DIC to measure a human body with a surface deformation DSP “sensing element” is to provide a personalized platform for health monitoring that could assist biomedical diagnosis. The improved portable transfer paper made the application of DSP more feasible and wearable, enabling its use as a body sensor. Body sensors have been widely used in daycare for the elderly, in managing patients with chronic disease, in monitoring well-being for general people, as well as in evaluating performance in sports.⁴¹ The development of body sensor networks (BSNs) has focused on both wearable and implantable sensors, e.g., stretchable sensor epidermal electronics,⁴²^,⁴³ graphene electrochemical sensors,⁴⁴ nano-electrochemical sensors,⁴⁵ and vision-based sensors.⁴⁶^,⁴⁷ Compared to traditional vision-based sensors, the stereo-DIC-based body sensor can noninvasively provide not only motion information but also full-field deformations (3-D displacements of in-plane and out-of-plane, strains, velocities, accelerations, etc.). Moreover, with the improved WTP, this smart “sensing element” improved the accuracy of the results obtained with the body sensor and accelerated the data processing. Therefore, the stereo-DIC-based body sensor is not only promising for in vivo clinical application but will also contribute to BSN development and biomedical measurements.

3. Experiments and Results

To verify the feasibility of the proposed stereo-DIC-based body sensor for deformation measurements of the human body in vivo, PWV³⁷^,³⁸ of a human carotid artery, which is considered to be an important parameter for cardiovascular health monitoring, was measured. This measurement was carried out in an ethical manner.

3.1.

Measurement Setup

Measurement setup includes the following devices: two synchronized cameras (FASTCAM SA3-Photron), two lenses (AF NIKKOR-Nikon F50 mm), illumination, safety goggles, personal computer, and calibration board. Images were recorded with 8-bit depth, 2000-fps frame rate, and $1024 \times 1024 pixel$ resolution. Before the experiments, the DSP used for speckle fabrication was tested on the skin surfaces. The flow chart of the proposed stereo-DIC-based body sensor used to obtain the deformations is illustrated in Fig. 4 and can be simply described in four steps:

Step 1: System setup and calibration.
Step 2: Transfer DSP onto tested surfaces.
Step 3: Image recording.
Step 4: Data analysis and medical diagnosis.

Fig. 4

Flowchart of the proposed stereo-DIC-based body sensor.

Note that step 1 was not required for every measurement because the calibration parameters remained unchanged until the setup was changed; for the same field of view, this setup was integrated and fixed. Thus, the major task in setting up the stereo-DIC-based body sensor was to transfer the DSP “sensing element” for image recording and data analysis. It is worth mentioning that only ordinary water was needed for transferring DSP, which is flexible and nonpolluting.

For validation, three young, healthy volunteers participated in our experiments. Before data collections and publications, informed consent was received from each of the volunteers. More than 2 s of 2000-fps recordings were obtained from each volunteer. The accuracy and efficiency of recorded images were analyzed, and the final PWV was obtained after data processing.

3.2.

Data Analysis

The following parameters for the stereo-DIC calculations were used: subset size of $21 \times 21 pixels$ , step size of 7 pixels, and strain window of $15 \times 15$ points. Further analysis of the experimental data was performed from the aspects of speckle quality assessment, rigid body motion removal, and PWV calculations.

3.2.1.

Quality assessment of digital speckle patterns

To express the quality of the transferred DSP, the recorded images were compared with the surrounding natural textures, as well as the typical speckles that marked a human hand.

Comparisons of the DSP with the surrounding textures are given in Figs. 5(a) and 5(b) to assess the accuracy and efficiency, respectively. The results showed that the DSP was consistent with a small STD and required less iteration. In particular, the mean STD of DSP was about 0.031 pixels, which is 15 times smaller than that of the natural textures, where STD was up to 0.450. Iteration times of DSP were $2.009 \pm 0.096$ , which was nearly 10 times faster than that for the natural textures.

Fig. 5

Quality assessment of DSP in human carotid: (a) accuracy assessment, where the mean STD is 0.031 pixels in the DSP area; and (b) efficiency assessment, where the mean value and the standard deviation of iteration times is 2.009 and 0.096 in the DSP area, respectively.

Comparisons were also performed between DSP and typical speckle pattern marked with dots, which was downloaded from a professional DIC company.⁴⁸ These two speckle patterns were pre-assigned with rigid displacements from 0 to 1 pixel using an increment of 0.05 pixels, using the method presented in Ref. 31. Statistics of the differences between the calculated and preassigned displacements, which should theoretically be 0, are given in Fig. 6. Note that, although accuracies and iteration times are varied with the preassigned displacements and bias from the theoretical value of 0, the priority of DSP was obvious in every preassigned displacement compared to the marked speckles because the values of the mean bias error, STD error, and iteration times of DSP transferred by the improved WTP were lower than those of the marked speckles. Thus, DSP was shown to be advantageous over traditional artificial speckles in performing stereo-DIC surface deformation measurements.

Fig. 6

Comparisons of speckle patterns transferred by improved WTP with speckle patterns marked with dots: (a) mean bias error, (b) standard deviation error, and (c) iteration times.

Through the quality assessment, DSP transferred by the improved WTP method was shown to have obvious priority in both accuracy and efficiency for stereo-DIC computations.

3.2.2.

Rigid body motion removal

Rigid body motion is inevitable during testing. Moreover, large rigid body motion may obscure real deformation and leads to failure of the measurement. Figure 7(a) shows an example of the full-field out-of-plane displacement with possible large rigid body motion, obtained after 0.25 s of the image recording. The distribution of the displacement obscured the original deformation of the carotid artery, making the cyclic wave difficult to distinguish.

Fig. 7

Out-of-plane displacement fields: (a) before removal of rigid body motion and (b) after removal of rigid body motion. (Red color indicates displacement of more than 50 microns).

To solve this problem, the rigid body motion must be removed. Here, after determining the equation for carotid artery deformation, the following equation is used to identify rigid body motion:

Eq. (4)

[\begin{matrix} u_{i}^{'} \\ v_{i}^{'} \\ w_{i}^{'} \\ 1 \end{matrix}] = T * [\begin{matrix} u_{0} \\ v_{0} \\ w_{0} \\ 1 \end{matrix}] = [\begin{matrix} R_{11} & R_{12} & R_{13} & t_{x} \\ R_{21} & R_{22} & R_{23} & t_{y} \\ R_{31} & R_{32} & R_{33} & t_{z} \\ 0 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} u_{i} \\ v_{i} \\ w_{i} \\ 1 \end{matrix}], (i = 1, 2, 3 \dots n),

where

{[\begin{matrix} u_{i}^{'} & v_{i}^{'} & w_{i}^{'} & 1 \end{matrix}]}^{T}

,

(i = 1, 2, 3 \dots n)

are displacements without the rigid body displacements;

T

is the global transformation with the rigid body components of translation

⌊ \begin{matrix} t_{x} & t_{y} & t_{z} \end{matrix} ⌋

and rotation

R_{m n}

,

(m, n = 1, 2, 3)

; and

{[\begin{matrix} u_{i} & v_{i} & w_{i} & 1 \end{matrix}]}^{T}

,

(i = 1, 2, 3 \dots n)

are the displacements with rigid body displacements. Displacements of more than three calculated points from the carotid artery were used to calculate

T

, which was used to remove rigid body displacements of the remaining points of interest using Eq. (4). The transformed displacements are given in Fig. 7(b), showing realistic displacement fields for carotid artery deformation.

3.2.3.

Pulse wave velocity calculation

After stereo-DIC calculation, deformations of both the full field and points of interest are extracted. A representative visualized pulse propagation is presented in Fig. 8 to better understand PWV, with a time interval of 0.05 s. From the full-field propagation of out-of-plane displacements, we found that a rapid increase followed by a gentle decrease occurred every cycle. Deformations were concentrated at one position in the carotid artery, near the point P; see Figs. 9(a) and 9(b) for further details. The maximum out-of-plane displacement is marked with point P [see Figs. 7(b) and 8(e)] and exported in Figs. 9(a), 9(c), and 9(e), which showed periodic heartbeats with corresponding cardiac-vascular cycles of 1.22, 1.21, and 1.41 Hz, respectively. Based on these temporal curves, PWV was calculated from the first derivative of the displacements, and is drawn over time with the light gray dotted lines in Figs. 9(b), 9(d), and 9(f). The PWV raw data were very noisy. The PWV data were improved using a low-pass temporal filter fast Fourier transform (FFT) with an appropriate filter window size of 5. Note that smaller filter windows (e.g., three points) could not eliminate noise as well, but larger filter windows (e.g., 100 points) resulted in fewer details. Comparisons of different windows are shown with navy solid lines and pink dashed lines in Figs. 9(b), 9(d), and 9(f). Thus, the given PWV curves and full-field propagations both showed a consistent tendency of carotid artery deformation. Since PWV of the arterial system is a measure for cardiovascular health,³⁷^,³⁸ the filtered optimum PWV curve could be further used for health monitoring.

Fig. 8

Out-of-plane displacement propagation of a carotid artery pulse: (a)-(r) visualized pulse propagation with a time interval of 0.05s.

Fig. 9

Different pulse wave results: (a) and (b), (c) and (d), (e) and (f) are, respectively, the carotid artery displacements and corresponding carotid artery velocities. In (b), (d), and (f), the light gray dotted lines show the original data, the navy solid line shows 5-point FFT smoothed data of the original data, and the pink dashed line shows 100-point FFT smoothed data of the original data.

4. Discussions and Conclusions

As a noninvasive, full-field, and environmental insensitive optical method, DIC has been proven to be a very useful technique in engineering structure monitoring⁴⁹ and mechanical testing.⁴⁸^,⁵⁰ However, consistency of the speckle pattern on varied surfaces has been a challenging problem in stereo-DIC. In this study, a stereo-DIC-based body sensor was proposed for surface deformation monitoring in order to solve such speckle pattern problems, especially for in vivo applications on the human body. By directly transferring the high-quality “sensing element” DSP onto human skin, the improved WTP technique was shown to be feasible and effective, resulting in a highly accurate and efficient body sensor for surface deformation monitoring.

In the experiments, we successfully applied the proposed method to monitor the carotid artery of the human body. Both the PWV and propagations of full-field displacements successfully demonstrated the validity of the proposed stereo-DIC-based body sensor. Furthermore, because of the development of the tiny camera, lens,⁵¹ and compact design²⁵ that stereo-DIC system utilized, as well as the thin ( $\sim 0.4 mm$ ) water-soluble paper,³⁶ the user-friendly operation, and the well-controlled DSP that improved WTP utilized, we believe that the proposed stereo-DIC-based body sensor will be smart and have more applications in biomedical research. In particular, application of the proposed stereo-DIC-based body sensor for strain measurement of skins⁵² will be reported in a forthcoming paper.

Disclosures

This work has no conflicts of interest.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (under Grants 11272089, 11327201, and 11532005).

References

1.

B. Delhaye et al., “Surface strain measurements of fingertip skin under shearing,” J. R. Soc. Interface, 13 (115), 20150874 (2016). http://dx.doi.org/10.1098/rsif.2015.0874 1742-5689 Google Scholar

2.

J. Mahmud, C. A. Holt and S. L. Evans, “An innovative application of a small-scale motion analysis technique to quantify human skin deformation in vivo,” J. Biomech., 43 (5), 1002 –1006 (2010). http://dx.doi.org/10.1016/j.jbiomech.2009.11.009 JBMCB5 0021-9290 Google Scholar

3.

S. P. Kearney et al., “Dynamic viscoelastic models of human skin using optical elastography,” Phys. Med. Biol., 60 (17), 6975 –6990 (2015). http://dx.doi.org/10.1088/0031-9155/60/17/6975 PHMBA7 0031-9155 Google Scholar

4.

H. V. Tran et al., “In vivo characterization of the mechanical properties of human skin derived from MRI and indentation techniques,” Comput. Methods. Biomech. Biomed. Eng., 10 (6), 401 –407 (2007). http://dx.doi.org/10.1080/10255840701550287 Google Scholar

5.

C. Regan et al., “Design and evaluation of a miniature laser speckle imaging device to assess gingival health,” J. Biomed. Opt., 21 (10), 104002 (2016). http://dx.doi.org/10.1117/1.JBO.21.10.104002 JBOPFO 1083-3668 Google Scholar

6.

D. Briers et al., “Laser speckle contrast imaging: theoretical and practical limitations,” J. Biomed. Opt., 18 (6), 066018 (2013). http://dx.doi.org/10.1117/1.JBO.18.6.066018 JBOPFO 1083-3668 Google Scholar

7.

A. Kishen and A. Rafique, “Investigations on the dynamics of water in the macrostructural dentine,” J. Biomed. Opt., 11 (5), 054018 (2006). http://dx.doi.org/10.1117/1.2360257 JBOPFO 1083-3668 Google Scholar

8.

A. Shrestha et al., “Effect of hydration on the strain gradients in dental hard tissues after heat and cold application,” J. Endod., 36 (10), 1643 –1647 (2010). http://dx.doi.org/10.1016/j.joen.2010.06.027 Google Scholar

9.

L. Yang et al., “Measurement of strain distributions in mouse femora with 3D-digital speckle pattern interferometry,” Opt. Lasers Eng., 45 (8), 843 –851 (2007). http://dx.doi.org/10.1016/j.optlaseng.2007.02.004 Google Scholar

10.

B. Kemper and V. B. Gert, “Digital holographic microscopy for live cell applications and technical inspection,” Appl. Opt., 47 (4), A52 –A61 (2008). http://dx.doi.org/10.1364/AO.47.000A52 APOPAI 0003-6935 Google Scholar

11.

J. Gall et al., “Motion capture using joint skeleton tracking and surface estimation,” in Proc. Conf. on Computer Vision and Pattern Recognition (CVPR), 1746 –1753 (2009). Google Scholar

12.

S. Zhang, “Recent progresses on real-time 3D shape measurement using digital fringe projection techniques,” Opt. Lasers Eng., 48 (2), 149 –158 (2010). http://dx.doi.org/10.1016/j.optlaseng.2009.03.008 Google Scholar

13.

D. Zhang et al., “Contact fracture of full-ceramic crowns subjected to occlusal loads,” J. Biomech., 41 (14), 2995 –3001 (2008). http://dx.doi.org/10.1016/j.jbiomech.2008.07.019 JBMCB5 0021-9290 Google Scholar

14.

S. P. Väänänen et al., “Repeatability of digital image correlation for measurement of surface strains in composite long bones,” J. Biomech., 46 (11), 1928 –1932 (2013). http://dx.doi.org/10.1016/j.jbiomech.2013.05.021 JBMCB5 0021-9290 Google Scholar

15.

R. Maiti et al., “In vivo measurement of skin surface strain and sub-surface layer deformation induced by natural tissue stretching,” J. Mech. Behav. Biomed. Mater., 62 556 –569 (2016). http://dx.doi.org/10.1016/j.jmbbm.2016.05.035 Google Scholar

16.

J. Ning et al., “Deformation measurements and material property estimation of mouse carotid artery using a microstructure-based constitutive model,” J. Biomech. Eng., 132 (12), 121010 (2010). http://dx.doi.org/10.1115/1.4002700 JBENDY 0148-0731 Google Scholar

17.

M. Hokka et al., “In-vivo deformation measurements of the human heart by 3D digital image correlation,” J. Biomech., 48 (10), 2217 –2220 (2015). http://dx.doi.org/10.1016/j.jbiomech.2015.03.015 JBMCB5 0021-9290 Google Scholar

18.

Y. Xue et al., “High-accuracy and real-time 3D positioning, tracking system for medical imaging applications based on 3D digital image correlation,” Opt. Lasers Eng., 88 82 –90 (2017). http://dx.doi.org/10.1016/j.optlaseng.2016.07.002 Google Scholar

19.

X. Shao et al., “Real-time 3D digital image correlation method and its application in human pulse monitoring,” Appl. Opt., 55 (4), 696 –704 (2016). http://dx.doi.org/10.1364/AO.55.000696 APOPAI 0003-6935 Google Scholar

20.

H. Khatam et al., “In-vivo quantification of human breast deformation associated with the position change from supine to upright,” Med. Eng. Phys., 37 (1), 13 –22 (2015). http://dx.doi.org/10.1016/j.medengphy.2014.09.016 MEPHEO 1350-4533 Google Scholar

21.

M. A. Sutton, J. J. Orteu and H. Schreier, Image Correlation for Shape, Motion and Deformation Measurements: Basic Concepts, Theory and Applications, Springer Science & Business Media, New York (2009). Google Scholar

22.

J. J. Orteu, “3-D computer vision in experimental mechanics,” Opt. Lasers Eng., 47 (3), 282 –291 (2009). http://dx.doi.org/10.1016/j.optlaseng.2007.11.009 Google Scholar

23.

X. Shao et al., “Calibration of stereo-digital image correlation for deformation measurement of large engineering components,” Meas. Sci. Technol., 27 (12), 125010 (2009). http://dx.doi.org/10.1088/0957-0233/27/12/125010 MSTCEP 0957-0233 Google Scholar

24.

L. Wu, J. Zhu and H. Xie, “A modified virtual point model of the 3D DIC technique using a single camera and a bi-prism,” Meas. Sci. Technol., 25 (11), 115008 (2014). http://dx.doi.org/10.1088/0957-0233/25/11/115008 MSTCEP 0957-0233 Google Scholar

25.

X. Shao et al., “Self-calibration single-lens 3D video extensometer for high-accuracy and real-time strain measurement,” Opt. Express, 24 (26), 30124 –30138 (2016). http://dx.doi.org/10.1364/OE.24.030124 OPEXFF 1094-4087 Google Scholar

26.

K. Genovese et al., “Stereo-digital image correlation (DIC) measurements with a single camera using a biprism,” Opt. Lasers Eng., 51 (3), 278 –285 (2013). http://dx.doi.org/10.1016/j.optlaseng.2012.10.001 Google Scholar

27.

L. Zhang et al., “High accuracy digital image correlation powered by GPU-based parallel computing,” Opt. Lasers Eng., 69 7 –12 (2015). http://dx.doi.org/10.1016/j.optlaseng.2015.01.012 Google Scholar

28.

X. Shao, X. Dai and X. He, “Noise robustness and parallel computation of the inverse compositional Gauss–Newton algorithm in digital image correlation,” Opt. Lasers Eng., 71 9 –19 (2015). http://dx.doi.org/10.1016/j.optlaseng.2015.03.005 Google Scholar

29.

Z. Y. Wang et al., “Statistical analysis of the effect of intensity pattern noise on the displacement measurement precision of digital image correlation using self-correlated images,” Exp. Mech., 47 (5), 701 –707 (2007). http://dx.doi.org/10.1007/s11340-006-9005-9 EXMCAZ 0014-4851 Google Scholar

30.

G. Crammond, S. W. Boyd and J. M. Dulieu-Barton, “Speckle pattern quality assessment for digital image correlation,” Opt. Lasers Eng., 51 (12), 1368 –1378 (2013). http://dx.doi.org/10.1016/j.optlaseng.2013.03.014 Google Scholar

31.

Y. Su et al., “Quality assessment of speckle patterns for DIC by consideration of both systematic errors and random errors,” Opt. Lasers Eng., 86 132 –142 (2016). http://dx.doi.org/10.1016/j.optlaseng.2016.05.019 Google Scholar

32.

B. Pan, L. Yu and D. Wu, “Thermo-mechanical response of superalloy honeycomb sandwich panels subjected to non-steady thermal loading,” Mater Des., 88 528 –536 (2015). http://dx.doi.org/10.1016/j.matdes.2015.09.016 Google Scholar

33.

R. Ghorbani, F. Matta and M. A. Sutton, “Full-field deformation measurement and crack mapping on confined masonry walls using digital image correlation,” Exp. Mech., 55 (1), 227 –243 (2015). http://dx.doi.org/10.1007/s11340-014-9906-y EXMCAZ 0014-4851 Google Scholar

34.

P. Mazzoleni et al., “Thermo-mechanical toner transfer for high-quality digital image correlation speckle patterns,” Opt. Lasers Eng., 75 72 –80 (2015). http://dx.doi.org/10.1016/j.optlaseng.2015.06.009 Google Scholar

35.

W. A. Scrivens et al., “Development of patterns for digital image correlation measurements at reduced length scales,” Exp. Mech., 47 (1), 63 –77 (2007). http://dx.doi.org/10.1007/s11340-006-5869-y EXMCAZ 0014-4851 Google Scholar

36.

Z. Chen et al., “A method to transfer speckle patterns for digital image correlation,” Meas. Sci. Technol., 26 (9), 095201 (2005). http://dx.doi.org/10.1088/0957-0233/26/9/095201 MSTCEP 0957-0233 Google Scholar

37.

S. Laurent et al., “Aortic stiffness is an independent predictor of all-cause and cardiovascular mortality in hypertensive patients,” Hypertension, 37 (5), 1236 –1241 (2001). http://dx.doi.org/10.1161/01.HYP.37.5.1236 Google Scholar

38.

A. Campo et al., “Digital image correlation for full-field time-resolved assessment of arterial stiffness,” J. Biomed. Opt., 19 (1), 016008 (2014). http://dx.doi.org/10.1117/1.JBO.19.1.016008 JBOPFO 1083-3668 Google Scholar

39.

Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans. Pattern Anal. Mach. Intell., 22 (11), 1330 –1334 (2000). http://dx.doi.org/10.1109/34.888718 ITPIDJ 0162-8828 Google Scholar

40.

B. Pan et al., “Study on subset size selection in digital image correlation for speckle patterns,” Opt. Express, 16 (10), 7037 –7048 (2008). http://dx.doi.org/10.1364/OE.16.007037 OPEXFF 1094-4087 Google Scholar

41.

G. Z. Yang and M. Yacoub, Body Sensor Networks, 2nd ed.Springer, London (2006). Google Scholar

42.

J. A. Rogers, T. Someya and Y. Huang, “Materials and mechanics for stretchable electronics,” Science, 327 (5973), 1603 –1607 (2010). http://dx.doi.org/10.1126/science.1182383 SCIEAS 0036-8075 Google Scholar

43.

D. H. Kim et al., “Epidermal electronics,” Science, 333 (6044), 838 –843 (2011). http://dx.doi.org/10.1126/science.1206157 SCIEAS 0036-8075 Google Scholar

44.

Y. Shao et al., “Graphene based electrochemical sensors and biosensors: a review,” Electroanalysis, 22 (10), 1027 –1036 (2010). http://dx.doi.org/10.1002/elan.200900571 ELANEU 1040-0397 Google Scholar

45.

X. Luo et al., “Application of nanoparticles in electrochemical sensors and biosensors,” Electroanalysis, 18 (4), 319 –326 (2006). http://dx.doi.org/10.1002/(ISSN)1521-4109 ELANEU 1040-0397 Google Scholar

46.

K. Kamiyama et al., “Vision-based sensor for real-time measuring of surface traction fields,” IEEE Comput. Graph. Appl., 25 (1), 68 –75 (2005). http://dx.doi.org/10.1109/MCG.2005.27 ICGADZ 0272-1716 Google Scholar

47.

F. Attal et al., “Physical human activity recognition using wearable sensors,” Sensors, 15 (12), 31314 –31338 (2015). http://dx.doi.org/10.3390/s151229858 SNSRES 0746-9462 Google Scholar

48.

( (2017) http://correlatedsolutions.com/software-downloads/ September ). 2017). Google Scholar

49.

D. Feng et al., “A vision-based sensor for noncontact structural displacement measurement,” Sensors, 15 (7), 16557 –16575 (2015). http://dx.doi.org/10.3390/s150716557 SNSRES 0746-9462 Google Scholar

50.

N. Cuando-Espitia, F. Sánchez-Arévalo and J. Hernández-Cordero, “Mechanical assessment of bovine pericardium using Müeller matrix imaging, enhanced backscattering and digital image correlation analysis,” Biomed. Opt. Express, 6 2953 –2960 (2015). http://dx.doi.org/10.1364/BOE.6.002953 BOEICL 2156-7085 Google Scholar

51.

S. Thiele et al., “3D-printed eagle eye: compound microlens system for foveated imaging,” Sci. Adv., 3 (2), e1602655 (2017). http://dx.doi.org/10.1126/sciadv.1602655 STAMCV 1468-6996 Google Scholar

52.

C. Dagdeviren et al., “Conformal piezoelectric systems for clinical and experimental characterization of soft tissue biomechanics,” Nat. Mater., 14 (7), 728 –736 (2015). http://dx.doi.org/10.1038/nmat4289 NMAACR 1476-1122 Google Scholar

Biography

Zhenning Chen is a PhD student at Southeast University of Experimental Mechanics, China. She received her BS degree in mechanics from Lanzhou University in 2011. Her current research interests include stereo-digital image correlation, biomedical engineering, and optomechanical systems.

Xinxing Shao is a PhD student at Southeast University of Experimental Mechanics, China. He received his BS degree in engineering mechanics from Southeast University in 2012. His current works are focus on image-based deformation measurement techniques, including the improvement of accuracy and efficiency of digital image correlation.

Xiaoyuan He received his BS degree from the Department of Applied Mechanics at Nanjing University of Science and Technology, China, in 1982, MS degree from the Department of Mathematics and Mechanics at Southeast University in Nanjing, China, in 1987, and PhD from the Institute of Mechanics Southwest Jiaotong University, Chengdu, China, in 1994. Currently, he is a professor in the Department of Engineering Mechanics at Southeast University. His research interests include 3D shape and deformation measurement by optical method and image processing techniques.

Jialin Wu is currently a MS degree student at Southeast University. His research interests include digital image correlation and deformation measurement of large scale structures.

Xiangyang Xu is currently a MS degree student at Southeast University. His research interests include digital image correlation and speckle pattern optimization.

Jinlin Zhang is currently a MS degree student at Southeast University. His research interests include digital image correlation and close range photogrammetry.

Citation Download Citation

Zhenning Chen, Xinxing Shao, Xiaoyuan He, Jialin Wu, Xiangyang Xu, and Jinlin Zhang "Noninvasive, three-dimensional full-field body sensor for surface deformation monitoring of human body in vivo," Journal of Biomedical Optics 22(9), 095001 (12 September 2017). https://doi.org/10.1117/1.JBO.22.9.095001

Received: 3 May 2017; Accepted: 21 August 2017; Published: 12 September 2017

Access the abstract

JOURNAL ARTICLE
10 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 23 scholarly publications.

Explore citations on Lens.org

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Digital signal processing

Sensors

Speckle pattern

In vivo imaging

Arteries

Digital image correlation

Skin

1.

Introduction

2.