Assessment of pulmonary functions is important for respiratory disease diagnosis and therapy evaluation. The methods currently used in clinical practice for pulmonary function tests, such as the spirometer and pneumotachography,1 enable evaluation and direct measurement of breathing capacity in real time. However, because these techniques measure exhaled and inhaled air volume or airflow speeds, they lack the ability to assess regional chest wall motion, which is essential for kinetic analysis of respiratory physiology2 and for diagnostic evaluation of the pulmonary diseases with abnormalities in chest wall movement, such as pulmonary edema and thoracoabdominal asynchrony.3 Additionally, the necessary use of the nose nip, mouth connectors, or masks that these techniques employ commonly interferes with subjects’ normal respiration, and is very difficult for the patients with breathing difficulties.
Konno and Mead4 established that respiratory function could be evaluated by measuring volume changes in the whole chest wall, including the rib cage and abdomen, and many techniques of chest wall measurement have been subsequently developed for respiratory volume estimation, which can provide noninterfering assessments of respiration. Some simple respiratory sensors, such as inductive plethysmography, strain gauges,3 magnetometers,5 long-period gating sensors,6 and fiber optic sensors,7 used to measure area, perimeter, diameter, or curvature variation of one or a few particular cross sections of the chest wall, have been applied in respiratory monitoring and analysis of chest wall motion. These techniques, however, mainly measure parameters in specific positions or sections, rather than whole surface motion. This results in technical difficulties when estimating respiratory volume, such as the need for complicated physiological models and additional calibration procedures with other instruments for each subject.
Unlike the preceding techniques, optical methods have the advantages of noncontact and whole-field measurement, and provide opportunities to accurately assess respiratory volume by measuring morphological changes in the chest wall. Saumarez8 used an automatic light-sectioning system to continuously measure movement of the torso surface during breathing. This method, however, was not time-efficient and was not feasible for routine use. Recently, Ferrigno 9 presented a method using an optical reflectance motion-analysis system to estimate volume changes in the chest wall by measuring 3-D coordinates of multimarkers placed on the rib cage and abdomen. The data acquisition speed of the technique was very high and the position determination was direct with high accuracy. The method could also assess compartmental breathing10 to provide capability for respiratory kinetic analysis.2, 11 However, the spatial resolution was dependent on the number of markers placed on the thoracoabdominal wall, whose distribution was sparse, and the marker attachment procedure is inconvenient.
Recently, progress in using structured light to measure surface topography12, 13, 14 has provided opportunities for 3-D surface shape acquisition with high spatial and temporal resolutions, which enable us to explore new ways to measure chest wall movement in respiration. Unlike the optical reflection system, which uses sparsely distributed markers, structured light can offer direct, full-field information about the chest wall surface with high spatial resolution. Aoki 15 used a near-IR multiple slit-light projection to monitor the respiration of subjects at supine posture. The volume change estimation in that study, however, may include bias due to variable projection coverage on the chest wall for different respiratory states.
In this paper, we present a newly developed technique to evaluate respiratory volumes by combining color structured light for 3-D shape measurement with the use of a small number of active light markers for chest wall region determination. The device is simple and convenient, using only a projector to illuminate the structured light over the chest wall with color stripes and a video camera to record the height-modulated images. The positions of the active markers recorded by the same camera define the measurement boundaries of the moving chest wall to offer a consistent region for volume evaluation in respiratory processes. With this system, the measurement can be performed dynamically from a single side of the human body when positioned against a wall, from two sides when the subject is in standing posture, and from one side with the subject in supine posture. The respiratory volume estimation using this method was validated by comparing the measured results with those from pneumotachography performed at the same time, during deep and spontaneous breathing by the subjects. Isovolume tests10 were also carried out to show the measurement variance of the optical system. Moreover, the proposed method can not only assess whole respiratory volume with high accuracy, but also estimates regional pulmonary function by measuring regional motions of the chest wall. After validation by isovolume tests, an experiment was conducted to demonstrate function estimations for thoraxes of a patient before and after clinical treatment.
To acquire a 3-D shape of the whole chest wall in real time, a color pattern consisting of parallel straight stripes encoded with the De Bruijn sequence16 was used to illuminate the object. Such a sequence of order over an alphabet of symbols is a circular string of length , providing a window property that the subsequence with length appears only once, which is critical for identifying each stripe in the pattern. The sequence is obtained by searching a Hamiltonian circuit over a De Bruijn graph whose vertices are the words of length16 . To enhance the robustness of the recognition, an additional rule that the adjacent symbols must be different is also employed in the De Bruijn sequence design.13, 14 With that rule, a sequence of length is generated from an altered De Bruijn graph produced by deleting all the vertices with same adjacent symbols from the traditional De Bruijn graph. Figure 1 shows a pattern generated by five colors ( , red, yellow, green, cyan, magenta) with a three-order De Bruijn sequence, where each stripe can be identified with a unique code in the whole sequence, composed by the colors of its two adjacent stripes and itself.
The surface measurement of the chest wall is based on triangulation of the structured light system, which consists of a projector for pattern illumination and a video camera for image recording. Figure 2 shows a typical arrangement of the system for surface height measurement. The pattern is projected onto the chest wall by the projector at from an inclined direction, and the video camera (Sony Digital HandyCam, TRV60E) placed at captures the pattern from the normal direction to the reference plane. The optical axis of the camera is perpendicular to the reference plane, and the connection line between the optical centers of the camera and projector is parallel to the reference plane. In this geometry, the pattern projected at point on the reference plane is intersected by the chest wall with height , causing a projection movement to the point on the chest wall surface, which results in a corresponding shift on the image plane of the camera. Because the triangle is similar to , the height can be obtained by measuring the lateral shift ( is the ratio of the length of real object and its image) on the recorded images, with triangulation of14and are the distances of the camera from the reference plane and from the projector, respectively, and can be obtained by the system calibration.
The planes of several boxes with different sizes are used as varied height references to determine the system parameters in calibration. The lateral shift corresponding to each known height is acquired from the images of the box planes, which are composed of a set of (seven pairs in our experiment) for the system. To calculate the surface height from image shift using Eq. 1, three parameters should be obtained in the system calibration: , , and . To simplify the calibration procedure, a linear equation is derived from Eq. 1: with two unknown parameters ( and ), which can be calculated by the linear least-squares method. Figure 3 shows an example of the calibration result as the nonlinear relationship between and and .
With the obtained 3-D surface topology, the volume between the reference plane and the surface can be calculated by volume integration (to be described in Sec. 2.6). However, there is a technical obstacle to volume change evaluation. For different respiratory states, the chest wall surface changes are mainly in the normal direction (forward and backward movement of the chest wall). In the proposed inclined illuminating system, the coverage of the projected pattern on the chest wall is variable due to these surface changes, which cause the integrating region to be anatomically inconsistent for different respiratory states. As a consequence, bias would be included in the volume change evaluation. Additionally, the necessity to measure all chest wall movements, including the motion in the direction parallel to the reference plane, also requires an anatomically consistent measurement region.
To overcome these obstacles, a small number of active markers (white LEDs) are attached to the chest wall to correct the volume change assessments from the same anatomic region. These markers form a closed boundary by connecting them with straight lines, which defines a measurement region to cover the major part of the moving chest wall. These markers can be conveniently traced in the images captured by the camera during respiration, as shown in Fig. 4 .
For every image of the captured pattern, three steps are carried out in image processing to identify both the locations and the unique codes of the color lines so as to obtain the surface height map.
Segmenting Active Markers and Enclosing Measuring Region
To define the boundary of the measured region, segmentation of the active markers is initially performed by thresholding the value channel of the captured image. The center location of each marker can be detected by a subpixel centroid algorithm based on segmentation.14 The measurement boundary is then determined by automatically connecting the markers with straight edges. Following this, the regions outside the boundary are excluded to save processing time. The bright spots of the white makers, which may interfere with the detection of the color stripes, are also excluded. As an example, Fig. 4 presents the segmented region from the original image, and Fig. 4 gives the cropped image of the color pattern.
Segmenting Color Pattern and Localizing Stripes
The process is initially performed by thresholding the second derivative of the value channel of a cropped image14 to segment the color stripes. The center locations of those stripes are then detected in the segmented region with subpixel accuracy, using a normalized centroid algorithm14 given byis the intensity function of the segmented region, is the coordinate function, is the pixel index, is the threshold (set to 0.9 in our case), and is the normalized intensity function. The result of the center locations is shown in Fig. 4.
Recognizing Unique Codes of Color Stripes and Calculating Surface Heights
The unique codes of each stripe are decoded by identifying its color and the colors of the adjacent stripes in the pattern through thresholding the hue value of the segmented regions, as shown in Fig. 4. To enhance the robustness of the decoding procedure, a mean value is calculated for every color stripe by averaging the hue values of the pixels around the center line. Errors in color recognition, however, may still exist due to noise and color aberration in the images. To provide more reliable matching, the crystal growth algorithm12 is used to extract a unique code for each color fringe. The crystal seed is found when three adjacent color stripes match unique codes with the designed sequence; the seed then grows in two opposite directions until an unmatched unique code is reached. With the unique codes of those stripes, whose spatial distributions are distorted by the object surface, the image shift between the deformed pattern and the reference pattern is determined by matching the unique codes of color stripes. Thus, the surface height is calculated by Eq. 1. The depth map of the captured image is shown in Fig. 4 in gray levels.
Chest Wall Volume Calculation
After the height map of the chest wall surface is obtained, a topographic mesh composed of small triangles is constructed. An integration process to obtain the whole chest wall volume is implemented by calculating the sum of the small columns with the triangles on the topographic mesh and their projection on the reference plane as the top and bottom surfaces, given by , where is the volume of the ’th column unit calculated by , whereas is the angle between the triangle and the reference plane, height is the distance between the centroid of the triangle and its projection, and is the area of the triangle (Fig. 5 ).
Notably, the chest wall volume calculation is done inside the measurement region confined by the active markers attached to anatomic locations, as described in Sec. 2.4. As a result, the measurement includes not only the chest wall movements in the direction normal to the reference plane, but also the motion components in the direction parallel to the reference plane, providing a full-scale evaluation of the chest wall volume change.
Validation Experiments and Results
Three Types of Measurement Setups
Based on this structured light 3-D measurement technique, we set up three types of illuminating and recording devices for different postures or positions to evaluate respiratory volumes. As shown in Fig. 6 , the single-sided setup is for the standing posture of a subject positioned against a wall and asked to not move their back from the wall during the experiment [Fig. 6]. The inclined single-sided setup for the supine posture, on the other hand, is used for subjects who are lying down on a bed with adjustable obliquity . In this position, the subject’s back should naturally be kept in contact with the bed surface [Fig. 6]. For subjects having no contact with a reference plane, the double-sided illuminating-recording setup [Fig. 6] is suitable, using a standing posture with two projector-camera pairs to cover the front and the back of the trunk. In this case, the rigid movement of the human trunk can be compensated by depth maps measured from both sides. For each projector-camera pair in these devices, the projector is placed below the camera at a distance of to illuminate the subject from the bottom, and the camera is positioned in front of the subject’s trunk.
For single-sided measurements, 12 active markers are attached to skin on the anterior side of the subject’s trunk using adhesive tape, after system calibration, as shown by the cross stars in Fig. 7 . For the double-sided setup, another 12 markers are attached on the posterior side of the trunk [Fig. 7]. The subject puts on a white close-fitting T-shirt to cover the body before testing begins.
After the study procedure and consent form was reviewed and approved by the institutional ethics board, and informed consents were obtained from all subjects in this study, 25 healthy volunteers (s1 to s10, female; s11 to s25, male) were included in the validation tests for three types of setups. The clinical characteristics of the testing population, including age, height, weight, and body mass index (BMI), are summarized in Table 1 .
Clinical characteristics of the population in the validation experiment.
To validate the established optical system for respiration measurement, two kinds of tests were performed. First, pneumotachography was utilized in conjunction with the optical measurement to validate results. During the tests, the subject breathed through a mouthpiece connected to a calibrated pneumotachography (Jaeger, Germany) to record air flow volume. Meanwhile, chest wall motion was measured by the optical system. Every subject performed several rounds of spontaneous and deep respirations; each respiration lasted approximately . Then the volume results (differences between peaks and valleys of the volume curve) of the two methods were compared with linear correlation and a Bland-Altman plot. For the single-sided setup for standing posture, which should be the most convenient posture for subjects and the simplest setup for measurement, 23 volunteers were included to validate the system. For the other two setups, only two subjects were tested to show the respiration demonstrations.
Second, an isovolume test10 was used to validate regional measurement of the optical system. Subjects performed an isovolume maneuver by holding their breath without any air flow through mouth and nose, while trying to move the thorax and abdomen for structured light measurement of the wall surface. The curve of total chest wall volume change should be flat during the isovolume test and the standard deviation of the measured volume curve was used to evaluate the accuracy of the optical measurement.
Single-Sided Setup for Standing Posture
Respiration volumes extracted from the volume curves of 23 subjects (s1 to s23) showed a high correlation of 0.998 between the results from the chest wall motion measurement using structured light method and pneumotachography measurements , as shown in Fig. 8 , with the Bland-Altman plot shown in Fig. 8. The mean difference between the two methods is 5.60% [mean of ] with a standard deviation of 8.49% [standard deviation of ]. The comparison indicates that the proposed method is valid for respiratory volume estimation, with values higher than those from pneumotachography. As an example, Figs. 9 and 9 present the data from a female of age 24 (s3), showing the volume curves measured by pneumotachography and by the optical system. Those curves coincide very well, with high correlations of 0.991 for spontaneous respiration and 0.997 for deep respiration, respectively. For the isovolume test, Fig. 10 shows the volume curve of a subject (s11) performing isovolume maneuver, showing a standard deviation of in the breath-holding period .
Inclined Single-Sided Setup for Supine Posture
Two healthy subjects (s11 and s24) were recruited for comparison between the inclined single-sided setup and pneumotachography. The respiration volumes calculated from the volume curves showed a high correlation of 0.998 . The measurement difference between the two methods was ( deviation). Also, a healthy subject (s12) who performed the isovolume maneuver, showed a standard deviation of in the breath-holding period.
Double-Sided Setup for Standing Position
The test results for two subjects (s11 and s25) using the double-sided setup show a correlation of 0.999 in comparison with the measurement results from pneumotachography. Measurement difference between the two methods was ( deviation). For another subject (s13) who performed the isovolume test, the standard deviation in the breath-holding period was .
Study of Regional Chest Wall Change
Compared with the techniques of measuring air flow to assess respiratory volume, such as using a spirometer or pneumotachography, the proposed optical measurement has the advantage of not only estimating whole lung volume changes without any contact on the human body or limitation of normal breathing, but also can evaluate regional pulmonary function by measuring the movement of specified regions of the chest wall. In this paper, we present a single-sided illuminating-recording setup for supine posture to evaluate regional lung function in a subject with different functions of the left and right lung.
To validate the method, a measurement was initially performed on a healthy subject (s13) using the isovolume test. Besides the 12 active markers positioned on the front side of chest wall as already shown, an additional 5 markers, as shown in Fig. 7 by the sun stars, were attached on the trunk to split it into the regions of the left thorax, right thorax, left abdomen, and right abdomen, respectively. During the tests, the subject sequentially performed spontaneous breathing, deep breathing, and then breath holding without air flow but exchanging his internal volume between thorax and abdomen. The volume changes in different regions are presented in Fig. 11 . For the volume curve of the whole chest wall [Fig. 11], the last part is almost a straight line (standard deviation: ), showing that the total volume during breath holding remained in the whole trunk, as expected from the isovolume maneuver. On the other hand, the volume curves of the whole thorax and the whole abdomen [Fig. 11] change with opposite phases in this period [correlation coefficient of the volume curves is ], reflecting the volume exchanges between them, which cannot be revealed by air flow measurement. Moreover, the volume changes are compared for the left and right abdomen [Fig. 11] and for the left and right thorax [Fig. 11], respectively. They have almost identical volume curves with nearly symmetrical functions of the healthy subject, with correlation coefficient of the volume curves at 0.991 and 0.982 , respectively.
After the study procedure and consent forms were reviewed and approved by the institutional ethics board, the optical method of color structured light was applied to the analysis of regional pulmonary function for a subject (s26: male; age, 15; height, , weight, , BMI, ) with pleural effusion in the left thorax (detected by X-ray photography). We measured the volume change of different chest wall regions during spontaneous and deep breathing with the optical system. The left thorax respiratory function was evaluated by the ratio of the volume value of the left thorax to that of the whole thorax. The test was initially performed before scheduled drainage to obtain the comparative functions for the left thorax. After drainage, the left thorax was measured again on the same day to evaluate the regional pulmonary function change after treatment. In this test, the contribution of the left thorax to the whole thorax changes from 39.23 [Fig. 12 ] to 41.88% [Fig. 12] for the spontaneous breath; and from 37.60 [Fig. 12] to 42.53% [Fig. 12] for the deep breath, respectively. The results provide the evidence of respiratory function improvement of the diseased side of the lung after therapy, and prove the capability of the optical method to estimate regional pulmonary functions.
Discussion and Conclusion
A new method using projecting structured light and tracking markers was proposed to evaluate pulmonary function by measuring chest wall motion in respiration. The structured light technique provides accurate 3-D topology of the chest wall surface with high resolution, and the traced markers define a consistent region for chest wall volume change estimation. With the advantage of noncontact measurement, the proposed method has no interference with the natural respiration of the subjects and is comparable to the functions of the traditional pneumotachography/spirometer systems for pulmonary function evaluation. Compared to other optical methods for evaluating the perimeter or curvature of the chest wall, this technique has the advantage of whole-field measurement with high spatial resolution, which enables the evaluation of exact respiratory volume during respiration without any individual registration procedure. Of course, the portability of the proposed method is not as good as some optical methods, such as the fiber optic sensor, but in this technique, convenience is sacrificed for whole-field measurement.
To demonstrate the applicability of the proposed method in different situations, three types of measuring setups were established. The single-sided illuminating-recording setup is suitable for most subjects with standing posture. The inclined single-sided setup, on the other hand, is more suitable for the subjects who must be in a supine posture. The double-sided setup can compensate for body movement and offers better accurate respiratory volume assessment than the single-sided setups, showing smaller variation during isovolume test than the single-sided setups.
In this paper, pneumotachography was used as the “gold standard” or “reference standard” to validate the new technique. This is reasonable as the accuracy of pneumotachography has been proven to have less than 3% error in volume assessment.17 All three setups of this method provide measurements highly correlated with those from pneumotachography [correlation coefficient is ], with small positive errors. The Bland-Altman plot also presented good agreement between the two methods. Another trend that can be seen from the Bland-Altman plot is that the absolute difference is relatively large when the respiration volume is large. A potential source of the measurement difference may come from the rigid forward movement of the subjects. For the single-sided setup designed to evaluate volume changes between the chest wall surface and the reference plane, it is not easy for subjects to keep their back close against the reference plane at all times during respiration, especially during deep breathing in the standing posture. In practice, the inaccuracy of the single-sided setup for supine posture and the double-sided setup has much less influence from the rigid movement of subjects. This was proved in the tests of the inclined single-sided setup and the double-sided setup for standing posture, which showed smaller differences than the single-sided setup for standing posture compared with pneumotachography (1.76 and 4.25%, compared with 5.60%). Also, a similar trend can be seen in the isovolume tests.
In the exploration of regional measurement, the isovolume test on the volunteers presents high negative correlation between the volume changes of the abdomen and thorax; whereas the volume change of the whole chest wall is nearly zero. The comparison of left and right regions (abdomen and thorax) shows almost the same volume change curves with high positive correlation. The results indicate that the new method can measure not only whole chest wall volume change, but regional pulmonary volume as well. As an application, the regional measurement of the left and right thorax movement of a patient with pleural effusion shows the proposed technique can really exhibit the difference of regional pulmonary functions. The result presents the improvement of regional pulmonary function after treatment in the diseased side. Although that regional volume change may be not significant due to the limited sample size, the experimental demonstration shows the potential application of the optical system.
In conclusion, this paper proposed a chest wall motion measuring technique for pulmonary function evaluation by combining projecting structured light and tracking markers. In the validation tests of the three established setups, the agreement with pneumotachography and the small variance in the isovolume maneuver proved the accuracy of the proposed technique. More significantly, its ability to evaluate regional chest wall volume was validated by the isovolume test, and the regional respiratory volume evaluation on the patient with pleural effusion shows promise in future clinical applications.
The support of the National Basic Research Program (No. 2007CB935602) and the National Natural Science Foundation of China (NSFC) Grants No. 90607004 and No. 10672005 is appreciated.