1.

Introduction

Monitoring respiratory mechanics during breathing is important for diagnosing various respiratory diseases and has the potential to be useful in numerous clinical and epidemiological settings.^1–3 One such application is the training instrument for people with chronic obstructive pulmonary disease (COPD).⁴ Real-time visual feedback would be useful during the patient’s breathing exercises. Another application is monitoring the chest wall’s motion during radiotherapy to inform the patient how to control the respiratory-related motion for accurate and effective lung or liver tumor treatment.⁵

However, the measurements with conventional instruments are limited due to the invasiveness and inaccuracy of the existing measuring devices, such as respiratory inductive plethysmographs,⁶ magnetometers,^7,8 stretching belts,⁹ and fiber optics.¹⁰ Devices that measure exhaled and inhaled airflow, such as spirometers or pneumotachographs enable real-time measurements, but lack the ability to evaluate the regional chest wall. Since human breathing is involuntary, all such devices also interfere with the nasal or oral airway and consequently change the pattern of breathing and make tidal volume measurements inaccurate.

On the other hand, optical methods do not disturb the breathing pattern due to the noninvasive principle of measurement. Most known optical methods are limited to measuring the chest-wall displacements at a certain number of predefined points (i.e., markers that are placed on the measured surface). The position of these points needs to be determined prior to signal acquisition, an objective which is often hard to achieve. Optical methods are mainly based on laser triangulation,^3,11 stereophotogrammetry,^12,13 photogrammetry,¹⁴ optoelectronic plethysmography,^15,16 and laser vibrometry.¹⁷

Mizobe et al. presented a laser triangulation method for measuring thorax surface by projecting a dot-matrix light pattern.¹⁸ They also examined the relationship between respiratory flow volume and volume change of thorax surface. Aoki et al. developed a noncontact method using a multiple-line projection pattern and a charge-coupled device (CCD) camera for monitoring the patient’s posture shifts and respiration during sleep.¹⁹ Recently, Jezeršek et al. improved this technique by using color modulation.²⁰ The main advantage of this method is in the possibility of using several measuring modules, an approach which enables one to measure the entire object simultaneously. With color modulation, each projector uses a different laser wavelength, thus avoiding the overlapping of the adjacent light patterns. The multiple-line projection system based on laser triangulation was also used to measure chest-wall deformations and volume changes during breathing in real time.^21,22 Most recently, Chen et al. described a system for chest-wall motion and volume evaluation based on colored structured light projection, where the active measurement region is predetermined with active markers placed on the torso of the patient.²³

In our paper, we present a 3-D measuring system based on the laser multiple-line triangulation method. The system enables real-time acquisition, processing, and display of the entire chest-wall shape, with the frequency generally limited by the camera (25 Hz in our case).

The main novelty is two-sided measurement of the subject’s torso from the front and the back simultaneously using the color modulation principle. This improves the accuracy of the measured volumes, because the lungs’ expansion is not only frontal but occurs in all directions. The color representation of the surface displacement is supported by the abdomen and rib-cage displacement difference, where the distinction between abdomen-dominant and rib-cage-dominant breathing provides additional clarity for assistance in breathing training.

The system verification is performed by comparing the measured values to such reference systems as the spirometer and calibration syringe. Visual feedback experiments are also performed in order to determine the system’s potential training and teaching capabilities for the medical treatment of various respiratory diseases.

2.

Experimental Setup

Figure 1 shows the measurement configuration, where the patient’s chest wall is acquired by the measuring module. The signal is processed and the 3-D shape is displayed on a personal computer in real time.

Fig. 1

Measurement configuration.

The schematic diagram of the 3-D measuring system based on the multiple-line laser triangulation principle is shown in Fig. 2. To illuminate the measured surface, a laser diode projector (Lasiris SNF-533L) was used in conjunction with a CCD camera (Sony XC-E150CE), which records the surface from a different viewpoint. Laser projector and CCD camera specifications are shown in Table 1.

Fig. 2

Schematic diagram of the laser 3-D measurement system.

Table 1

CCD camera and laser projector specifications.

The optical elements of the laser projector diffract the beam to generate 33 equally inclined light planes. We used a narrow-band interference filter, which only allows the laser light to pass (centered at 670 nm, 10 nm FWHM), and placed it between the lens and the camera’s CCD sensor. This improves the contrast of the acquired image by reducing the effects of ambient light. With the interference filter, the measurements can be performed under normal indoor lighting conditions, where the illuminance is less than 500 lx. After the image is acquired, additional postprocessing is required for successful 3-D shape reconstruction as described in Sec. 2.2.

Two different measurement setups for standing posture (one-sided and two-sided) were developed. The first measures only the chest-wall shape; whereas the second expands the measurement area to the shape of the patient’s back (see Fig. 3). The two-sided setup is based on the color modulation principle,²⁰ where each measuring module uses a different laser wavelength. That means that the narrow-band interference filter transmits only the light wavelength of the corresponding laser projector. In our case, the measuring modules are set to wavelengths of 670 and 635 nm. This enables accurate measurement and evaluation of the entire shape of the patient’s torso during breathing.

Fig. 3

Two-sided measurement setup (top view).

The measuring apparatuses were approximately 1.5 m away from the patient. The laser-light planes were aligned vertically, perpendicular to the floor. Such orientation of the measuring apparatus produced better visibility of the monitored chest area. The measuring range depends on the camera’s and projector’s focal lengths, CCD sensor dimensions, the triangulation angle, and the distance between the laser projector and the camera. In our case, the range is approximately $400\times 600\mathrm{mm}$ in width and height and approximately 500 mm in depth. In case of weak lighting conditions (dark skin or high ambient light), the patient should wear a white, thin elastic T-shirt, which reflects the light more effectively.

he entire process of successful 3-D shape reconstruction is divided into four sequential transformations. First, the image coordinates are transformed to normalized distorted coordinates. Next, the image distortion is corrected and the transformation to the 3-D camera coordinate system is performed. Finally, the transformation from the camera coordinate system to the global coordinate system is made by calculating the correct rotation and translation parameters. This procedure is executed on all points of the detected line segments. The points in 3-D space are then triangulated, displayed, and analyzed to determine the chest-wall shape and evaluate the volume differences during breathing.

2.1.

Calibration

The calibration is based on observing the surface of a reference object of known shape [see Fig. 4(a)]. A groove-shaped plate machined on the three-axe computer numerical control (CNC) milling machine with the accuracy of $\pm 0.01\mathrm{mm}$ was used for calibration along the $z$-axis from fifteen different positions at intervals of $20\pm 0.05\mathrm{mm}$ [see Fig. 4(b)]. At each position, the information of the light pattern is stored for further calibration steps.

Fig. 4

Calibration of the measuring module using a reference object (a) and the reference-object measured at 15 different positions (b).

The internal parameters, as well as the translation and orientation of the measuring module with regards to the global coordinate system, are determined by comparing the measured data with the reference surface, where the least-squares optimization is used. The reference-object geometry is groove shaped in order to detect also variation of transformation parameters, which have influence on the lateral distortion of the measured object [see Fig. 4(b)]. As a final result, the standard deviation between the measured and reference surface is 0.7 mm, which is equal to the accuracy along the $z$-axis. The details of the transformation as well as calibration procedure are in detail described in Ref. 20. The major advantages are that all the unknown transformation parameters are determined in a single measuring step and that the same measuring principle is used during calibration and during normal operation.

2.2.

Real-Time Data Processing

The real-time data-processing diagram is shown in Fig. 5. In the initial step, the body area is manually determined by selecting the outline of the patient’s torso. This value is retained for each individual measurement. After the acquisition, the line detection and line indexing are performed. The subpixel line-detection algorithm is based on the first-derivative zero crossing of the pixel values perpendicular to the lines. Indexing is based on the correspondence between the detected lines and the projected laser-light planes on the measured surface. The detected lines are indexed from left to right on each acquired image. The middle line with index zero is based on the higher pixel intensity of the corresponding projected laser-light plane (see “Recorded image” in Fig. 2). The next step includes the measured-area calculation for each acquired image in such a way that it detects the outer laser lines and calculates the surface of the inner area, limited by these lines. The values of the measured area are changeable and are used for exact volume calculation. Finally, the transformation from 2- to 3-D space occurs for every point of the detected line segment, after which the points in 3-D space are analyzed and displayed in a shaded or color mode.

Fig. 5

Data processing diagram.

Surface displacements in the viewing direction of the camera are calculated by subtracting the current measured surface from the reference one. This is performed by using OpenGL architecture and a z-buffer matrix, where the $z$ coordinate of each point is saved.²⁴ Surface displacements are displayed with a color palette, where blue represents the inward (negative) and red represents the outward (positive) movement.

With the selection of the Region of Interest (ROI), only the manually selected regions are observed. The surface displacement is numerically integrated over all points inside the measured area ($Am$) to obtain the measured volume ($Vm$). Due to inconstant measured area (see Fig. 6 and Videos 1 and 2), the ratio between the body area ($Ab$) and the measured area ($Am$) is used for calculating the volume of deformation under the body area (see Fig. 2). Thus, the deformation of areas which are not visible to the 3-D measuring system is approximated as an average deformation of the measuring area. Consequently, the corrected volume $Vc$ is determined by the following equation:

Eq. (1)

$$Vc=Vm·\frac{Ab}{Am}.$$

Fig. 6

Two examples of 3-D measurements of the chest wall for the abdomen-dominant breathing (upper row) of the male subject (Video 1) and the rib-cage-dominant breathing (bottom row) of the female subject (Video 2). (Video 1, MPEG, 3 MB [URL: http://dx.doi.org/10.1117/1.JBO.17.3.036004.1]; Video 2. MPEG, 3 MB. [URL: http://dx.doi.org/10.1117/1.JBO.17.3.036004.2])

3.

Verification of the Method by Measuring the Inhaled and Exhaled Volume of Air

One of the purposes of the study was to verify the accuracy of the laser 3-D system using standard methods of measuring volumes, such as a calibration syringe and spirometer. The calibration syringe verification method was performed on seven adult volunteers using one-sided measuring setup and on one volunteer using two-sided measuring setup (five healthy male, two healthy female, and one female with bronchiectasis). The average body mass index (BMI) was $24.2\pm 3.1\mathrm{kg}/{\mathrm{m}}^2$ and the average age was $26\pm 3$ years. The measuring system was also verified with spirometer on three individuals using one-sided measuring setup, where the BMI was $24.0\pm 2.1\mathrm{kg}/{\mathrm{m}}^2$ and the age was $28\pm 3$ years. Prior to the measurement tests, informed consent from all volunteers was obtained. In both verification methods, each volunteer was measured at least three times.

The calibration syringe is a piston device for a steady and controlled injection of gas into a respiratory-gas measuring instrument during its calibration procedure. The volume capacity of the cylinder is $0.5{\mathrm{dm}}^3$ with an accuracy of 0.5%.

Figure 7 shows examples of the measured and corrected volume values acquired with the laser measuring system while successively blowing $0.5{\mathrm{dm}}^3$ of air into the volunteer’s mouth. The measured volumes were corrected using the ratio between the body area and measured area (see Sec. 2.2). The measurement begins with the full exhalation of air from the volunteer’s lungs. After that, the air is injected into the lungs using the calibration syringe, which corresponds to a vertical step as the volume increases (see Fig. 7). Following the rapid vertical volume increase, the values remain quite constant, as this is the part where the calibration syringe refills. The described procedure is successively repeated with every syringe injection. A nose clip was used on the volunteers to avoid the loss of injected air through the nasal airway.

Fig. 7

Examples of chest volume difference for measured and corrected volumes during successively blowing $0.5{\mathrm{dm}}^3$ of air into the lungs. Four healthy male (a–d), two healthy female (e–f), and one diseased female volunteer (h) were measured using one-sided measuring setup. Another healthy male volunteer (g) was measured using two-sided setup.

The compression effect of the lungs is also visible from the last few injections in Fig. 7, where the volumes during the syringe refill decrease by more than 20%. After the measurements, the values of the laser measuring system and the calibration syringe are linearly interpolated, with the intention of determining the repeatability of each measurement. Although the correlation between the measured volumes is high ($R=0.998$), the slopes of the lines for individual measurements vary by more than 10%.

In our estimation, the main reason that the comparison with the calibration syringe achieves less accurate and less repeatable results is substantially connected to human respiratory mechanics. When the air is injected into the lungs, it remains in the windpipe or mouth, consequently making these effects invisible to our laser measuring system. The breathing diaphragm moves toward the abdominal cavity during the insufflation of air and there the compression effect (compression of gas within a bowel) is much larger. That would result in a smaller displacement of the thoraco-abdominal wall than that expected from the insufflated volume. Furthermore, when the lungs expand above the limit of voluntary, shallow respiration, the expansion ceases to be mainly frontal but occurs in all directions, which are not in the range of the laser’s measurement area. The occurrences described have a significant impact on the accuracy of the volume measurement.

To limit the aforementioned errors, the second verification was performed using a spirometer (Jaeger MasterScope Rhino), which is an apparatus for measuring the volume of air inspired and expired by the lungs during ventilation. Its principle is based on measuring the pressure differences with the flow accuracy of 0.2 to $12\mathrm{L}/\mathrm{s}\pm 2\%$. Verification with a spirometer is based on normal breathing cycles where the amplitude between inhalation and exhalation is used as a reference. The amplitude between successive breathing cycles was calculated based on peak and valley detection. Figure 8 shows the examples of corrected volumes measured by the laser system and a spirometer simultaneously for three volunteers. The duration of the entire breathing cycle is approximately 40 sec, and it is clearly visible that the volumes measured by the laser sufficiently follow the spirometer’s volumes throughout the measurement.

Fig. 8

Three examples of respiration waveforms measured with laser (−) and spirometer (---) for one healthy female volunteer (a) and two healthy male volunteers (b, c) using one-sided measuring setup.

Figure 9 shows a Bland-Altman plot²⁵ that displays the difference between the spirometer and the laser system. Most of the data points are well within the limits of the confidence interval ($1.96*\sigma =0.14{\mathrm{dm}}^3$).

Fig. 9

Bland-Altman plot for three healthy volunteers (one female and two males) comparing the spirometer and laser method with 95% limits of agreement ($n=3905$ points).

Using a spirometer for the verification process gives us better results in comparison with the calibration syringe, as shown in Fig. 10.

Fig. 10

Comparison of the measured volumes between the two reference systems (Vr) and the laser measurement system (Vl) for all measurements. The linear relations between the measured volumes are represented as a mean with a standard deviation for both the spirometer and the calibration-syringe reference method.

Contrary to the breathing induced by the calibration syringe, the breathing was more steady and shallow, and therefore the lungs’ expansion was mainly frontal when the spirometer was used. The laser system evidently measures volume more accurately in this case, as it covers the entire frontal area of the volunteer.

4.

Visual Feedback for Teaching and Correcting Breathing

One of the main advantages of the 3-D measuring system is the color representation of the chest-wall surface displacement (see Fig. 6). This is useful for rapid and simple real-time acquisition and the assessment of regional displacements only or the entire chest-wall breathing pattern. Visual feedback experiments were performed using both the one-sided and two-sided measuring setup.

Figure 6 and two animations (Videos 1 and 2) represent two examples of 3-D chest shape measurements, where the red and blue colors clearly expose the most active regions. In the upper row, the abdomen-dominant breathing of a male subject was measured, where the entire abdomen moved more or less uniformly outward. The rib-cage-dominant breathing of a female subject is shown in the bottom series. It can be noticed that the upper-chest part moves reasonably outward from the body and that the bottom part stays in the same position or even goes a little inward.

In addition to the color representation, the system can measure the regional displacements in order to determine the breathing pattern more accurately. The distinction between the abdomen-dominant and the rib-cage-dominant breathing for each subject is clearly visible from Fig. 11, where the differences between the rib-cage and abdomen area for the healthy male volunteer are shown. The displacement points were manually selected on the reconstructed body shape. The abdomen displacement was measured 50 mm above the navel point, and the rib-cage displacement was measured as an average of two points located 50 mm above both breast nipples.

Fig. 11

Example of the displacement waveforms of rib cage and abdomen in case of the abdomen-dominant (−) and the rib-cage-dominant (---) type of breathing for healthy male subject.

The abdomen-dominant breathing is mainly performed with the abdominal region, with the slight movements of the rib-cage following the outward motion of the abdomen. This accounts for the relatively small displacement difference. On the contrary, the rib-cage-dominant breathing shows that the displacements between both regions are greater, due to the simultaneous outward movement of the rib cage and inward movement of the abdomen.

5.

Discussion and Conclusion

A 3-D measuring system for teaching and correcting breathing in real time based on the multiple-line triangulation principle is presented. The surface shape is measured with an accuracy of $\pm 0.7\mathrm{mm}$. The most important features of the system are the ability to measure the complete chest surface, the high measuring speed, the noninvasive measurement principle, and the system’s portability.

In the medical field of correcting breathing irregularities or teaching breathing, the color representation of the active body regions is intuitive and simple to understand for variety of differently aged patients. Furthermore, because of the contactless nature of the system, the patient is not subjected to any physical load during the exercise. The system is also capable of measuring compartmental volumes and displacements, an approach which shows great promise in various disease treatments where only the selected chest-wall region is observed.

Although the current indexing method may cause problems when dealing with discontinuous surfaces in the $z$-axis direction, the measurements in our case showed no such problems due to the simplicity of the chest-wall surface and small displacements during breathing.

Verification of inhaled and exhaled volume using two standard methods of volume measurements was performed. The experiments show that the corrected volumes give more accurate results compared to the reference systems. This correction approximates the deformation of the areas that are outside the system’s measuring range (see Sec. 2.2). As expected, the comparison of the measured volumes with the spirometer data shows better agreement than with the calibration syringe, mainly because the accuracy of the measurements made with a syringe is limited by the physiological characteristics of the human body (see Sec. 3). The presented Bland-Altman plot offers additional evidence to the spirometer verification test, as it shows good agreement between the measured volumes with a mean near zero and a standard deviation of $\pm 0.07{\mathrm{dm}}^3$. Furthermore, the verification tests show that the system accuracy of the volume measurement is better during shallow breathing, when the lungs’ expansion is mainly frontal (i.e., aligned with the measurement direction). Therefore, to improve the accuracy of the volume measurement, the two-direction shape measurement using color modulation is an improvement on expanding the measurement area and consequently monitoring the deformation of the patient’s torso from the front and back simultaneously.

Another possible origin of the inaccuracy in volume measurement due to the lateral body expansion is the assumption of constant body area (Ab) throughout the measurement. By comparing the values of body area during consecutive breathing cycles, we found that the change is limited to less than 2% during deep breathing. This is much smaller than the relative change of the measuring area (approximately 8%).

The results demonstrate that the presented 3-D laser measuring system shows potential and a wide range of applications in the medical field as a diagnostic and training tool. The ability to measure regional volume changes and simultaneously evaluate the breathing pattern in real time can be used for monitoring or teaching breathing, when graphical communication with the patient is more effective than in other known methods. However, to justify fully the use of the system for teaching and correcting breathing, further clinical evaluation needs to be done.