The hearing process involves a series of physical events in which acoustic waves in the outer ear are transduced into mechanical motions of the middle ear, acoustic and mechanical motions in the inner ear, and then into chemo-electro-mechanical reactions of the inner ear sensors that are interpreted by the brain.1 Air in the ear canal has low mechanical impedance, whereas the mechanical impedance at the center of the eardrum, the umbo, is high. The eardrum or tympanic membrane (TM) must act as a transformer between these two impedances; otherwise, most of the energy will be reflected rather than transmitted.2,3 The acousto-mechanical transformer behavior of the TM is determined by its shape, internal fibrous structure, and mechanical properties.45.–6 Therefore, full-field-of-view techniques are required to quantify shape, sound-induced displacements, and mechanical properties of the TM.78.–9 In our previous works,1011.–12 we have reported holographic interferometric measurements of sound-induced displacements over a majority of the surface of mammalian TMs. A potential criticism of these measurements is that the displacements were measured only along one direction that was along the normal vector to the tympanic ring. Therefore, it was not possible to characterize all three-dimensional (3-D) motion components including those tangent (in-plane) and normal (out-of-plane) to the local plane of the TM. In this paper, developments of a single holographic system capable of measuring both shape with sub-millimeter resolution, and 3-D sound-induced motion of the TM with sub-micrometer resolution, are described. The accuracy and repeatability of the measuring system is tested and verified using artificial samples with geometries similar to those of human TMs. Then the system is used to measure the shape and 3-D sound-induced motions of human cadaveric TM samples at different tonal frequencies. Data obtained from the shape of the membrane are combined with the measured 3-D sound-induced motion components along three axes, , , and , in order to obtain the motion’s components tangent and normal to the local plane of the TM, enabling a more comprehensive view of TM mechanics.
Lensless Digital Holography
In conventional holography, an optical lens is used to focus on an object of interest; however, our techniques are based on lensless digital holography in which reconstructions and focusing of the holograms are numerically obtained by Fresnel–Kirchhoff integrals.13 Using temporal phase-stepping algorithms,14,15 the complex amplitude of the hologram, , is obtained withFig. 1, numerical reconstruction algorithms are based on two-dimensional (2-D) Fast Fourier Transform (FFT) of the product of a reconstruction reference wave, , complex amplitude of the hologram, , and a chirp function, , that can be obtained with Fig. 1, the chirp function is a complex 2-D oscillatory signal, where the frequency of oscillation linearly varies with the spatial coordinate and is used for numerical reconstruction of the hologram at different distances of . The reconstructed hologram, , is a complex function that contains both the amplitude and optical phase, , that is defined by
The fringe-locus function of a double-exposure (DE) hologram, i.e., the unwrapped optical phase difference of two reconstructed holograms corresponding to deformed and reference states of the object, is related to displacement with15,16
Stroboscopic Measurements of Displacement
Sound-induced vibrations of the TM are fast phenomena that require high-speed acquisition methods to be captured. In our system, we use stroboscopic measurements1718.19.20.–21 with a conventional speed camera to capture the repetitive fast motions produced by sinusoidal stimuli. Acoustically induced motions of the TM are frozen at different stimulus phases using pulses of laser light to illuminate the sample at particular points during the sinusoidal excitation signal. As shown in Fig. 2, a dual-channel function generator is used with one of the channels set to a sine wave for stimulating the TM through a speaker. The second channel is set to the same frequency but with a pulse wave to drive an acousto-optic modulator (AOM) to enable and disable the laser beam illumination. Typically, each laser pulse has a duration of 2% to 5% of the period of the tonal stimulus.11,18 This generates the same effect as a strobe light by only capturing the motion of the TM at desired phases of the stimulus wave.
A DE technique that compares the deformed state strobe hologram gathered at one phase and a hologram gathered at a reference phase (usually 0) is used to compute the displacement of a series of strobe holograms to describe the phase-locked sound-induced variation in the optical phase. The result is a wrapped phase map that describes the differences in optical phase between the deformed and reference states. At every DE strobe hologram, including reference and deformed states, the system records four images containing holographic patterns that result from the phase stepping of the reference beam (RB) in steps of multiples of . Considering the intensities at each pixel measured by the camera at each of the four phase steps to be in the reference state, and in the deformed state, the wrapped optical phase difference between any two states is related to displacements of the sample and is obtained with
Dual-Wavelengths Shape Measurement
The shape of the TM is measured with the method of dual-wavelength holographic contouring.22,23 The technique requires acquisitions of a set of optical amplitude and phase information at wavelength , as well as a second set of amplitude and phase information at wavelength . As shown in Fig. 3, depth contours related to the shape of the object under investigation are defined by
The phase difference obtained with Eq. (8) is equivalent to performing measurements with a synthetic wavelength of given by
In dual-wavelength contouring using phase-stepping algorithms, the phase difference, , is a discontinuous wrapped function varying in the interval , thus phase unwrapping algorithms are applied to obtain a continuous phase distribution, , for calculation of the relative height of each point on the surface of the object by
3-D Displacement Measurements
To ensure that measurements are reliable and independent of the measuring method, the holographic system is configured so that principal components of displacements, , , , can be measured with two different holographic interferometric approaches. The first one is based on the method of multiple illumination directions, whereas the second one uses a hybrid in-plane and out-of-plane displacement measurement method to obtain 3-D displacement data. In the hybrid method, in-plane measurements provide displacements’ components along the - and -axes (perpendicular to the observation direction) and out-of-plane measurements provide displacements’ components along the -axis (along the observation direction).
Method of multiple illumination directions
Full field-of-view, 3-D, sound-induced displacements of the TM are measured with the method of multiple illumination directions in holographic interferometry.24,25 In order to measure the three components of the displacement vector, , shown in Eq. (6), at least three independent measurements with different sensitivity vectors are required. In our approach, and to minimize experimental errors,24,26 optical phase maps are obtained with four sensitivity vectors to form an over determined system of equations that is solved in Matlab with the least-squares error minimization method withFig. 4, and is the fringe-locus function vector. In this method, all the sensitivity vectors need to be as linearly independent as possible for the system to provide accurate results. Therefore, the condition number, , of the square matrix, , characterizing the geometry of a holographic setup is calculated with27 15,28 However, because of the particular cone-like geometry of the TM and the presence of the bony structures around it, as illustrated in Fig. 4, the maximum possible angles of illumination are limited. Therefore, a holographic setup has to be arranged to achieve the largest angles of illumination within the constraints imposed by the geometry of the TM. In this case, the condition number is greater than one, therefore, the accuracy of the measurements obtained with such a holographic setup has to be verified.
Hybrid in-plane and out-of-plane method
A hybrid method that utilizes independent and direct measurements of “out-of-plane” and “in-plane” displacements is used to test and verify the measurements obtained with the method of multiple sensitivity vectors. In this hybrid method, in-plane measurements provide displacement components along the - and -axes (perpendicular to the observation direction), and out-of-plane measurements provide displacements along the -axis (along the observation direction), so that all three displacement components can be obtained independently.
Figure 5 shows optical configurations of the two measuring schemes. As shown in Fig. 5(a), an out-of-plane displacement induces a change in the OPL of the laser light (OPD). Based on the geometry of the system, OPD is related to the out-of-plane displacement with
In the case of in-plane measurements, the object is illuminated with two symmetrical beams that interfere with each other and realize a self-reference configuration.29 As shown in Fig. 5(b), once in-plane displacement, , occurs, the OPL of each interfering beam changes, which causes a relative phase change between the two interfering beams. Figure 5(b) shows only one of the in-plane components of displacement, however, displacement components along both the - and -axes are independently measured with this method.16) and (17), in-plane displacement, , is calculated with
The schematic of the developed Digital Opto-Electronic Holographic System (DOEHS) is shown in Fig. 6. The DOEHS can measure microscale variations in shape as well as nanoscale displacements of the TM using both 3-D displacement measurement methods described in Sec. 2.4. The DOEHS consists of three main subsystems including laser delivery (LD), computing platform (CP), and optical head (OH). An external-cavity tunable diode laser capable of continuous tuning with minimal mode hopping (New Focus, Velocity) is placed in the LD, which provides a near IR laser light with a central wavelength of 780 nm. A polarization maintaining fiber coupler (Thorlabs PM-780-HP) splits the light into two beams to be used as reference (10%) and object (90%) beams. A MEMS optical switch, with a response time of less than 0.5 ms (Thorlabs OSW 8104), is used to multiplex between the object beams (OB) of each illumination direction once triggered by a Digital to Analog signal. The CCD is illuminated with the RB through a beam splitter and through a piezo-mounted mirror that is used for phase stepping. A 5 Megapixel () digital camera with a pixel pitch of in the OH is used for image recording and the CP acquires and processes images in either time-averaged or DE stroboscopic modes.30 The sinusoidal output of the function generator is amplified by a unity gain power amplifier and is used to drive the speaker. A microphone with a calibrated probe tube was positioned just at the edge of the TM to make measurements of sound pressure. In stroboscopic measurements, the light illumination is synchronized with the sound-waveform by means of a digital pulse signal from the function generator to the AOM.
Determination of the Sensitivity Vectors
In the method of multiple illumination directions, accurate determination of the position of each illumination source is required to define the corresponding sensitivity vectors. A new technique is developed in which all the sensitivity vectors are obtained by automatic analyses of the images of a specular sphere illuminated from different directions in order to avoid the uncertainties introduced by manual measurements. As shown in Fig. (7), a specular sphere is placed in front of the imaging system to acquire images from all four directions of illumination. The center of the image is marked in the acquisition software LaserView30 and the sphere is accurately located in this position and within a circular mask provided by the software to ensure that the sphere is placed along the optical axis of the holographic system. As shown in Fig. 7(a), the mirror-like reflection creates a specular highlighted area on the sphere.31 Each image is cropped and enhanced by image processing techniques that include histogram equalization, threshold definition, and edge detection. Then, a sphere is numerically fitted on the detected circular edge which represents the outline of the sphere. The normal vector of the fitted sphere at the centroid of the specular highlighted area, for each direction of illumination, defines the sensitivity vector, , which can be expressed as32 A circle with radius and center can be described with the parametric equations
Validation of Measuring Accuracy and Repeatability
Considering the acquisition speed of the 3-D holographic system and the time-dependent mechanical behavior of biological samples like the TM, we characterized the accuracy and repeatability of the measurements of the 3-D holographic system using measurements of artificial samples that have negligible time-varying behaviors. Based on the particular concave shape of the TM, a semispherical membrane with a geometry similar to the TMs is used. For 3-D displacement measurements of non-flat membranes, the large illumination angles necessary for a well-conditioned 3-D holographic system create shadows on the surface of the membrane. Therefore, the maximum angles of illumination are limited by geometrical constraints induced by the membrane, leaving the holographic system with condition numbers greater than one, as described in Sec. 2.4.1. To calibrate the measuring system and to verify the accuracy of the measurements obtained with such non-ideal 3-D holographic configuration, 3-D displacement components are measured using both of the methods described in Sec. 2.4, in order to ensure that the obtained displacement components are accurate and independent of the measuring approach. Once the accuracy of the measurements is verified, repeatability of the stroboscopic measurements is tested and verified with sound-induced displacement measurements of a latex membrane.
Accuracy of 3-D displacement measurements
The shape and 3-D displacements of a thin semispherical membrane, shown in Fig. 8, with a radius of 6 mm and a thickness of are measured with the developed holographic system. The shape of the membrane is measured with a dual-wavelength contouring method with wavelengths of 779.8 and 780.6 nm. Figure 8(b) shows the wrapped optical phase, which is unwrapped and scaled to obtain the corresponding 3-D shape, as shown in Fig. 8(c).
The semispherical membrane is mounted on a piezoelectric shaker (JODON EV-100) that can operate at frequencies as high as 150 kHz. By sweeping the excitation frequency and monitoring the membrane’s time-averaged motions,10,16,33 an appropriate mode of vibration is chosen. Representative results are shown in Fig. 9, in which the membrane is excited with a frequency of 25,418 Hz and amplitude of 0.4 V. Comparisons of the displacement components along the -, -, -axes, and the magnitudes of displacements obtained with both methods show correlation coefficients , illustrating that the measurements are accurate, repeatable, and independent of the measuring method.
Repeatability of stroboscopic measurements
Repeatability of the results obtained with the 3-D holographic system is tested and verified by a series of consecutive stroboscopic measurements of a 10 mm diameter latex membrane excited with a tone of 2917 Hz at 91 dB sound pressure level (SPL), as shown in Fig. 10. Figures 10(a)–10(c) show representative examples of the obtained modulation, wrapped optical phase, and magnitude of displacements, respectively. Furthermore, repeatability of the measurements is shown in Fig. 10(d), where vertical (shown with dashed lines) and horizontal (shown with solid lines) cross-sections of six consecutive displacement measurements lie on top of each other.
Shape and 3-D Sound-Induced Displacements of Human TM
The TM sample was part of a human right ear temporal bone from a 49-year-old male donor. The sample was prepared in accordance with previously established procedures.9,12 In order to have the least amount of shadow on the surface of the TM, all the bony structures around the TM were removed. In preparing the specimen, it was necessary to also open the middle ear cavity, however, those openings were filled by silicone impression materials (Westone Inc.) prior to these measurements in order to avoid and minimize the air flow through the middle ear cavity. Also, to enhance light reflection from the sample and to reduce the required camera exposure time in order to have a better signal to noise ratio, the lateral surface of the TM was coated with a solution of zinc oxide. The effect of this coating on the vibrational patterns of the TM has been shown to be negligible.910.–11 The shape of the sample was measured with dual-wavelength holographic contouring, as shown in Fig. 11, with two wavelengths of 780.2 and 780.6 nm. Figure 11(b) shows the wrapped optical phase, which is unwrapped and scaled to obtain corresponding 3-D shape, as shown in Fig. 11(c).
Tones with different frequencies were used to stimulate the membrane and 3-D sound-induced displacement components of the surface of the TM were acquired. Figure 12 shows 3-D displacement components of the TM along the -, -, and -axes produced by tones of 1560 Hz at 108 dB SPL, 4480 Hz at 101 dB SPL, and 8021 Hz at 96 dB SPL. The levels were selected to generate measurable sound-induced TM displacements. As shown in Fig. 12, as sound excitation frequency increases, the number of local maxima in the displacement patterns also increases and sound-induced motion patterns of the TM become more complex.
Combining data obtained from the shape of the membrane, shown in Fig. 11, with measured 3-D sound-induced displacement components, shown in Fig. 12, displacement components tangent (in-plane) and normal (out-of-plane) to the TM surface are obtained. A numerical rotation matrix is used34 to rotate the original Euclidean coordinate system of the measuring system , so that the new coordinate system has unit vectors tangent and normal to the TM surface. Figure 13 shows the results of the rotation of the coordinate system, where rotated displacements have components tangent (in-plane components and ), and normal to the TM surface (out-of-plane component ). As shown in this figure, in-plane components are much smaller than out-of-plane components (), so that the displacement vectors can be considered to be mainly normal to the surface of the TM.
These data support hypotheses based on considering the motions of the TM similar to those of thin-shells, in which the tangential motions’ components are negligible and the motion vectors are hypothesized to be mainly along the normal vector of the surface of the membrane.8,9
While there are many hypotheses of how the TM couples sound to the rest of the ear, there is little data to support them.2,3,8 Knowledge about the shape and 3-D sound-induced displacement of the TM are necessary sets of data in order to better understand the acousto-mechanical transformer behavior of the mammalian TMs. In this direction, we are developing optoelectronic holographic systems capable of measuring shape with sub-millimeter resolution, and 3-D sound-induced displacement of the TM with sub-micrometer resolution. Combining the measured shape and 3-D sound-induced displacements of the TM at each point on its surface enables characterization of the motion’s components tangent and normal to the TM surface. Results show that the tangential motions’ components are much smaller () than the out-of-plane motions’ components. These results are consistent with the modeling of mammalian TM as thin-shells in which the tangential motions’ components are negligible.
Rotation Matrix used to Obtain In-Plane and Out-of-Plane Displacements
The original Euclidean coordinate system , , is mathematically rotated in order to obtain the local in-plane and out-of-plane displacement components. In the holographic system and based on the definition of the sensitivity vectors, the observation vector , i.e., a vector perpendicular to the CCD sensor, has unit vector components , , and equal to [0, 0, 1]. Also, by having the 3D shape of the membrane, the unit normal vector, , at every point on the surface of the TM is quantified. Since, both and are unit vectors, the angle between them is calculated with dot product of the two vectors; and the cross product of these two vectors provide a vector, , normal to both of them that, in this case, is tangent to the local plane of the membrane and is considered as the axis of rotation.
The rotation matrix , is used to rotate the original displacement vector , based on the rotation angle and the unit vector of the axis of rotation withFig. 14, at each point , , the rotated displacement vector has components tangent ( and ) and normal () to the local TM plane and is calculated with the matrix multiplication of the rotation matrix, , with the original displacement vector with
This work was supported by the National Institute on Deafness and other Communication Disorders (NIDCD), the National Institutes of Health (NIH), Massachusetts Eye and Ear Infirmary (MEEI), and Mechanical Engineering Department at Worcester Polytechnic Institute. We also acknowledge the support of all of the members of the CHSLT labs at WPI and Eaton-Peabody labs at MEEI, in particular Ellery Harrington and Ivo Dobrev.
Morteza Khaleghi is a PhD candidate in mechanical engineering at Worcester Polytechnic Institute, Worcester, Massachusetts. His research interests include biomedical imaging and developments of optical and nondestructive metrology systems.
Cosme Furlong is an associate professor of mechanical engineering at Worcester Polytechnic Institute, Worcester, Massachusetts, working in mechanics, optical metrology, and nanoengineering science and technology.
Mike Ravicz is an associate scientist and biomechanical engineer at Massachusetts Eye and Ear Infirmary, Boston, Massachusetts, exploring the mechanics and acoustics of the external, middle, and inner ear as well as working in signal processing and noise and vibration control.
Jeffrey Tao Cheng is an instructor in otology and laryngology at Massachusetts Eye and Ear Infirmary, Boston, Massachusetts, working on middle-ear mechanics and ear tissue biomechanics through measurements and modeling.