Translator Disclaimer
Open Access Presentation + Paper
23 April 2020 Machine learning for structural health monitoring: challenges and opportunities
Author Affiliations +
A physics-based approach to structural health monitoring (SHM) has practical shortcomings which restrict its suitability to simple structures under well controlled environments. With the advances in information and sensing technology (sensors and sensor networks), it has become feasible to monitor large/diverse number of parameters in complex real-world structures either continuously or intermittently by employing large in-situ (wireless) sensor networks. The availability of this historical data has engendered a lot of interest in a data-driven approach as a natural and more viable option for realizing the goal of SHM in such structures. However, the lack of sensor data corresponding to different damage scenarios continues to remain a challenge. Most of the supervised machine-learning/deep-learning techniques, when trained using this inherently limited data, lack robustness and generalizability. Physics-informed learning, which involves the integration of domain knowledge into the learning process, is presented here as a potential remedy to this challenge. As a step towards the goal of automated damage detection (mathematically an inverse problem), preliminary results are presented from dynamic modelling of beam structures using physics-informed artificial neural networks. Forward and inverse problems involving partial differential equations are solved and comparisons reveal a clear superiority of physics-informed approach over one that is purely datadriven vis-à-vis overfitting/generalization. Other ways of incorporating domain knowledge into the machine learning pipeline are then presented through case-studies on various aspects of NDI/SHM (visual inspection, impact diagnosis). Lastly, as the final attribute of an optimal SHM approach, a sensing paradigm for non-contact full-field measurements for damage diagnosis is presented.
Conference Presentation



Structural health monitoring (SHM) is a transdisciplinary field of engineering devoted to ensuring the structural integrity and operational safety of a component or structure. This is achieved by facilitating the (in-service) detection and characterization of damage in a structure that may adversely affect its ability to fully and safely perform its intended function. As such, the problem of damage detection lies at the heart of SHM and the goal is to identify the damage at the earliest possible stage (in near real-time). This is indispensable for ensuring timely corrective action in order to minimize the system downtime, overall operational and maintenance costs, and to reduce the risk of catastrophic failure. Broadly speaking, as can be seen in Fig. 1, two approaches have traditionally been adopted for automated damage detection in SHM: physics-based and data-driven.

Figure 1:

The big picture of structural health monitoring (SHM) indicating that as the complexity of the structure increases, exclusive reliance on the system physics for analyzing the measured sensor data is reduced, and an approach that is primarily data-driven is adopted. An optimal approach for large/complex real-world structures must be data-driven with the system-physics/domain knowledge embedded in some form.


Physics-based methods, in some form or the other, rely primarily on the physical laws governing the structural behavior in order to extract meaningful information about the damage and its evolution from the measured sensor data. However, difficulty in the modelling of complex real-world structures, considerations of multiple sensing modalities, material and/or geometric non-linearity, and uncertainty in material properties, boundary conditions, environmental/operational variations are some of the factors that make this exclusive reliance on the system physics rather impractical in case of complex real-world structures. This limits the application of such methods to the health monitoring of rather simple structures with pre-defined boundary conditions and well-controlled environments [1-3]. As the underlying system complexity increases, such an approach becomes much less dependable. Owing to the advances in information and sensing technologies in recent times, it has now become feasible to monitor a large number of parameters in-situ in large/complex real-world structures on either a continuous or sporadic basis. This motivates the use of a data-driven approach for SHM where-in damage assessment is dealt with, at least at a lower level, primarily as being a type of statistical pattern recognition problem thus circumventing some of the major challenges associated with the physics-based approach.

During the last few decades, machine learning (ML) techniques have been extensively employed by researchers both for vibration based and ultrasonic guided wave based damage detection [4-14]. Machine learning in SHM aims at building models or representations for mapping input patterns in measured sensor data to output targets for damage assessment at different levels, Rytter [15]. Conventional machine learning techniques are however limited in their ability to process the large amounts of measured sensor data in their raw form. As such, careful engineering and considerable domain knowledge are required to extract damage-sensitive features from the raw data which are then fed into a suitable ML model. Historically, the choice of these hand-crafted damage-sensitive features has stemmed from the wealth of well-developed literature on physics-based SHM techniques including but not limited to modal assurance criterion (MAC) and Coordinate MAC [16, 17], modal strain energy (MSE) [18, 19], modal curvature (MC) [20, 21], modal flexibility (MF) [22-24], damage locating vector (DLV) [25], wavelet transform [26-30], Hilbert-Huang transform [31], probabilistic reconstruction algorithm (PRA) [32], ZLCC [33] etc,. Multi-layer feedforward artificial neural networks (ANNs), also referred to as multi-layer perceptrons (MLPs), have been the most well-known and widely used ML technique for automated damage detection [34-37]. Apart from the algorithm itself, the overall performance of such a damage detection system depends on the choice of the damage-sensitive features used. The problem with hand-crafted features is that while they may be sub-optimal even for the structure under consideration, there is no guarantee that the same set of features can be adopted for other structures. In order to overcome this shortcoming, deep learning (DL) methods have attracted much attention in recent years. These allow data to be used in their raw form and features are learned automatically from data using a general-purpose learning procedure. Deep learning automatically discovers intricate features in high-dimensional data which has led to its widespread adoption across many application domains [38-41]. It is certain that DL will continue to flourish in the future and the progress will only be accelerated with advances in computational capabilities through the development of high-tech central processing units (CPUs)/graphics processing units (GPUs), availability of large amounts of data (big data), and development of new learning algorithms.

While supervised learning is the most common and well developed form of learning, a fundamental challenge in many SHM applications is that damage detection must often be performed in an unsupervised manner. This is because, for real-world structures, it is highly unlikely to have data corresponding to different damage scenarios. Among the four levels of damage assessment identified by Rytter [15], the lowest level (i.e., establishing the presence of damage) has been achieved in the past through unsupervised learning by what is referred to as novelty (outlier) detection [42-44]. Higher levels of damage assessment in real-world structures require either a mechanism for augmenting the insufficient/incomplete training data by incorporating some form of prior knowledge into the learning/training process and/or the development of more comprehensive data acquisition systems. These will henceforth be referred to as the two aspects of an optimal SHM system. This paper introduces these two aspects through a discussion on physics-informed learning in Sections 2, 3 and 4 and non-contact full-field measurements using state-of-the-art high-speed digital cameras for vision-based SHM in Section 5. Summary, conclusions and future prospects are finally presented in Section 6.


Physics-informed Machine Learning

Mathematically speaking, the problem of automated damage detection from measured sensor data is an inverse problem. In conventional machine learning/deep learning, it is usually formulated as a minimization problem with a purely data-based loss function. However, in most SHM applications, not only is the cost of data acquisition prohibitive, but it is highly unlikely to have sufficient data capturing different damage scenarios, especially for structures that have been in service for relatively short periods of time. When trained using this inherently limited data, the models tend to overfit the given data which eventually leads to poor generalization. This gives rise to the need for some form of model regularization which primarily entails guiding the training process towards an optimal solution in one way or the other. Accomplishing this can result in what may be thought of as data-efficient function approximators.

The requisite domain knowledge for model regularization (guiding the training process) can originate from various sources, can be represented in different forms, and can be integrated into the model in different ways [45]. From the literature on informed machine learning, in case of ANNs, knowledge has been integrated at all three stages–input (pattern) layer [46-53], hidden layers [54-57], output (target) layer [58-60]. Specifically within the context of SHM, domain knowledge can:

  • (I) Aid in the extraction of damage-sensitive features from raw sensor data in cases where conventional machine learning is used and data cannot be used in their raw form.

  • (II) Augment the measured sensor data through transfer learning or by providing secondary data sets from simulations, historical NDI/SHM records or data from similar structures. The diversity of the available historical data usually leads to the need for some form of data fusion as a pre-processing step.

  • (III) Dictate the network architecture by defining local connectivity of nodes within the network, choice of activation functions, learning algorithm, etc.

  • (IV) Impose constraints on the output through proper loss function design or knowledge about damage types and characteristics.

The list is by no means exhaustive as knowledge-based learning in SHM is still in its infancy and is envisioned to bloom in the coming years. In this paper, two classes of problems are discussed based on the type of domain knowledge utilized:

  • 1. When the prior knowledge is in the form of physical laws governing the structural behavior and the same is representable in the form of compact mathematical models governed by partial differential equations (PDEs). The integration of this knowledge into the model is done directly at the loss function level i.e., apart from having a data-based component, the loss function is designed to have a component that is physics-based as well. This is discussed in more detail in Section 3 with the help of forward and inverse problems for dynamic modelling of beams which can be thought of as a promising first step towards the eventual goal of damage detection.

  • 2. When the prior knowledge is available in other less succinct forms like expert human knowledge about a particular system/structure, results from computer simulations which mimic the real-world situation, general information about an underlying physical phenomenon etc,. This form of knowledge can be integrated into the model through strategies like transfer learning, network architecture customization etc,. Section 4 discusses this in more detail with case studies directly related to NDI/SHM.


Prior knowledge in the form of Compact Mathematical Models (PDEs)

Figure 2:

Conventional and ANN-based approaches for solving differential equations


Differential equations is the lingua franca long developed to formulate physical phenomena with both accuracy and succinctness. A purely analytical solution to these equations is feasible only when the underlying structure/phenomenon being modeled is rather simple and straight-forward. Also, the Green’s function approach is usually limited to linear time-invariant cases for unbounded domains. Often times, owing to the inherent structural complexity and/or the nature of the initial/boundary conditions, such approaches no longer remain tractable. It is in these situations that numerical methods for solving differential equations come to the rescue. Some commonly used methods include Runge-Kutta methods, collocation method, weighted residual method etc. The recently popular methods such as finite-element, finite-volume, finite-difference etc., which involve discretization of the domain and weakly solving the differential equations over this discretization yield solutions that are discrete and have limited differentiability.

All these numerical methods have been studied for a long time and are well developed, however, when the source functions and/or initial/boundary conditions change, the results need to be computed afresh from scratch in each case. The experience gained from each computation is lost as there is no fundamental learning of the underlying mechanics. As the size of the structure increases, each simulation becomes computationally expensive owing to the curse of dimensionality and as a result these methods become prohibitively slow. If however a mapping is learned for the structure/phenomenon being modelled, slight variations in the input can be handled rather quickly. ANNs are comprised of a densely interconnected set of very simple processing units (nodes) that can learn highly challenging non-linear mappings with high robustness, adaptability, and fault tolerance due to its distributed representation [61]. Researchers have long been interested in developing representations based on ANNs (which act as universal function approximators [62]) for approximate solutions to these differential equations [63-71]. Research in this direction blossomed especially after 2010 with the advent of GPUs (monstrous computational power) and TensorFlow (a deep learning framework with embedded automatic differentiation).

Raissi et al. [72, 73] solved a variety of non-linear PDEs including Brown equations, Schrodinger equations etc., using TensorFlow. Then they combined this purely numerical method for solving differential equations with traditional data-driven approach to function approximation and actualized the idea of physics-informed learning. By doing so, they were able to not only use sparse datasets to reliably train neural networks, but also to inversely determine unknown parameters of PDEs [74]. Around the same time, several other researchers started investigating physics-informed learning and some of that work can be found in [75-80]. In this paper, the solution of a beam vibration problem using an ANN is presented first followed by a direct demonstration of system physics coming to the rescue when training neural nets with limited data within the context of SHM. The latter is achieved by solving an inverse problem using the same ANN architecture used for the forward problem, for reconstructing the full displacement field using measured sensor data from a sparsely distributed array mounted on the beam. Comparison with the exact analytical solutions is made to highlight the accuracy and generalization capabilities of the physical-informed learning paradigm.


Beam Vibration Modeling using ANNs (Forward Problem)

Based on Euler-Bernoulli beam theory, the vibration of a simply supported beam under transverse load q(x,t) is governed by Eq. (1) with boundary conditions (BCs) described by Eq. (2)

Figure 3:

Setup of a simply supported beam


For free vibration of the beam (q=0), let us suppose the initial conditions (ICs) are given by


where μEI/ρA, E is the elastic modulus of the beam’s material, I is the area moment of inertia, ρ is the density of the material, A is the cross sectional area, l is its length, and w is the transverse displacement. Collocation, which entails choosing a number of points (called collocation points) in the spatial and temporal domains, is adopted and the loss function ℒ for solving the PDE using an ANN consists of three terms and can be expressed as follows:


where (x,t) is the predicted displacement and the first term in ℒ is the mean squared error (MSE) term associated with the initial conditions, the second term corresponds to the boundary conditions and the third term to the governing equation. NI, NB, and NG stand for the number of collocation points selected from T, ∂Ω, and Ω respectively. The weights αI, αB, and αG for the three terms are determined heuristically to be 1, 1, and 2×10-4 respectively. For free vibration of the simple supported beam with μ=1, l=1 and


the exact solution for transverse displacement can be easily obtained as w(x,t)= sin(3πx).cos(9π2t). The steady-state solution can be considered as a superposition of two transient waves with identical amplitude and propagating in opposite direction as w(x,t)= ½{sin[(3π(x−3πτ)]+ sin[(3π(x+3πτ)]}.

The collocation method implemented via the ANN shows good convergence as long as the time is short, however, for long time durations, convergence problems are encountered and the solutions often converge to the trivial solution. To circumvent this problem, the entire time duration is segmented into a finite number of short time-windows and solved sequentially. By invoking Huygen’s principle, which states that every point on a wavefront is itself the source of secondary wavelets, the solution at the end of a particular time-window can be treated as the initial condition for the subsequent time-window. For this example, the time domain is divided into six time windows (to satisfy the Nyquist criterion). For each window, an ANN with eight hidden layers each of which consists of sixty four neurons is used to approximate the full displacement field. For expediting the process and ensuring convergence, the training process in each case is divided into two phases. The first phase primarily focuses on fitting the ICs and BCs on a data-driven basis, thus the last term in the loss function ℒ is excluded. A gradient descent method is performed first (number of collocation points NI = NB = 50) followed by quasi-Newton algorithm (a limited-memory quasi-Newton code for bound-constrained optimization, L-BFGS-B) (NI = NB = 150) to guide the solution closer to the global minimum in a rather coarse but speedy manner. In the second phase, the term corresponding to the governing equation is added back to the loss function, then L-BFGS-B is employed to finely optimize the parameters of the ANN. The collocation points are selected from a uniform 150 × 150 grid such that NI = NB = 150 and NG = 22500. The results displayed in Fig. 4 show good agreement between the exact solution w(x,t) and the predicted solution (x,t).

Figure 4:

Comparison of the MLP predicted solution with the exact solution for the simply-supported beam with given initial conditions



Displacement field Reconstruction using Physics-informed ANNs (Inverse Problem)

After demonstrating the applicability of ANNs for solving PDEs in the previous example as a forward problem, the power of physics-informed learning for the inverse problem is now demonstrated vis-à-vis training with limited samples. This is done via a problem involving displacement field reconstruction from a sparse sensor array. Fig. 5 shows the schematic of a simply supported beam with a surface mounted sparse sensor array used in the study. The sparse sensor data (xi,t) is synthetically obtained such that (xi,t) = w(xi,t) + n(xi,t) where xi is the location of the ith sensor, w(xi,t) is the exact displacement at xi sampled from the analytical solution and n(xi,t) is the Gaussian noise with zero mean. The goal is to reconstruct the full displacement field from the sparse measurements and highlight the advantage of embedding the governing equation into the loss function. The loss function for solving it using an ANN consisting of four terms can be defined as follows:

Figure 5:

Setup of a simply supported beam with a sparse sensor array


where the first term is the purely data-driven loss, the second term is for the Dirichlet boundary conditions, the third term is for the governing equation, and the fourth term is for high order boundary conditions. The weights αD, αB1, αG and αB2 for the four terms are determined heuristically to be 5000, 25000, 1 and 1 respectively. For the simple supported beam shown in Fig. 5 with w(x,t)= sin(πx).cos(2π2t) and μ=4, in the first phase, the parameters of the ANN are trained with Adam optimizer with a learning rate of 0.01 for 10000 iterations. During each iteration, while all the ND measurements are used, the collocation points are selected from the defined domain. To be specific, NB = 50 points are sampled for the boundary conditions and NG = 2500 points are sampled for the governing equation. After that, the parameters are further optimized with L-BFGS-B on a larger number of static collocation points until the loss converges. The collocation points for L-BFGS-B optimizer are taken from an equally spaced 150×150 grid over the entire domain, so that NB = 150 and NG = 22500. Fig. 6(a) shows the comparison between the results obtained with physics-informed learning and those obtained using the sparse data only and Fig. 6(b) shows the error for the two approaches as the sparsity of the data and SNR changes. Clearly physics-informed learning outperforms traditional data-driven learning vis-à-vis overfitting when data is sparse and/or noisy. Table 1 presents the comparison in a tabulated manner. The maximum relative error metric used for comparison is defined as (max |w|/max |w|).

Table 1:

Physics-informed learning vs data-driven learning

SNR=20 20 SensorsSNR=10 20 SensorsSNR=15 13 SensorsSNR=20 5 SensorsSNR=10 5 Sensors
Physics InformedTraining Time (s)2293.331808.921864.431846.191398.11
Max. Error2.7%2.4%0.72%0.95%1.9%
Data-DrivenTraining Time (s)48.2548.2442.8852.3537.21
Max. Error3.9%7.2%6.4%29.7%30.3%

Figure 6:

(a) Comparison of the results for the displacement field reconstruction using physics-informed approach and purely data-driven approach (b) Variation of the error for physics-informed learning and data-driven learning as SNR value and data sparsity changes


If the system parameter μ is unknown and is to be identified, it can be set as a trainable parameter and inversely determined through training. For instance, a displacement field 00102_psisdg11379_1137903_page_9_2.jpg is constructed by linearly combining six mode shapes where the coefficients λn are random sampled from [-1, 1]. The system parameter for generating this displacement field is μ=0.0156 but assumed unknown. Noisy measurements extracted from this field are then provided to an MLP for it to reconstruct this displacement field and identify the unknown parameter at the same time. Same training strategy is used as above and the result is shown in Fig. 7. μ is determined to be 0.01555, with a relative error of only 0.32%.

Figure 7:

Results for the full displacement field reconstruction for the given beam when the parameter μ is not provided


To summarize, in this section, initial promise has been demonstrated for the use of ANNs for solving PDEs in forward problems and simple inverse dynamic problems for beams. This can be regarded as a promising first step towards the ultimate goal of solving more complicated inverse problems for automated damage detection in SHM. This logic is analogous to the historical evolution of conventional numerical methods for inverse problems through the development of different regularization schemes after methods for solving forward problems were well matured. Next, two case studies are presented to demonstrate how knowledge in other forms could be utilized for augmenting the ML/DL models. This is done through cases directly falling within the context of NDI/SHM. It is worth mentioning that while the discussion in Section 3 dealt primarily with discrete sensor array data, Sections 4 shifts the focus to vision based health monitoring with Section 4.1 being a static problem (still images as input) and Section 4.2 being a dynamic problem (wavefield videos as input).


Examples on Physics-informed Deep Learning in NDI/SHM

In this Section, a physics-informed deep learning (DL) approach for enhanced visual inspection via augmented reality is first presented as a demonstration of how domain knowledge in the form of expert human feedback and historical NDI/SHM records can be used to augment the training data. This is followed by a demonstration of impact diagnosis using a physics-informed deep learning model.


Enhanced Visual Inspection via Augmented Reality (AR)

The first direct demonstration of the role of domain knowledge in augmenting the ML/DL models for tasks directly related to NDI/SHM is done through what is referred to as enhanced visual inspection. In the aerospace industry, for safety critical structures, visual inspection (VI) is usually the first line of defense and given its various advantages, it accounts for at least 80% of all aircraft inspections according to FAA [81]. Apart from being considerably flexible, it is the most direct, intuitive, straightforward, and economical method for accessing the condition of a structure [82]. However, when performed in its traditional form (by a human inspector without any external aids), VI has certain inherent shortcomings like, being error-prone, labor-intensive, tedious, subjective/inconsistent, etc [83-85]. An inspector is required to first acquire relevant training over a period of time and then to bridge the gap between the acquired knowledge and the physical systems in the field mentally which limits his/her ability to make the best use of the wealth of information available about the inspection task being performed.

The use of augmented reality (AR) as an information delivery paradigm has been proposed in order to overcome some of the major shortcomings of traditional VI. AR encompasses a set of technologies that superimpose digital data and images on the physical world thereby enhancing the user’s perception of reality [86]. It has the potential to bridge the disconnect between the physical and digital worlds, thus enabling humans to make efficient use of the information available and consequently make more informed decisions. As such, it can act as a tool to enhance the traditional visual inspection process. A proof-of-concept for such an enhanced visual inspection system using a head mounted AR device (AR glasses) is presented here. The primary focus in the study was the detection of most commonly occurring defects in metallic structures– corrosion, fatigue cracks, and/or a combination of the two. This was achieved by deploying a deep learning (DL) based computer vision model on the AR device.

Oftentimes, considerations of system downtime and accessibility prohibit a detailed inspection of the entire system. Under such circumstances, while traditional VI may give rise to high rates of false-positives (false alarms) and/or false-negatives (misdetections), using a purely data-driven approach to search for damages/anomalies over the entire region of interest (ROI) can be extremely expensive computationally making real-time detection almost impractical with the current state-of-the-art AR devices. Also, depending on the distance and/or angle between the inspector and the area being inspected, a damage may or may not be easily discernible. As such, with the aim of improving the probability of detection (POD) enhancing the overall efficiency of the detection process, prior domain knowledge was incorporated into the ML pipeline. Fig 8 gives an overview of the type of domain knowledge available and its integration into the machine learning pipeline. The details are explained next.

Figure 8:

Overview of the type of domain knowledge available and its integration into the machine learning (ML) pipeline for accomplishing enhanced visual inspection


Rather than searching only for damages (in this case cracks/corrosion) in the actual ROI, regions that are prone to damage are also categorized and grouped. The information about damage prone areas can come either from expert knowledge about the system or from historical data. As an example, areas prone to cracking may include regions with similar types of fasteners like bolts, rivets, etc. In this way, a preliminary analysis of the ROI may lead to the detection of prominent damages and/or damage prone regions. If a damage prone region is detected, the inspector is prompted to zoom into the region in order to carry out a more detailed analysis with the aim of detecting less prominent damages i.e., damages that are otherwise below the detectability threshold of the algorithm at the original scale of view. This can be achieved either by using the zoom-in function in the AR device camera or by simply walking closer to the region if there are no accessibility issues. In this way, analyzing the ROI at multiple scales (prompted by expert/domain knowledge about the system) has the potential to greatly increase the POD while reducing false alarms. It should also be noted that since a detailed analysis is carried out only of the highlighted damage prone regions, this region-based detection paradigm has the potential to make the overall inspection process much more efficient than carrying out a detailed inspection of the entire system. A schematic of the proposed physics-informed methodology is shown in Fig. 9 and some representative results are shown in Fig 10. The results were obtained using Epsom BT-300 smart glasses (AR/Developer edition) with a pre-trained MobileNet (SSD MobileNet V2) and k-means clustering was used to group the detected fasteners together. More details about the DL algorithms used in the study and the details of the dataset used for training can be found in [87].

Figure 9:

Multi-scale analysis of the region-of-interest (ROI) prompted by the domain knowledge


Figure 10:

(a) Original ROI (b) Fasteners detected in the ROI, no cracks detected (c) Regions identified based on the fasteners after zooming-in into the identified regions, the cracks become more prominent thereby increasing the overall POD. Each detection in the figure has the associated confidence level with it.


While the identification of visible damage is an important aspect of monitoring the health of structures, the real challenge in most SHM applications is posed by hidden damage. This is discussed in the next section.


A Physics-informed Deep Learning (DL) Approach for Impact Diagnosis

Effective, reliable, and robust identification of impact events that have the potential to damage a structure is an aspect of SHM that is of prime importance in the aerospace industry. This is commonly referred to as impact diagnosis and its importance stems from the fact that low-velocity impact events (like tool drops, runway debris, bird strikes etc.) can lead to what is called barely-visible-impact-damage (BVID) in composites which has been found to be the most prominent cause of in-service damage to aerospace structures. The existence and/or extent of this damage can be correlated with the impact energy (which is the area underneath the impact force time-history curve), with no damage being assumed below a certain energy threshold. Impact diagnosis, which entails the identification of the impact location and reconstruction of the impact force time-history, can quantify this impact energy and in this way can serve as an indirect method for the detection of an otherwise invisible damage right at its inception [88].

Low-velocity impact on a structure emanates an elastic wave that propagates through the structure carrying a wealth of information about the impact event. Most of the classical impact diagnosis methods (both physics-based [89, 90] and data-driven [91-94]) capture information about the wave propagation phenomenon only at discrete locations by employing a limited number of sensors spatially distributed on the structure. Both of these approaches are limited in their practicality as only a limited amount of information about the scattered wavefield can be captured by the discrete sensor array. This becomes especially problematic in complex real-world structures with geometric features like joints, stiffeners, etc., where the discrete sensor signals fail to provide sufficient insights as they are often corrupted by high levels of coherent noise from multiple wave reflections, wave mode conversions etc. In order to address this issue of limited wavefield data, it was proposed to use the full wavefield for impact diagnosis. A proof-of-concept for a DL based approach for analyzing full wavefields for impact diagnosis is presented here. For this study, while simulated wavefields were employed, the feasibility of an integrated high-speed camera system for non-contact full-field wavefield capture is discussed in the next section.

The high spatio-temporal dimensionality of the wavefield mandates the use deep learning for analysis. Also, the nature of the impact diagnosis problem requires the capturing of context from the wavefield evolution which necessitates learning across multiple time-frames of the wavefield simultaneously rather than focusing independently on each frame. Fig 11 shows an end-to-end trainable CNN (convolutional neural network)–RNN (recurrent neural network) network architecture employed for the spatio-temporal analysis of the scattered wavefield. The model was trained on simulated wavefield data and care was exercised to ensure that the training data was not significantly deviated from the data the model is expected to encounter in the real world. This was assured by incorporating different noise levels and location biases in the training data and by testing the model on simulated wavefields in response to real impact force profiles generated using an impact hammer. Overall, this lies under the paradigm of letting DL models learn in a simulated environment before transferring their knowledge to the real world.

Figure 11:

A unified CNN-RNN network architecture for spatio-temporal analysis of the impact generated wavefield


Impact diagnosis clearly represents a case which is extremely data-intensive and where collecting sufficient real data is hard, as such, an ideal candidate for incorporating prior domain knowledge into the ML pipeline. Fig 12 gives an overview of the type of domain knowledge available and its integration into the machine learning pipeline. A wealth of information available about the dynamics of the wave propagation phenomenon is available in this case. In particular, Huygens’ principle, which states that different time-frames of the wavefield are not independent of each other and an inherent temporal structure exists between them, dictated the overall network architecture. This formed the basis for treating the wavefield frames/feature vectors extracted from the frames by the CNN part of the network as a sequence with the order of the individual feature vectors being meaningful/important. Also, the classical time-reversal (T-R) technique pioneered by Fink [95-97] which is usually employed for source identification in many disciplines as an optimal focusing technique formed the basis for the so-called reversed-time learning in the recurrent part of the network. Fig 13 shows some of the results for impact diagnosis and the effect of incorporating domain knowledge into the model. As can be seen, reasonable improvement, especially in impact force reconstruction accuracy is obtained by incorporating the concept of time-reversal with accuracies in the range of (89-93)% being obtained for SNR values of 20 and 10. The results obtained for SNR value 5 are expectedly poor because of the very extreme noise level.

Figure 12:

Overview of the type of domain knowledge available and its integration into the machine learning (ML) pipeline for accomplishing impact diagnosis


Figure 13:

Results for impact diagnosis including impact location and impact force reconstruction using the physics-informed DL-based methodology


So far, in Sections 2-4, the focus has been on augmenting the incomplete/insufficient training data by incorporating domain knowledge into the model during training. While this is indispensable for realizing the goal of real-time health monitoring in large/complex real-world structure, effort should also be made to develop more efficient data acquisition systems for SHM. To this end, the next Section presents a non-contact sensing paradigm for full-field measurements for vision-based SHM.


A Non-contact Full-field Sensing Paradigm for Vision-based SHM

Computer vision techniques for automated damage assessment have been engendering a lot of interest in recent times in civil infrastructures, especially during post-disaster inspections, when the number of structures to be inspected is far beyond the capability of available inspectors [98-103]. In the field of civil infrastructure assessment, safety-critical damage is usually big enough to be visible to the naked eye, as such, static images captured using ordinary digital cameras suffice for the purpose of automated damage assessment using state-of-the-art computer vision algorithms. For instance, a fusion CNN was proposed by Dr. Li’s group [104-105] to successfully detect distributed cracks on steel girders of actual bridges. However, for aerospace structures, the most critical type of damage is barely visible to the naked eye. In such (plate-like) structures, hidden details of the structure, including information about the location of the damage, if present, and its characteristics can be unearthed using ultrasonic guided wave based techniques, which entail investigating the scattered guided waves in the structure using appropriate signal/image processing algorithms. These scattered waves can be sensed using either contact (e.g., a network of piezoelectric (PZT) sensor arrays) [106] or fully non-contact means (e.g., air-coupled transducers, laser Doppler vibrometer (LDV)) [107-112]. Because of the multitude of advantages associated with non-contact full-field sensing, the scanning LDV has attracted a lot of interest in recent times, however, it is limited to point-by-point measurement/sensing (i.e., one point at a time) [113].

In order to overcome these limitations, a sensing paradigm capable of taking all the measurements simultaneously is desirable. Digital image correlation (DIC), which is a non-interferometric optical technique, extracts surface deformations by comparing digital images of the structure’s surface captured before and after deformation [114-118]. Advances in computers and digital imaging technology have already enabled 2-D/3-D DIC methods to extend from static or quasi-static to dynamic applications, such as low-frequency vibration (in the range of kHz) and full-field modal analysis for characterizing the behavior of structures [119-121]. In order to expand DIC applications into the ultrasonic range (> 20 kHz), for example, in the area of ultrasonic guided-wave based damage imaging, image acquisition systems with extremely high spatial/temporal resolution are required.

High-speed digital cameras, even though not there yet, provide the most viable means for accomplishing this. Not only do they enable simultaneous data acquisition at all the points in the region-of-interest (ROI) but the reconstructed wavefield using the sensed data provides the most spatially continuous information about the wave propagation phenomena as each pixel is considered to effectively act as a sensor. The requirement of extremely high spatial/temporal resolution, however, continues to remain a bottleneck in the adoption of this technology for ultrasonic guided wavefield reconstruction. However, the problem is purely related to the processing speed and will improve with the development of computer technology.

Furthermore, as IC technology rapidly advances following Moore’s law, new high-speed cameras promise to have high potential for capturing the propagating guided wave on the surface of the structures.

Fig. 14 shows the schematic of a proof-of-concept for an integrated high-speed camera system for non-contact full-field sensing of the scattered wavefield in structures [122] and the damage image generated from the captured wavefield using modified wavenumber index (WI) imaging condition [123]. It should be noted that since only a single high-speed camera was used, it was possible to capture only the in-plane scattered wavefield. In order to circumvent the current hardware limitations associated with using a single high-speed camera for the acquisition of ultrasonic guided in-plane wavefields, three strategies were adopted. First, for improving the spatial/temporal resolution, sample interleaving and image stitching techniques were used to get an effective data acquisition rate that was nearly 250 times that of the original rate specified by the camera. Second, the concept was demonstrated on a thin, flat, low-modulus, high-density polyethylene (HDPE) plate, which is a flexible polymer sheet with less than one percent of the stiffness of regular carbon fiber reinforced polymers (CFRPs). The low-modulus and high-density plate enables large deformations and low wave speeds, which make it possible to successfully detect the in-plane wavefields using the affordable in-house high-speed camera (Photron Fastcam Mini AX200). Third, a large PZT actuator was used in conjunction with a power amplifier to generate sufficient energy to excite measurable guided waves in the plate, and the excitation frequency was carefully chosen to be a continuous harmonic wave such that not only the fundamental A0 and S0 modes exist, but a strong fundamental SH0 wave mode also persists (due to multiple reflections from the damage and plate boundaries).

Figure 14:

(a) Schematic of the experimental setup for non-contact full-field sensing (b) Damage image of a BVID generated from the captured guided wavefield using modified wavenumber index (WI) imaging condition


While currently the concept could only be demonstrated on the HDPE sheet due to hardware limitations associated with using a single high-speed camera, the study shows significant initial promise for the adoption of this technology for non-contact full-field measurements with the ultimate goal of visualizing hidden damage such as barely visible impact damage (BVID) in complex flat/curved composite structures. In order to continue development of this concept past this successful feasibility study and begin optimizing it for practical applications, the following tasks are planned. (1) Multiple high-speed cameras will be employed to enable the capture of three-dimensional displacement/strain scattered wavefields with larger fields of view. It is envisioned that out-of-plane anti-symmetric wave modes (such as the fundamental A0 wave mode) will provide significantly higher SNR values to yield better damage images. (2) Physics-based image stitching techniques will be explored in order to handle more-complicated geometries and damage scenarios. (3) For using the captured full wavefields in conjunction with physics-informed learning for realizing the eventual goal of developing an end-to-end non-contact vision based damage detection system in SHM, other transfer learning models like the one developed in Section 4.2 will be developed.


Summary, Conclusions and Future Prospects

When the structures being monitored (either continuously or intermittently) are significantly large/complex and/or when the variations in environmental/operational conditions cannot be neglected, a physics-based approach for assessing the structural health becomes highly unreliable. In recent times, the number of parameters being monitored in-situ in real-world structures has been steadily increasing through the use of miniaturized wireless sensor networks and that has led to an abundance of historical data for these structures. The heterogeneous nature of this data (due to multiple sensing modalities) and the emphasis on developing long-term SHM systems has engendered a lot of interest in a data-driven approach for SHM. However, for these real-world structures, it is highly impractical to collect sufficient data (corresponding to different damage scenarios) to train a robust and generalizable model for automated damage detection and characterization. Consequently, an optimal approach to SHM must encompass, on one hand, some means of augmenting the collected sensor data in order to compensate for the lack of completeness in the data and on the other hand comprehensive sensing mechanisms with a framework for subsequent data fusion and processing of the large volumes of collected data.

In this paper, these aspects of such an optimal SHM paradigm were addressed individually through: (1) The use of physics-informed machine learning for the analysis of the measured sensor data. (2) The use of high-speed digital cameras for capturing ultrasonic guided wavefields in structures as a means of providing the most spatially continuous information about the propagating ultrasonic guided waves within the structure. Owing to the attention it has been engendering in recent times, physics-informed machine learning was discussed in some detail with the eventual goal of SHM in mind. By demonstrating the applicability of ANNs for solving forward and inverse problems involving PDEs, initial promise was demonstrated towards solving more complicated inverse problems for the task of automated damage detection in SHM. It was shown how the overfitting/generalization issue generally associated with the insufficient nature of the training data can be mitigated by incorporating physical laws/domain knowledge to guide the back-propagation during the training process. Other ways of incorporating domain knowledge into the ML/DL pipeline were also discussed through the case studies related to different aspects of SHM i.e., enhanced visual inspection and impact diagnosis.

Internet of Things (IoT) has recently attracted great attention due to its promising potential and capacity to be integrated into complex systems [124, 125]. With rapid advances in information and sensing technologies such as wireless communication, cloud computing, and wireless sensor networks (WSNs), IoT is emerging as an enabling technology for the development of low cost, efficient, reliable, and scalable SHM systems. Recent advances in deep learning (both supervised and unsupervised) and breakthroughs in the integration of system physics/domain knowledge into the ML/DL models have had a catalytic effect on the adoption of a physics-informed data-driven approach to learning in many application domains. It is envisioned that as robust non-contact full-field sensing mechanisms are developed and more success in physics-informed learning is achieved, especially for solving complex inverse problems in SHM, the goal of in-situ real-time vision based health monitoring can be achieved in near future.


One of the authors, Mr. Sakib A. Zargar would like to express their sincere thanks to Dr. Jeremy Yagle of NASA Langley Research Center for the support through National Institute of Aerospace (NIA).



Giurgiutiu, V., Structural health monitoring: with piezoelectric wafer active sensors, Elsevier.2007). Google Scholar


Structural health monitoring (SHM) in aerospace structures, Woodhead Publishing.2016). Google Scholar


Cawley, P., “Structural health monitoring: Closing the gap between research and industrial deployment.,” Structural Health Monitoring, 17 (5), 1225 –1244 (2018). Google Scholar


Doebling, S. W., Farrar, C. R., & Prime, M. B., “A summary review of vibration-based damage identification methods.,” Shock and vibration digest, 30 (2), 91 –105 (1998). Google Scholar


Farrar, C. R., Doebling, S. W., & Nix, D. A., “Vibration–based structural damage identification.,” Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, 359 (1778), 131 –149 (2001). Google Scholar


Alvandi, A., & Cremona, C., “Assessment of vibration-based damage identification techniques.,” Journal of sound and vibration, 292 (1-2), 179 –202 (2006). Google Scholar


Montalvao, D., Maia, N. M. M., & Ribeiro, A. M. R., “A review of vibration-based structural health monitoring with special emphasis on composite materials.,” Shock and vibration digest, 38 (4), 295 –324 (2006). Google Scholar


Fan, W., & Qiao, P., “Vibration-based damage identification methods: a review and comparative study.,” Structural health monitoring, 10 (1), 83 –111 (2011). Google Scholar


Raghavan, A., & Cesnik C. E. S., “Review of guided-wave structural health monitoring,” The Shock and Vibration Digest, 39 (2), 91 –114 (2007). Google Scholar


Su, Z., & Ye, L., Identification of damage using Lamb waves: from fundamentals to applications, 48 Springer Science & Business Media.2009). Google Scholar


Mitra, M., & Gopalakrishnan, S., “Guided wave based structural health monitoring: A review.,” Smart Materials and Structures, 25 (5), 053001 (2016). Google Scholar


Wu, X., Ghaboussi, J., & Garrett Jr, J. H., “Use of neural networks in detection of structural damage.,” Computers & structures, 42 (4), 649 –659 (1992). Google Scholar


Worden, K., & Manson, G., “The application of machine learning to structural health monitoring.,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 365 (1851), 515 –537 (2007). Google Scholar


Farrar, C. R., & Worden, K., Structural health monitoring: a machine learning perspective, John Wiley & Sons.2012). Google Scholar


Rytter, A., Vibrational based inspection of civil engineering structures, (1993). Google Scholar


Allemang, R. J., & Brown, D. L., “A correlation coefficient for modal vector analysis,” in Proceedings of the 1st international modal analysis conference, 110 –116 (1982). Google Scholar


Allemang, R. J., “The modal assurance criterion–twenty years of use and abuse.,” Sound and vibration, 37 (8), 14 –23 (2003). Google Scholar


Kim, J. T., Ryu, Y. S., Cho, H. M., & Stubbs, N., “Damage identification in beam-type structures: frequency-based method vs mode-shape-based method.,” Engineering structures, 25 (1), 57 –67 (2003). Google Scholar


Shi, Z., Law, S. S., & Zhang, L., “Structural damage detection from modal strain energy change.,” Journal of engineering mechanics, 126 (12), 1216 –1223 (2000). Google Scholar


Pandey, A. K., Biswas, M., & Samman, M. M., “Damage detection from changes in curvature mode shapes.,” Journal of sound and vibration, 145 (2), 321 –332 (1991). Google Scholar


Wahab, M. A., & De Roeck, G., “Damage detection in bridges using modal curvatures: application to a real damage scenario.,” Journal of Sound and vibration, 226 (2), 217 –235 (1999). Google Scholar


Pandey, A. K., & Biswas, M., “Damage detection in structures using changes in flexibility.,” Journal of sound and vibration, 169 (1), 3 –17 (1994). Google Scholar


Pandey, A. K., & Biswas, M., “Damage diagnosis of truss structures by estimation of flexibility change.,” Modal Analysis-the International Journal of Analytical and Experimental Modal Analysis, 10 (2), 104 –117 (1995). Google Scholar


Jaishi, B., & Ren, W. X., “Damage detection by finite element model updating using modal flexibility residual.,” Journal of sound and vibration, 290 (1-2), 369 –387 (2006). Google Scholar


Bernal, D., “Load vectors for damage localization.,” Journal of Engineering Mechanics, 128 (1), 7 –14 (2002). Google Scholar


Staszewski, W. J., Worden, K., & Tomlinson, G. R., “Time–frequency analysis in gearbox fault detection using the Wigner–Ville distribution and pattern recognition.,” Mechanical systems and signal processing, 11 (5), 673 –692 (1997). Google Scholar


Sohn, H., Park, G., Wait, J. R., Limback, N. P., & Farrar, C. R., “Wavelet-based active sensing for delamination detection in composite structures.,” Smart Materials and structures, 13 (1), 153 (2003). Google Scholar


Ovanesova, A. V., & Suarez, L. E., “Applications of wavelet transforms to damage detection in frame structures.,” Engineering structures, 26 (1), 39 –49 (2004). Google Scholar


Kim, H., & Melhem. H., “Damage detection of structures by wavelet analysis,” Engineering structures, 26 (3), 347 –362 (2004). Google Scholar


Taha, M. R., Noureldin, A., Lucero, J. L., & Baca, T. J., “Wavelet transform for structural health monitoring: a compendium of uses and features.,” Structural health monitoring, 5 (3), 267 –295 (2006). Google Scholar


Yang, J. N., Lei, Y., Lin, S., & Huang, N., “Hilbert-Huang based approach for structural damage detection.,” Journal of engineering mechanics, 130 (1), 85 –95 (2004). Google Scholar


Hay, T. R., Royer, R. L., Gao, H., Zhao, X., & Rose, J. L., “A comparison of embedded sensor Lamb wave ultrasonic tomography approaches for material loss detection.,” Smart materials and structures, 15 (4), 946 (2006). Google Scholar


He, J., & Yuan, F. G., “Lamb-wave-based two-dimensional areal scan damage imaging using reverse-time migration with a normalized zero-lag cross-correlation imaging condition.,” Structural Health Monitoring, 16 (4), 444 –457 (2017). Google Scholar


Kudva, J. N., Munir, N., & Tan, P. W., “Damage detection in smart structures using neural networks and finite-element analyses.,” Smart Materials and Structures, 1 (2), 108 (1992). Google Scholar


Teboub, Y., & Hajela, P., “A neural network based damage analysis of smart composite beams,” in 4th Symposium on Multidisciplinary Analysis and Optimization, 4685 (1992). Google Scholar


Mehrjoo, M., Khaji, N., Moharrami, H., & Bahreininejad, A., “Damage detection of truss bridge joints using Artificial Neural Networks.,” Expert Systems with Applications, 35 (3), 1122 –1131 (2008). Google Scholar


Liu, Y. Y., Ju, Y. F., Duan, C. D., & Zhao, X. F., “Structure damage diagnosis using neural network and feature fusion.,” Engineering applications of artificial intelligence, 24 (1), 87 –92 (2011). Google Scholar


Hinton, G. E., Osindero, S., & Teh, Y. W., “A fast learning algorithm for deep belief nets,” Neural computation, 18 (7), 1527 –1554 (2006). Google Scholar


LeCun, Y., Bengio, Y., & Hinton, G., “Deep learning.,” Nature, 521 (7553), 436 –444 (2015). Google Scholar


Schmidhuber, J., “Deep learning in neural networks: An overview.,” Neural networks, 61 85 –117 (2015). Google Scholar


Sze, V., Chen, Y. H., Yang, T. Y., & Emer, J. S., “Efficient processing of deep neural networks: A tutorial and survey.,” Proceedings of the IEEE, 105 (12), 2295 –2329 (2017). Google Scholar


Diez, A., Khoa, N. L. D., Alamdari, M. M., Wang, Y., Chen, F., & Runcie, P., “A clustering approach for structural health monitoring on bridges,” Journal of Civil Structural Health Monitoring, 6 (3), 429 –445 (2016). Google Scholar


Alves, V., Cury, A., Roitman, N., Magluta, C., & Cremona, C., “Novelty detection for SHM using raw acceleration measurements.,” Structural Control and Health Monitoring, 22 (9), 1193 –1207 (2015). Google Scholar


Toivola, J., Prada, M. A., & Hollmén, J., “Novelty detection in projected spaces for structural health monitoring. In,” International Symposium on Intelligent Data Analysis, 208 –219 Springer, Berlin, Heidelberg (2010). Google Scholar


Von Rueden, L., Mayer, S., Garcke, J., Bauckhage, C., & Schuecker, J., “Informed machine learning–towards a taxonomy of explicit integration of knowledge into machine learning.,” Learning, 18 (19), 20 (2019). Google Scholar


Kramer, M. A., “Nonlinear principal component analysis using autoassociative neural networks,” AIChE journal, 37 (2), 233 –243 (1991). Google Scholar


Mavrovouniotis, M. L., & Chang, S., “Hierarchical neural networks.,” Computers & chemical engineering, 16 (4), 347 –369 (1992). Google Scholar


Tan, S., & Mayrovouniotis, M. L., “Reducing data dimensionality through optimizing neural network inputs.,” AIChE Journal, 41 (6), 1471 –1480 (1995). Google Scholar


Yun, C. B., & Bahng, E. Y., “Substructural identification using neural networks.,” Computers & Structures, 77 (1), 41 –52 (2000). Google Scholar


Marwala, T., “Damage identification using committee of neural networks.,” Journal of Engineering Mechanics, 126 (1), 43 –50 (2000). Google Scholar


Chen, S. L., & Jen, Y. W., “Data fusion neural network for tool condition monitoring in CNC milling machining.,” International journal of machine tools and manufacture, 40 (3), 381 –400 (2000). Google Scholar


Zang, C., & Imregun, M., “Structural damage detection using artificial neural networks and measured FRF data reduced via principal component projection.,” Journal of sound and vibration, 242 (5), 813 –827 (2001). Google Scholar


Ghosh, N., Ravi, Y. B., Patra, A., Mukhopadhyay, S., Paul, S., Mohanty, A. R., & Chattopadhyay, A. B., “Estimation of tool wear during CNC milling using neural network-based sensor fusion.,” Mechanical Systems and Signal Processing, 21 (1), 466 –479 (2007). Google Scholar


Kramer, M. A., “Autoassociative neural networks,” Computers & chemical engineering, 16 (4), 313 –328 (1992). Google Scholar


Zhang, J., Walter, G. G., Miao, Y., & Lee, W. N. W., “Wavelet neural networks for function learning,” IEEE transactions on Signal Processing, 43 (6), 1485 –1497 (1995). Google Scholar


Wang, F., & Zhang, Q. J., “Knowledge-based neural models for microwave design,” IEEE Transactions on Microwave Theory and Techniques, 45 (12), 2333 –2343 (1997). Google Scholar


Jiang, X., & Adeli, H., “Pseudospectra, MUSIC, and dynamic wavelet neural network for damage detection of highrise buildings,” International Journal for Numerical Methods in Engineering, 71 (5), 606 –629 (2007). Google Scholar


Psichogios, D. C., & Ungar, L. H., “A hybrid neural network-first principles approach to process modeling,” AIChE Journal, 38 (10), 1499 –1511 (1992). Google Scholar


Kramer, M. A., Thompson, M. L., & Bhagat, P. M., “Embedding theoretical models in neural networks,” in In 1992 American Control Conference, 475 –479 (1992). Google Scholar


Thompson, M. L., & Kramer, M. A., “Modeling chemical processes using prior knowledge and neural networks,” AIChE Journal, 40 (8), 1328 –1340 (1994). Google Scholar


Jain, A. K., Mao, J., & Mohiuddin, K. M., “Artificial neural networks: A tutorial.,” Computer, 29 (3), 31 –44 (1996). Google Scholar


Hornik, K., Stinchcombe, M., & White, H., “Multilayer feedforward networks are universal approximators.,” Neural networks, 2 (5), 359 –366 (1989). Google Scholar


Dissanayake, M. W. M. G., & Phan-Thien, N., “Neural-network-based approximations for solving partial differential equations.,” communications in Numerical Methods in Engineering, 10 (3), 195 –201 (1994). Google Scholar


Lagaris, I. E., Likas, A., & Fotiadis, D. I., “Artificial neural networks for solving ordinary and partial differential equations.,” IEEE transactions on neural networks, 9 (5), 987 –1000 (1998). Google Scholar


Lagaris, I. E., Likas, A. C., & Papageorgiou, D. G., “Neural-network methods for boundary value problems with irregular boundaries.,” IEEE Transactions on Neural Networks, 11 (5), 1041 –1049 (2000). Google Scholar


He, S., Reif, K., & Unbehauen, R., “Multilayer neural networks for solving a class of partial differential equations.,” Neural networks, 13 (3), 385 –396 (2000). Google Scholar


Aarts, L. P., & Van Der Veer, P., “Neural network method for solving partial differential equations.,” Neural Processing Letters, 14 (3), 261 –271 (2001). Google Scholar


Alli, H., Uçar, A., & Demir, Y., “The solutions of vibration control problems using artificial neural networks.,” Journal of the Franklin Institute, 340 (5), 307 –325 (2003). Google Scholar


Malek, A., & Beidokhti, R. S., “Numerical solution for high order differential equations using a hybrid neural network—optimization method.,” Applied Mathematics and Computation, 183 (1), 260 –271 (2006). Google Scholar


Beidokhti, R. S., & Malek, A., “Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques.,” Journal of the Franklin Institute, 346 (9), 898 –913 (2009). Google Scholar


Shirvany, Y., Hayati, M., & Moradian, R., “Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations.,” Applied Soft Computing, 9 (1), 20 –29 (2009). Google Scholar


Raissi, M., Perdikaris, P., & Karniadakis, G. E., “Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations.,” arXiv preprint arXiv:1711.10561, (2017). Google Scholar


Raissi, M., Perdikaris, P., & Karniadakis, G. E., “Physics informed deep learning (part II): Data-driven discovery of nonlinear partial differential equations. arXiv.,” arXiv preprint arXiv:1711.10561, (2017). Google Scholar


Raissi, M., Perdikaris, P., & Karniadakis, G. E., “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.,” Journal of Computational Physics, 378 686 –707 (2019). Google Scholar


Sirignano, J., & Spiliopoulos, K., “DGM: A deep learning algorithm for solving partial differential equations.,” Journal of Computational Physics, 375 1339 –1364 (2018). Google Scholar


Anitescu, C., Atroshchenko, E., Alajlan, N., & Rabczuk, T., “Artificial neural network methods for the solution of second order boundary value problems.,” Computers, Materials & Continua, 59 (1), 345 –359 (2019). Google Scholar


Berg, J., & Nyström, K., “A unified deep artificial neural network approach to partial differential equations in complex geometries.,” Neurocomputing, 317 28 –41 (2018). Google Scholar


Meng, X., & Karniadakis, G. E., “A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems.,” Journal of Computational Physics, 401 109020 (2020). Google Scholar


Zhu, Y., Zabaras, N., Koutsourelakis, P. S., & Perdikaris, P., “Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data.,” Journal of Computational Physics, 394 56 –81 (2019). Google Scholar


Lutter, M., Ritter, C., & Peters, J., “Deep Lagrangian networks: Using physics as model prior for deep learning.,” arXiv preprint arXiv:1907.04490, (2019). Google Scholar


Federal Aviation Administration, Visual inspection for aircraft, U.S. Department of Transportation, Advisory circular [online],”–204.pdf Google Scholar


Drury, C. G., & Watson, J., “Good practices in visual inspection.,” Human factors in aviation maintenance-phase nine, progress report, FAA/Human Factors in Aviation Maintenance.@, (2002) Google Scholar


Swain, A. D., & Guttmann, H. E., Handbook of human-reliability analysis with emphasis on nuclear power plant applications. Final report, Sandia National Labs.1983). Google Scholar


See, J. E., “Visual inspection reliability for precision manufactured parts.,” Human factors, 57 (8), 1427 –1442 (2015). Google Scholar


See, J. E., “Visual inspection: a review of the literature.,” Sandia National Laboratories, Albuquerque, New Mexico, (2012). Google Scholar


Azuma, R. T., “A survey of augmented reality.,” Presence: Teleoperators & Virtual Environments, 6 (4), 355 –385 (1997). Google Scholar


Wang, S., Zargar, S. A., Xu, C. & Yuan, F. G., “An Efficient Augmented Reality (AR) System for Enhanced Visual Inspection.,” 1575 –1582 (2019). Google Scholar


Zargar, S. A. & Yuan, F. G., “Impact Diagnosis in Stiffened Structural Panels Using a Deep Learning Approach,” Structural Health Monitoring Journal, Accepted, (2020). Google Scholar


Inoue, H., Harrigan, J. J., & Reid, S. R., “Review of inverse analysis for indirect measurement of impact force.,” Applied Mechanics Reviews, 54 (6), 503 –524 (2001). Google Scholar


Chen, C., Li, Y., & Yuan, F. G., “Impact source identification in finite isotropic plates using a time-reversal method: experimental study.,” Smart Materials and Structures, 21 (10), 105025 (2012). Google Scholar


Worden, K., & Staszewski, W. J., “Impact location and quantification on a composite panel using neural networks and a genetic algorithm.,” Strain, 36 (2), 61 –68 (2000). Google Scholar


Haywood, J., Coverley, P. T., Staszewski, W. J., & Worden, K., “An automatic impact monitor for a composite panel employing smart sensor technology.,” Smart Materials and Structures, 14 (1), 265 (2004). Google Scholar


Ghajari, M., Sharif-Khodaei, Z., Aliabadi, M. H., & Apicella, A., “Identification of impact force for smart composite stiffened panels.,” Smart Materials and Structures, 22 (8), 085014 (2013). Google Scholar


Seno, A. H., & Aliabadi, M. H., “Impact localisation in composite plates of different stiffness impactors under simulated environmental and operational conditions.,” Sensors, 19 (17), 3659 (2019). Google Scholar


Fink, M., “Time reversal of ultrasonic fields. I. Basic principles.,” IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 39 (5), 555 –566 (1992). Google Scholar


Wu, F., Thomas, J. L., & Fink, M., “Time reversal of ultrasonic fields. Il. Experimental results.,” IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 39 (5), 567 –578 (1992). Google Scholar


Cassereau, D., & Fink, M., “Time-reversal of ultrasonic fields. III. Theory of the closed time-reversal cavity.,” IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 39 (5), 579 –592 (1992). Google Scholar


Yeum, C. M., & Dyke, S. J., “Vision-based automated crack detection for bridge inspection,” Computer-Aided Civil and Infrastructure Engineering, 30 (10), 759 –770 (2015). Google Scholar


Feng, D., & Feng, M. Q., “Experimental validation of cost-effective vision-based structural health monitoring,” Mechanical Systems and Signal Processing, 88 199 –211 (2017). Google Scholar


Hoskere, V., Narazaki, Y., Hoang, T., & Spencer Jr, B., “Vision-based structural inspection using multiscale deep convolutional neural networks,” arXiv preprint arXiv:1805.01055, (2018). Google Scholar


Narazaki, Y., Hoskere, V., Hoang, T. A., & Spencer Jr, B. F., “Automated vision-based bridge component extraction using multiscale convolutional neural networks,” arXiv preprint arXiv:1805.06042, (2018). Google Scholar


Spencer Jr, B. F., Hoskere, V., & Narazaki, Y., “Advances in computer vision-based civil infrastructure inspection and monitoring,” Engineering, (2019). Google Scholar


Bao, Y., Chen, Z., Wei, S., Xu, Y., Tang, Z., & Li, H., “The State of the Art of Data Science and Engineering in Structural Health Monitoring,” Engineering, 5 (2), 234 –242 (2019). Google Scholar


Xu, Y., Bao, Y., Chen, J., Zuo, W., & Li, H., “Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images,” Structural Health Monitoring, 18 (3), 653 –674 (2019). Google Scholar


Wei, S., Bao, Y., & Li, H., “Optimal policy for structure maintenance: A deep reinforcement learning framework,” Structural Safety, 83 101906 (2020). Google Scholar


V. Giurgiutiu, “Tuned Lamb Wave Excitation and Detection with Piezoelectric Wafer Active Sensors for Structural Health Monitoring,” Journal of Intelligent Material Systems and Structures, 16 (4), 291 –305 (2005). Google Scholar


Z. Tian, L. Yu, and C. Leckey, “Delamination Detection and Quantification on Laminated Composite Structures with Lamb Waves and Wavenumber Analysis,” Journal of Intelligent Material Systems and Structures, 26 (13), 1723 –1738 (2015). Google Scholar


B. Park, Y. K. An, and H. Sohn, “Visualization of Hidden Delamination and Debonding in Composites through Noncontact Laser Ultrasonic Scanning,” Composites Science and Technology, 100 10 –18 (2014). Google Scholar


J. He and F. G. Yuan, “Lamb Wave-based Subwavelength Damage Imaging using the DORT-MUSIC Technique in Metallic Plates,” Structural Health Monitoring, 15 (1), 65 –80 (2016). Google Scholar


J. He and F. G. Yuan, “Damage Identification for Composite Structures using a Cross-correlation Reverse-time Migration Technique,” Structural Health Monitoring, 14 (6), 558 –570 (2015). Google Scholar


M. S. Harb and F. G. Yuan, “Damage Imaging using Non-contact Air-coupled Transducer/Laser Doppler Vibrometer System,” Structural Health Monitoring, 15 (2), 193 –203 (2016). Google Scholar


E. B. Flynn, S. Y. Chong, G. J. Jarmer, and J. R. Lee, “Structural Imaging through Local Wavenumber Estimation of Guided Waves,” Ndt & E International, 59 1 –10 (2013). Google Scholar


Reu, P. L., Rohe, D. P., & Jacobs, L. D., “Comparison of DIC and LDV for practical vibration and modal measurements.,” Mechanical Systems and Signal Processing, 86 2 –16 (2017). Google Scholar


Chu, T. C., Ranson, W. F., & Sutton, M. A., “Applications of digital-image-correlation techniques to experimental mechanics.,” Experimental mechanics, 25 (3), 232 –244 (1985). Google Scholar


Luo, P. F., Chao, Y. J., Sutton, M. A., & Peters, W. H., “Accurate measurement of three-dimensional deformations in deformable and rigid bodies using computer vision.,” Experimental mechanics, 33 (2), 123 –132 (1993). Google Scholar


Luo, P. F., Chao, Y. J., & Sutton, M. A., “Application of stereo vision to three-dimensional deformation analyses in fracture experiments.,” Optical Engineering, 33 (3), 981 –991 (1994). Google Scholar


Sutton, M. A., Matta, F., Rizos, D., Ghorbani, R., Rajan, S., Mollenhauer, D. H., … & Lasprilla, A. O., “Recent progress in digital image correlation: background and developments since the 2013 WM Murray Lecture.,” Experimental Mechanics, 57 (1), 1 –30 (2017). Google Scholar


Pan, B., “Digital image correlation for surface deformation measurement: historical developments, recent advances and future goals.,” Measurement Science and Technology, 29 (8), 082001 (2018). Google Scholar


Helfrick, M. N., Niezrecki, C., Avitabile, P., & Schmidt, T., “3D digital image correlation methods for full-field vibration measurement.,” Mechanical systems and signal processing, 25 (3), 917 –927 (2011). Google Scholar


Trebuňa, F., & Hagara, M., “Experimental modal analysis performed by high-speed digital image correlation system.,” Measurement, 50 78 –85 (2014). Google Scholar


Beberniss, T. J., & Ehrhardt, D. A., “High-speed 3D digital image correlation vibration measurement: Recent advancements and noted limitations.,” Mechanical Systems and Signal Processing, 86 35 –48 (2017). Google Scholar


Chang, H. Y., Damage Visualization of Scattered Ultrasonic Wavefield via Integrated High-speed Camera System, (2019). Google Scholar


H. Y. Chang and F. G. Yuan, “Damage Imaging in a Stiffened Curved Composite Sandwich Panel with Wavenumber Index via Riesz Transform,” Structural Health Monitoring, 1475921719858432 (2019). Google Scholar


Tokognon, C. A., Gao, B., Tian, G. Y., & Yan, Y., “Structural health monitoring framework based on Internet of Things: A survey,” IEEE Internet of Things Journal, 4 (3), 619 –635 (2017). Google Scholar


Abdelgawad, A., & Yelamarthi, K., “Internet of things (IoT) platform for structure health monitoring,” Wireless Communications and Mobile Computing, (2017). Google Scholar
© (2020) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Fuh-Gwo Yuan, Sakib Ashraf Zargar, Qiuyi Chen, and Shaohan Wang "Machine learning for structural health monitoring: challenges and opportunities", Proc. SPIE 11379, Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2020, 1137903 (23 April 2020);

Back to Top