Translator Disclaimer
Open Access Presentation + Paper
23 April 2020 Machine learning for structural health monitoring: challenges and opportunities
Author Affiliations +
Abstract
A physics-based approach to structural health monitoring (SHM) has practical shortcomings which restrict its suitability to simple structures under well controlled environments. With the advances in information and sensing technology (sensors and sensor networks), it has become feasible to monitor large/diverse number of parameters in complex real-world structures either continuously or intermittently by employing large in-situ (wireless) sensor networks. The availability of this historical data has engendered a lot of interest in a data-driven approach as a natural and more viable option for realizing the goal of SHM in such structures. However, the lack of sensor data corresponding to different damage scenarios continues to remain a challenge. Most of the supervised machine-learning/deep-learning techniques, when trained using this inherently limited data, lack robustness and generalizability. Physics-informed learning, which involves the integration of domain knowledge into the learning process, is presented here as a potential remedy to this challenge. As a step towards the goal of automated damage detection (mathematically an inverse problem), preliminary results are presented from dynamic modelling of beam structures using physics-informed artificial neural networks. Forward and inverse problems involving partial differential equations are solved and comparisons reveal a clear superiority of physics-informed approach over one that is purely datadriven vis-à-vis overfitting/generalization. Other ways of incorporating domain knowledge into the machine learning pipeline are then presented through case-studies on various aspects of NDI/SHM (visual inspection, impact diagnosis). Lastly, as the final attribute of an optimal SHM approach, a sensing paradigm for non-contact full-field measurements for damage diagnosis is presented.
Conference Presentation

1.

Introduction

Structural health monitoring (SHM) is a transdisciplinary field of engineering devoted to ensuring the structural integrity and operational safety of a component or structure. This is achieved by facilitating the (in-service) detection and characterization of damage in a structure that may adversely affect its ability to fully and safely perform its intended function. As such, the problem of damage detection lies at the heart of SHM and the goal is to identify the damage at the earliest possible stage (in near real-time). This is indispensable for ensuring timely corrective action in order to minimize the system downtime, overall operational and maintenance costs, and to reduce the risk of catastrophic failure. Broadly speaking, as can be seen in Fig. 1, two approaches have traditionally been adopted for automated damage detection in SHM: physics-based and data-driven.

Figure 1:

The big picture of structural health monitoring (SHM) indicating that as the complexity of the structure increases, exclusive reliance on the system physics for analyzing the measured sensor data is reduced, and an approach that is primarily data-driven is adopted. An optimal approach for large/complex real-world structures must be data-driven with the system-physics/domain knowledge embedded in some form.

00102_psisdg11379_1137903_page_3_1.jpg

Physics-based methods, in some form or the other, rely primarily on the physical laws governing the structural behavior in order to extract meaningful information about the damage and its evolution from the measured sensor data. However, difficulty in the modelling of complex real-world structures, considerations of multiple sensing modalities, material and/or geometric non-linearity, and uncertainty in material properties, boundary conditions, environmental/operational variations are some of the factors that make this exclusive reliance on the system physics rather impractical in case of complex real-world structures. This limits the application of such methods to the health monitoring of rather simple structures with pre-defined boundary conditions and well-controlled environments [1-3]. As the underlying system complexity increases, such an approach becomes much less dependable. Owing to the advances in information and sensing technologies in recent times, it has now become feasible to monitor a large number of parameters in-situ in large/complex real-world structures on either a continuous or sporadic basis. This motivates the use of a data-driven approach for SHM where-in damage assessment is dealt with, at least at a lower level, primarily as being a type of statistical pattern recognition problem thus circumventing some of the major challenges associated with the physics-based approach.

During the last few decades, machine learning (ML) techniques have been extensively employed by researchers both for vibration based and ultrasonic guided wave based damage detection [4-14]. Machine learning in SHM aims at building models or representations for mapping input patterns in measured sensor data to output targets for damage assessment at different levels, Rytter [15]. Conventional machine learning techniques are however limited in their ability to process the large amounts of measured sensor data in their raw form. As such, careful engineering and considerable domain knowledge are required to extract damage-sensitive features from the raw data which are then fed into a suitable ML model. Historically, the choice of these hand-crafted damage-sensitive features has stemmed from the wealth of well-developed literature on physics-based SHM techniques including but not limited to modal assurance criterion (MAC) and Coordinate MAC [16, 17], modal strain energy (MSE) [18, 19], modal curvature (MC) [20, 21], modal flexibility (MF) [22-24], damage locating vector (DLV) [25], wavelet transform [26-30], Hilbert-Huang transform [31], probabilistic reconstruction algorithm (PRA) [32], ZLCC [33] etc,. Multi-layer feedforward artificial neural networks (ANNs), also referred to as multi-layer perceptrons (MLPs), have been the most well-known and widely used ML technique for automated damage detection [34-37]. Apart from the algorithm itself, the overall performance of such a damage detection system depends on the choice of the damage-sensitive features used. The problem with hand-crafted features is that while they may be sub-optimal even for the structure under consideration, there is no guarantee that the same set of features can be adopted for other structures. In order to overcome this shortcoming, deep learning (DL) methods have attracted much attention in recent years. These allow data to be used in their raw form and features are learned automatically from data using a general-purpose learning procedure. Deep learning automatically discovers intricate features in high-dimensional data which has led to its widespread adoption across many application domains [38-41]. It is certain that DL will continue to flourish in the future and the progress will only be accelerated with advances in computational capabilities through the development of high-tech central processing units (CPUs)/graphics processing units (GPUs), availability of large amounts of data (big data), and development of new learning algorithms.

While supervised learning is the most common and well developed form of learning, a fundamental challenge in many SHM applications is that damage detection must often be performed in an unsupervised manner. This is because, for real-world structures, it is highly unlikely to have data corresponding to different damage scenarios. Among the four levels of damage assessment identified by Rytter [15], the lowest level (i.e., establishing the presence of damage) has been achieved in the past through unsupervised learning by what is referred to as novelty (outlier) detection [42-44]. Higher levels of damage assessment in real-world structures require either a mechanism for augmenting the insufficient/incomplete training data by incorporating some form of prior knowledge into the learning/training process and/or the development of more comprehensive data acquisition systems. These will henceforth be referred to as the two aspects of an optimal SHM system. This paper introduces these two aspects through a discussion on physics-informed learning in Sections 2, 3 and 4 and non-contact full-field measurements using state-of-the-art high-speed digital cameras for vision-based SHM in Section 5. Summary, conclusions and future prospects are finally presented in Section 6.

2.

Physics-informed Machine Learning

Mathematically speaking, the problem of automated damage detection from measured sensor data is an inverse problem. In conventional machine learning/deep learning, it is usually formulated as a minimization problem with a purely data-based loss function. However, in most SHM applications, not only is the cost of data acquisition prohibitive, but it is highly unlikely to have sufficient data capturing different damage scenarios, especially for structures that have been in service for relatively short periods of time. When trained using this inherently limited data, the models tend to overfit the given data which eventually leads to poor generalization. This gives rise to the need for some form of model regularization which primarily entails guiding the training process towards an optimal solution in one way or the other. Accomplishing this can result in what may be thought of as data-efficient function approximators.

The requisite domain knowledge for model regularization (guiding the training process) can originate from various sources, can be represented in different forms, and can be integrated into the model in different ways [45]. From the literature on informed machine learning, in case of ANNs, knowledge has been integrated at all three stages–input (pattern) layer [46-53], hidden layers [54-57], output (target) layer [58-60]. Specifically within the context of SHM, domain knowledge can:

  • (I) Aid in the extraction of damage-sensitive features from raw sensor data in cases where conventional machine learning is used and data cannot be used in their raw form.

  • (II) Augment the measured sensor data through transfer learning or by providing secondary data sets from simulations, historical NDI/SHM records or data from similar structures. The diversity of the available historical data usually leads to the need for some form of data fusion as a pre-processing step.

  • (III) Dictate the network architecture by defining local connectivity of nodes within the network, choice of activation functions, learning algorithm, etc.

  • (IV) Impose constraints on the output through proper loss function design or knowledge about damage types and characteristics.

The list is by no means exhaustive as knowledge-based learning in SHM is still in its infancy and is envisioned to bloom in the coming years. In this paper, two classes of problems are discussed based on the type of domain knowledge utilized:

  • 1. When the prior knowledge is in the form of physical laws governing the structural behavior and the same is representable in the form of compact mathematical models governed by partial differential equations (PDEs). The integration of this knowledge into the model is done directly at the loss function level i.e., apart from having a data-based component, the loss function is designed to have a component that is physics-based as well. This is discussed in more detail in Section 3 with the help of forward and inverse problems for dynamic modelling of beams which can be thought of as a promising first step towards the eventual goal of damage detection.

  • 2. When the prior knowledge is available in other less succinct forms like expert human knowledge about a particular system/structure, results from computer simulations which mimic the real-world situation, general information about an underlying physical phenomenon etc,. This form of knowledge can be integrated into the model through strategies like transfer learning, network architecture customization etc,. Section 4 discusses this in more detail with case studies directly related to NDI/SHM.

3.

Prior knowledge in the form of Compact Mathematical Models (PDEs)

Figure 2:

Conventional and ANN-based approaches for solving differential equations

00102_psisdg11379_1137903_page_4_1.jpg

Differential equations is the lingua franca long developed to formulate physical phenomena with both accuracy and succinctness. A purely analytical solution to these equations is feasible only when the underlying structure/phenomenon being modeled is rather simple and straight-forward. Also, the Green’s function approach is usually limited to linear time-invariant cases for unbounded domains. Often times, owing to the inherent structural complexity and/or the nature of the initial/boundary conditions, such approaches no longer remain tractable. It is in these situations that numerical methods for solving differential equations come to the rescue. Some commonly used methods include Runge-Kutta methods, collocation method, weighted residual method etc. The recently popular methods such as finite-element, finite-volume, finite-difference etc., which involve discretization of the domain and weakly solving the differential equations over this discretization yield solutions that are discrete and have limited differentiability.

All these numerical methods have been studied for a long time and are well developed, however, when the source functions and/or initial/boundary conditions change, the results need to be computed afresh from scratch in each case. The experience gained from each computation is lost as there is no fundamental learning of the underlying mechanics. As the size of the structure increases, each simulation becomes computationally expensive owing to the curse of dimensionality and as a result these methods become prohibitively slow. If however a mapping is learned for the structure/phenomenon being modelled, slight variations in the input can be handled rather quickly. ANNs are comprised of a densely interconnected set of very simple processing units (nodes) that can learn highly challenging non-linear mappings with high robustness, adaptability, and fault tolerance due to its distributed representation [61]. Researchers have long been interested in developing representations based on ANNs (which act as universal function approximators [62]) for approximate solutions to these differential equations [63-71]. Research in this direction blossomed especially after 2010 with the advent of GPUs (monstrous computational power) and TensorFlow (a deep learning framework with embedded automatic differentiation).

Raissi et al. [72, 73] solved a variety of non-linear PDEs including Brown equations, Schrodinger equations etc., using TensorFlow. Then they combined this purely numerical method for solving differential equations with traditional data-driven approach to function approximation and actualized the idea of physics-informed learning. By doing so, they were able to not only use sparse datasets to reliably train neural networks, but also to inversely determine unknown parameters of PDEs [74]. Around the same time, several other researchers started investigating physics-informed learning and some of that work can be found in [75-80]. In this paper, the solution of a beam vibration problem using an ANN is presented first followed by a direct demonstration of system physics coming to the rescue when training neural nets with limited data within the context of SHM. The latter is achieved by solving an inverse problem using the same ANN architecture used for the forward problem, for reconstructing the full displacement field using measured sensor data from a sparsely distributed array mounted on the beam. Comparison with the exact analytical solutions is made to highlight the accuracy and generalization capabilities of the physical-informed learning paradigm.

3.1

Beam Vibration Modeling using ANNs (Forward Problem)

Based on Euler-Bernoulli beam theory, the vibration of a simply supported beam under transverse load q(x,t) is governed by Eq. (1) with boundary conditions (BCs) described by Eq. (2)

Figure 3:

Setup of a simply supported beam

00102_psisdg11379_1137903_page_5_1.jpg
00102_psisdg11379_1137903_page_6_1.jpg
00102_psisdg11379_1137903_page_6_2.jpg

For free vibration of the beam (q=0), let us suppose the initial conditions (ICs) are given by

00102_psisdg11379_1137903_page_6_3.jpg

where μEI/ρA, E is the elastic modulus of the beam’s material, I is the area moment of inertia, ρ is the density of the material, A is the cross sectional area, l is its length, and w is the transverse displacement. Collocation, which entails choosing a number of points (called collocation points) in the spatial and temporal domains, is adopted and the loss function ℒ for solving the PDE using an ANN consists of three terms and can be expressed as follows:

00102_psisdg11379_1137903_page_6_4.jpg

where (x,t) is the predicted displacement and the first term in ℒ is the mean squared error (MSE) term associated with the initial conditions, the second term corresponds to the boundary conditions and the third term to the governing equation. NI, NB, and NG stand for the number of collocation points selected from T, ∂Ω, and Ω respectively. The weights αI, αB, and αG for the three terms are determined heuristically to be 1, 1, and 2×10-4 respectively. For free vibration of the simple supported beam with μ=1, l=1 and

00102_psisdg11379_1137903_page_6_5.jpg

the exact solution for transverse displacement can be easily obtained as w(x,t)= sin(3πx).cos(9π2t). The steady-state solution can be considered as a superposition of two transient waves with identical amplitude and propagating in opposite direction as w(x,t)= ½{sin[(3π(x−3πτ)]+ sin[(3π(x+3πτ)]}.

The collocation method implemented via the ANN shows good convergence as long as the time is short, however, for long time durations, convergence problems are encountered and the solutions often converge to the trivial solution. To circumvent this problem, the entire time duration is segmented into a finite number of short time-windows and solved sequentially. By invoking Huygen’s principle, which states that every point on a wavefront is itself the source of secondary wavelets, the solution at the end of a particular time-window can be treated as the initial condition for the subsequent time-window. For this example, the time domain is divided into six time windows (to satisfy the Nyquist criterion). For each window, an ANN with eight hidden layers each of which consists of sixty four neurons is used to approximate the full displacement field. For expediting the process and ensuring convergence, the training process in each case is divided into two phases. The first phase primarily focuses on fitting the ICs and BCs on a data-driven basis, thus the last term in the loss function ℒ is excluded. A gradient descent method is performed first (number of collocation points NI = NB = 50) followed by quasi-Newton algorithm (a limited-memory quasi-Newton code for bound-constrained optimization, L-BFGS-B) (NI = NB = 150) to guide the solution closer to the global minimum in a rather coarse but speedy manner. In the second phase, the term corresponding to the governing equation is added back to the loss function, then L-BFGS-B is employed to finely optimize the parameters of the ANN. The collocation points are selected from a uniform 150 × 150 grid such that NI = NB = 150 and NG = 22500. The results displayed in Fig. 4 show good agreement between the exact solution w(x,t) and the predicted solution (x,t).

Figure 4:

Comparison of the MLP predicted solution with the exact solution for the simply-supported beam with given initial conditions

00102_psisdg11379_1137903_page_7_1.jpg

3.2

Displacement field Reconstruction using Physics-informed ANNs (Inverse Problem)

After demonstrating the applicability of ANNs for solving PDEs in the previous example as a forward problem, the power of physics-informed learning for the inverse problem is now demonstrated vis-à-vis training with limited samples. This is done via a problem involving displacement field reconstruction from a sparse sensor array. Fig. 5 shows the schematic of a simply supported beam with a surface mounted sparse sensor array used in the study. The sparse sensor data (xi,t) is synthetically obtained such that (xi,t) = w(xi,t) + n(xi,t) where xi is the location of the ith sensor, w(xi,t) is the exact displacement at xi sampled from the analytical solution and n(xi,t) is the Gaussian noise with zero mean. The goal is to reconstruct the full displacement field from the sparse measurements and highlight the advantage of embedding the governing equation into the loss function. The loss function for solving it using an ANN consisting of four terms can be defined as follows:

Figure 5:

Setup of a simply supported beam with a sparse sensor array

00102_psisdg11379_1137903_page_7_2.jpg
00102_psisdg11379_1137903_page_8_1.jpg

where the first term is the purely data-driven loss, the second term is for the Dirichlet boundary conditions, the third term is for the governing equation, and the fourth term is for high order boundary conditions. The weights αD, αB1, αG and αB2 for the four terms are determined heuristically to be 5000, 25000, 1 and 1 respectively. For the simple supported beam shown in Fig. 5 with w(x,t)= sin(πx).cos(2π2t) and μ=4, in the first phase, the parameters of the ANN are trained with Adam optimizer with a learning rate of 0.01 for 10000 iterations. During each iteration, while all the ND measurements are used, the collocation points are selected from the defined domain. To be specific, NB = 50 points are sampled for the boundary conditions and NG = 2500 points are sampled for the governing equation. After that, the parameters are further optimized with L-BFGS-B on a larger number of static collocation points until the loss converges. The collocation points for L-BFGS-B optimizer are taken from an equally spaced 150×150 grid over the entire domain, so that NB = 150 and NG = 22500. Fig. 6(a) shows the comparison between the results obtained with physics-informed learning and those obtained using the sparse data only and Fig. 6(b) shows the error for the two approaches as the sparsity of the data and SNR changes. Clearly physics-informed learning outperforms traditional data-driven learning vis-à-vis overfitting when data is sparse and/or noisy. Table 1 presents the comparison in a tabulated manner. The maximum relative error metric used for comparison is defined as (max |w|/max |w|).

Table 1:

Physics-informed learning vs data-driven learning

SNR=20 20 SensorsSNR=10 20 SensorsSNR=15 13 SensorsSNR=20 5 SensorsSNR=10 5 Sensors
Physics InformedTraining Time (s)2293.331808.921864.431846.191398.11
Max. Error2.7%2.4%0.72%0.95%1.9%
Data-DrivenTraining Time (s)48.2548.2442.8852.3537.21
Max. Error3.9%7.2%6.4%29.7%30.3%

Figure 6:

(a) Comparison of the results for the displacement field reconstruction using physics-informed approach and purely data-driven approach (b) Variation of the error for physics-informed learning and data-driven learning as SNR value and data sparsity changes

00102_psisdg11379_1137903_page_9_1.jpg

If the system parameter μ is unknown and is to be identified, it can be set as a trainable parameter and inversely determined through training. For instance, a displacement field 00102_psisdg11379_1137903_page_9_2.jpg is constructed by linearly combining six mode shapes where the coefficients λn are random sampled from [-1, 1]. The system parameter for generating this displacement field is μ=0.0156 but assumed unknown. Noisy measurements extracted from this field are then provided to an MLP for it to reconstruct this displacement field and identify the unknown parameter at the same time. Same training strategy is used as above and the result is shown in Fig. 7. μ is determined to be 0.01555, with a relative error of only 0.32%.

Figure 7:

Results for the full displacement field reconstruction for the given beam when the parameter μ is not provided

00102_psisdg11379_1137903_page_10_1.jpg

To summarize, in this section, initial promise has been demonstrated for the use of ANNs for solving PDEs in forward problems and simple inverse dynamic problems for beams. This can be regarded as a promising first step towards the ultimate goal of solving more complicated inverse problems for automated damage detection in SHM. This logic is analogous to the historical evolution of conventional numerical methods for inverse problems through the development of different regularization schemes after methods for solving forward problems were well matured. Next, two case studies are presented to demonstrate how knowledge in other forms could be utilized for augmenting the ML/DL models. This is done through cases directly falling within the context of NDI/SHM. It is worth mentioning that while the discussion in Section 3 dealt primarily with discrete sensor array data, Sections 4 shifts the focus to vision based health monitoring with Section 4.1 being a static problem (still images as input) and Section 4.2 being a dynamic problem (wavefield videos as input).

4.

Examples on Physics-informed Deep Learning in NDI/SHM

In this Section, a physics-informed deep learning (DL) approach for enhanced visual inspection via augmented reality is first presented as a demonstration of how domain knowledge in the form of expert human feedback and historical NDI/SHM records can be used to augment the training data. This is followed by a demonstration of impact diagnosis using a physics-informed deep learning model.

4.1

Enhanced Visual Inspection via Augmented Reality (AR)

The first direct demonstration of the role of domain knowledge in augmenting the ML/DL models for tasks directly related to NDI/SHM is done through what is referred to as enhanced visual inspection. In the aerospace industry, for safety critical structures, visual inspection (VI) is usually the first line of defense and given its various advantages, it accounts for at least 80% of all aircraft inspections according to FAA [81]. Apart from being considerably flexible, it is the most direct, intuitive, straightforward, and economical method for accessing the condition of a structure [82]. However, when performed in its traditional form (by a human inspector without any external aids), VI has certain inherent shortcomings like, being error-prone, labor-intensive, tedious, subjective/inconsistent, etc [83-85]. An inspector is required to first acquire relevant training over a period of time and then to bridge the gap between the acquired knowledge and the physical systems in the field mentally which limits his/her ability to make the best use of the wealth of information available about the inspection task being performed.

The use of augmented reality (AR) as an information delivery paradigm has been proposed in order to overcome some of the major shortcomings of traditional VI. AR encompasses a set of technologies that superimpose digital data and images on the physical world thereby enhancing the user’s perception of reality [86]. It has the potential to bridge the disconnect between the physical and digital worlds, thus enabling humans to make efficient use of the information available and consequently make more informed decisions. As such, it can act as a tool to enhance the traditional visual inspection process. A proof-of-concept for such an enhanced visual inspection system using a head mounted AR device (AR glasses) is presented here. The primary focus in the study was the detection of most commonly occurring defects in metallic structures– corrosion, fatigue cracks, and/or a combination of the two. This was achieved by deploying a deep learning (DL) based computer vision model on the AR device.

Oftentimes, considerations of system downtime and accessibility prohibit a detailed inspection of the entire system. Under such circumstances, while traditional VI may give rise to high rates of false-positives (false alarms) and/or false-negatives (misdetections), using a purely data-driven approach to search for damages/anomalies over the entire region of interest (ROI) can be extremely expensive computationally making real-time detection almost impractical with the current state-of-the-art AR devices. Also, depending on the distance and/or angle between the inspector and the area being inspected, a damage may or may not be easily discernible. As such, with the aim of improving the probability of detection (POD) enhancing the overall efficiency of the detection process, prior domain knowledge was incorporated into the ML pipeline. Fig 8 gives an overview of the type of domain knowledge available and its integration into the machine learning pipeline. The details are explained next.

Figure 8:

Overview of the type of domain knowledge available and its integration into the machine learning (ML) pipeline for accomplishing enhanced visual inspection

00102_psisdg11379_1137903_page_11_1.jpg

Rather than searching only for damages (in this case cracks/corrosion) in the actual ROI, regions that are prone to damage are also categorized and grouped. The information about damage prone areas can come either from expert knowledge about the system or from historical data. As an example, areas prone to cracking may include regions with similar types of fasteners like bolts, rivets, etc. In this way, a preliminary analysis of the ROI may lead to the detection of prominent damages and/or damage prone regions. If a damage prone region is detected, the inspector is prompted to zoom into the region in order to carry out a more detailed analysis with the aim of detecting less prominent damages i.e., damages that are otherwise below the detectability threshold of the algorithm at the original scale of view. This can be achieved either by using the zoom-in function in the AR device camera or by simply walking closer to the region if there are no accessibility issues. In this way, analyzing the ROI at multiple scales (prompted by expert/domain knowledge about the system) has the potential to greatly increase the POD while reducing false alarms. It should also be noted that since a detailed analysis is carried out only of the highlighted damage prone regions, this region-based detection paradigm has the potential to make the overall inspection process much more efficient than carrying out a detailed inspection of the entire system. A schematic of the proposed physics-informed methodology is shown in Fig. 9 and some representative results are shown in Fig 10. The results were obtained using Epsom BT-300 smart glasses (AR/Developer edition) with a pre-trained MobileNet (SSD MobileNet V2) and k-means clustering was used to group the detected fasteners together. More details about the DL algorithms used in the study and the details of the dataset used for training can be found in [87].

Figure 9:

Multi-scale analysis of the region-of-interest (ROI) prompted by the domain knowledge

00102_psisdg11379_1137903_page_12_1.jpg

Figure 10:

(a) Original ROI (b) Fasteners detected in the ROI, no cracks detected (c) Regions identified based on the fasteners after zooming-in into the identified regions, the cracks become more prominent thereby increasing the overall POD. Each detection in the figure has the associated confidence level with it.

00102_psisdg11379_1137903_page_12_2.jpg

While the identification of visible damage is an important aspect of monitoring the health of structures, the real challenge in most SHM applications is posed by hidden damage. This is discussed in the next section.

4.2

A Physics-informed Deep Learning (DL) Approach for Impact Diagnosis

Effective, reliable, and robust identification of impact events that have the potential to damage a structure is an aspect of SHM that is of prime importance in the aerospace industry. This is commonly referred to as impact diagnosis and its importance stems from the fact that low-velocity impact events (like tool drops, runway debris, bird strikes etc.) can lead to what is called barely-visible-impact-damage (BVID) in composites which has been found to be the most prominent cause of in-service damage to aerospace structures. The existence and/or extent of this damage can be correlated with the impact energy (which is the area underneath the impact force time-history curve), with no damage being assumed below a certain energy threshold. Impact diagnosis, which entails the identification of the impact location and reconstruction of the impact force time-history, can quantify this impact energy and in this way can serve as an indirect method for the detection of an otherwise invisible damage right at its inception [88].

Low-velocity impact on a structure emanates an elastic wave that propagates through the structure carrying a wealth of information about the impact event. Most of the classical impact diagnosis methods (both physics-based [89, 90] and data-driven [91-94]) capture information about the wave propagation phenomenon only at discrete locations by employing a limited number of sensors spatially distributed on the structure. Both of these approaches are limited in their practicality as only a limited amount of information about the scattered wavefield can be captured by the discrete sensor array. This becomes especially problematic in complex real-world structures with geometric features like joints, stiffeners, etc., where the discrete sensor signals fail to provide sufficient insights as they are often corrupted by high levels of coherent noise from multiple wave reflections, wave mode conversions etc. In order to address this issue of limited wavefield data, it was proposed to use the full wavefield for impact diagnosis. A proof-of-concept for a DL based approach for analyzing full wavefields for impact diagnosis is presented here. For this study, while simulated wavefields were employed, the feasibility of an integrated high-speed camera system for non-contact full-field wavefield capture is discussed in the next section.

The high spatio-temporal dimensionality of the wavefield mandates the use deep learning for analysis. Also, the nature of the impact diagnosis problem requires the capturing of context from the wavefield evolution which necessitates learning across multiple time-frames of the wavefield simultaneously rather than focusing independently on each frame. Fig 11 shows an end-to-end trainable CNN (convolutional neural network)–RNN (recurrent neural network) network architecture employed for the spatio-temporal analysis of the scattered wavefield. The model was trained on simulated wavefield data and care was exercised to ensure that the training data was not significantly deviated from the data the model is expected to encounter in the real world. This was assured by incorporating different noise levels and location biases in the training data and by testing the model on simulated wavefields in response to real impact force profiles generated using an impact hammer. Overall, this lies under the paradigm of letting DL models learn in a simulated environment before transferring their knowledge to the real world.

Figure 11:

A unified CNN-RNN network architecture for spatio-temporal analysis of the impact generated wavefield

00102_psisdg11379_1137903_page_13_1.jpg

Impact diagnosis clearly represents a case which is extremely data-intensive and where collecting sufficient real data is hard, as such, an ideal candidate for incorporating prior domain knowledge into the ML pipeline. Fig 12 gives an overview of the type of domain knowledge available and its integration into the machine learning pipeline. A wealth of information available about the dynamics of the wave propagation phenomenon is available in this case. In particular, Huygens’ principle, which states that different time-frames of the wavefield are not independent of each other and an inherent temporal structure exists between them, dictated the overall network architecture. This formed the basis for treating the wavefield frames/feature vectors extracted from the frames by the CNN part of the network as a sequence with the order of the individual feature vectors being meaningful/important. Also, the classical time-reversal (T-R) technique pioneered by Fink [95-97] which is usually employed for source identification in many disciplines as an optimal focusing technique formed the basis for the so-called reversed-time learning in the recurrent part of the network. Fig 13 shows some of the results for impact diagnosis and the effect of incorporating domain knowledge into the model. As can be seen, reasonable improvement, especially in impact force reconstruction accuracy is obtained by incorporating the concept of time-reversal with accuracies in the range of (89-93)% being obtained for SNR values of 20 and 10. The results obtained for SNR value 5 are expectedly poor because of the very extreme noise level.

Figure 12:

Overview of the type of domain knowledge available and its integration into the machine learning (ML) pipeline for accomplishing impact diagnosis

00102_psisdg11379_1137903_page_14_1.jpg

Figure 13:

Results for impact diagnosis including impact location and impact force reconstruction using the physics-informed DL-based methodology

00102_psisdg11379_1137903_page_14_2.jpg

So far, in Sections 2-4, the focus has been on augmenting the incomplete/insufficient training data by incorporating domain knowledge into the model during training. While this is indispensable for realizing the goal of real-time health monitoring in large/complex real-world structure, effort should also be made to develop more efficient data acquisition systems for SHM. To this end, the next Section presents a non-contact sensing paradigm for full-field measurements for vision-based SHM.

5.

A Non-contact Full-field Sensing Paradigm for Vision-based SHM

Computer vision techniques for automated damage assessment have been engendering a lot of interest in recent times in civil infrastructures, especially during post-disaster inspections, when the number of structures to be inspected is far beyond the capability of available inspectors [98-103]. In the field of civil infrastructure assessment, safety-critical damage is usually big enough to be visible to the naked eye, as such, static images captured using ordinary digital cameras suffice for the purpose of automated damage assessment using state-of-the-art computer vision algorithms. For instance, a fusion CNN was proposed by Dr. Li’s group [104-105] to successfully detect distributed cracks on steel girders of actual bridges. However, for aerospace structures, the most critical type of damage is barely visible to the naked eye. In such (plate-like) structures, hidden details of the structure, including information about the location of the damage, if present, and its characteristics can be unearthed using ultrasonic guided wave based techniques, which entail investigating the scattered guided waves in the structure using appropriate signal/image processing algorithms. These scattered waves can be sensed using either contact (e.g., a network of piezoelectric (PZT) sensor arrays) [106] or fully non-contact means (e.g., air-coupled transducers, laser Doppler vibrometer (LDV)) [107-112]. Because of the multitude of advantages associated with non-contact full-field sensing, the scanning LDV has attracted a lot of interest in recent times, however, it is limited to point-by-point measurement/sensing (i.e., one point at a time) [113].

In order to overcome these limitations, a sensing paradigm capable of taking all the measurements simultaneously is desirable. Digital image correlation (DIC), which is a non-interferometric optical technique, extracts surface deformations by comparing digital images of the structure’s surface captured before and after deformation [114-118]. Advances in computers and digital imaging technology have already enabled 2-D/3-D DIC methods to extend from static or quasi-static to dynamic applications, such as low-frequency vibration (in the range of kHz) and full-field modal analysis for characterizing the behavior of structures [119-121]. In order to expand DIC applications into the ultrasonic range (> 20 kHz), for example, in the area of ultrasonic guided-wave based damage imaging, image acquisition systems with extremely high spatial/temporal resolution are required.

High-speed digital cameras, even though not there yet, provide the most viable means for accomplishing this. Not only do they enable simultaneous data acquisition at all the points in the region-of-interest (ROI) but the reconstructed wavefield using the sensed data provides the most spatially continuous information about the wave propagation phenomena as each pixel is considered to effectively act as a sensor. The requirement of extremely high spatial/temporal resolution, however, continues to remain a bottleneck in the adoption of this technology for ultrasonic guided wavefield reconstruction. However, the problem is purely related to the processing speed and will improve with the development of computer technology.

Furthermore, as IC technology rapidly advances following Moore’s law, new high-speed cameras promise to have high potential for capturing the propagating guided wave on the surface of the structures.

Fig. 14 shows the schematic of a proof-of-concept for an integrated high-speed camera system for non-contact full-field sensing of the scattered wavefield in structures [122] and the damage image generated from the captured wavefield using modified wavenumber index (WI) imaging condition [123]. It should be noted that since only a single high-speed camera was used, it was possible to capture only the in-plane scattered wavefield. In order to circumvent the current hardware limitations associated with using a single high-speed camera for the acquisition of ultrasonic guided in-plane wavefields, three strategies were adopted. First, for improving the spatial/temporal resolution, sample interleaving and image stitching techniques were used to get an effective data acquisition rate that was nearly 250 times that of the original rate specified by the camera. Second, the concept was demonstrated on a thin, flat, low-modulus, high-density polyethylene (HDPE) plate, which is a flexible polymer sheet with less than one percent of the stiffness of regular carbon fiber reinforced polymers (CFRPs). The low-modulus and high-density plate enables large deformations and low wave speeds, which make it possible to successfully detect the in-plane wavefields using the affordable in-house high-speed camera (Photron Fastcam Mini AX200). Third, a large PZT actuator was used in conjunction with a power amplifier to generate sufficient energy to excite measurable guided waves in the plate, and the excitation frequency was carefully chosen to be a continuous harmonic wave such that not only the fundamental A0 and S0 modes exist, but a strong fundamental SH0 wave mode also persists (due to multiple reflections from the damage and plate boundaries).

Figure 14:

(a) Schematic of the experimental setup for non-contact full-field sensing (b) Damage image of a BVID generated from the captured guided wavefield using modified wavenumber index (WI) imaging condition

00102_psisdg11379_1137903_page_16_1.jpg

While currently the concept could only be demonstrated on the HDPE sheet due to hardware limitations associated with using a single high-speed camera, the study shows significant initial promise for the adoption of this technology for non-contact full-field measurements with the ultimate goal of visualizing hidden damage such as barely visible impact damage (BVID) in complex flat/curved composite structures. In order to continue development of this concept past this successful feasibility study and begin optimizing it for practical applications, the following tasks are planned. (1) Multiple high-speed cameras will be employed to enable the capture of three-dimensional displacement/strain scattered wavefields with larger fields of view. It is envisioned that out-of-plane anti-symmetric wave modes (such as the fundamental A0 wave mode) will provide significantly higher SNR values to yield better damage images. (2) Physics-based image stitching techniques will be explored in order to handle more-complicated geometries and damage scenarios. (3) For using the captured full wavefields in conjunction with physics-informed learning for realizing the eventual goal of developing an end-to-end non-contact vision based damage detection system in SHM, other transfer learning models like the one developed in Section 4.2 will be developed.

6.

Summary, Conclusions and Future Prospects

When the structures being monitored (either continuously or intermittently) are significantly large/complex and/or when the variations in environmental/operational conditions cannot be neglected, a physics-based approach for assessing the structural health becomes highly unreliable. In recent times, the number of parameters being monitored in-situ in real-world structures has been steadily increasing through the use of miniaturized wireless sensor networks and that has led to an abundance of historical data for these structures. The heterogeneous nature of this data (due to multiple sensing modalities) and the emphasis on developing long-term SHM systems has engendered a lot of interest in a data-driven approach for SHM. However, for these real-world structures, it is highly impractical to collect sufficient data (corresponding to different damage scenarios) to train a robust and generalizable model for automated damage detection and characterization. Consequently, an optimal approach to SHM must encompass, on one hand, some means of augmenting the collected sensor data in order to compensate for the lack of completeness in the data and on the other hand comprehensive sensing mechanisms with a framework for subsequent data fusion and processing of the large volumes of collected data.

In this paper, these aspects of such an optimal SHM paradigm were addressed individually through: (1) The use of physics-informed machine learning for the analysis of the measured sensor data. (2) The use of high-speed digital cameras for capturing ultrasonic guided wavefields in structures as a means of providing the most spatially continuous information about the propagating ultrasonic guided waves within the structure. Owing to the attention it has been engendering in recent times, physics-informed machine learning was discussed in some detail with the eventual goal of SHM in mind. By demonstrating the applicability of ANNs for solving forward and inverse problems involving PDEs, initial promise was demonstrated towards solving more complicated inverse problems for the task of automated damage detection in SHM. It was shown how the overfitting/generalization issue generally associated with the insufficient nature of the training data can be mitigated by incorporating physical laws/domain knowledge to guide the back-propagation during the training process. Other ways of incorporating domain knowledge into the ML/DL pipeline were also discussed through the case studies related to different aspects of SHM i.e., enhanced visual inspection and impact diagnosis.

Internet of Things (IoT) has recently attracted great attention due to its promising potential and capacity to be integrated into complex systems [124, 125]. With rapid advances in information and sensing technologies such as wireless communication, cloud computing, and wireless sensor networks (WSNs), IoT is emerging as an enabling technology for the development of low cost, efficient, reliable, and scalable SHM systems. Recent advances in deep learning (both supervised and unsupervised) and breakthroughs in the integration of system physics/domain knowledge into the ML/DL models have had a catalytic effect on the adoption of a physics-informed data-driven approach to learning in many application domains. It is envisioned that as robust non-contact full-field sensing mechanisms are developed and more success in physics-informed learning is achieved, especially for solving complex inverse problems in SHM, the goal of in-situ real-time vision based health monitoring can be achieved in near future.

Acknowledgement

One of the authors, Mr. Sakib A. Zargar would like to express their sincere thanks to Dr. Jeremy Yagle of NASA Langley Research Center for the support through National Institute of Aerospace (NIA).

References

[1] 

Giurgiutiu, V., Structural health monitoring: with piezoelectric wafer active sensors, Elsevier.2007). Google Scholar

[2] 

Structural health monitoring (SHM) in aerospace structures, Woodhead Publishing.2016). Google Scholar

[3] 

Cawley, P., “Structural health monitoring: Closing the gap between research and industrial deployment.,” Structural Health Monitoring, 17 (5), 1225 –1244 (2018). https://doi.org/10.1177/1475921717750047 Google Scholar

[4] 

Doebling, S. W., Farrar, C. R., & Prime, M. B., “A summary review of vibration-based damage identification methods.,” Shock and vibration digest, 30 (2), 91 –105 (1998). https://doi.org/10.1177/058310249803000201 Google Scholar

[5] 

Farrar, C. R., Doebling, S. W., & Nix, D. A., “Vibration–based structural damage identification.,” Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, 359 (1778), 131 –149 (2001). https://doi.org/10.1098/rsta.2000.0717 Google Scholar

[6] 

Alvandi, A., & Cremona, C., “Assessment of vibration-based damage identification techniques.,” Journal of sound and vibration, 292 (1-2), 179 –202 (2006). https://doi.org/10.1016/j.jsv.2005.07.036 Google Scholar

[7] 

Montalvao, D., Maia, N. M. M., & Ribeiro, A. M. R., “A review of vibration-based structural health monitoring with special emphasis on composite materials.,” Shock and vibration digest, 38 (4), 295 –324 (2006). https://doi.org/10.1177/0583102406065898 Google Scholar

[8] 

Fan, W., & Qiao, P., “Vibration-based damage identification methods: a review and comparative study.,” Structural health monitoring, 10 (1), 83 –111 (2011). https://doi.org/10.1177/1475921710365419 Google Scholar

[9] 

Raghavan, A., & Cesnik C. E. S., “Review of guided-wave structural health monitoring,” The Shock and Vibration Digest, 39 (2), 91 –114 (2007). https://doi.org/10.1177/0583102406075428 Google Scholar

[10] 

Su, Z., & Ye, L., Identification of damage using Lamb waves: from fundamentals to applications, 48 Springer Science & Business Media.2009). https://doi.org/10.1007/978-1-84882-784-4 Google Scholar

[11] 

Mitra, M., & Gopalakrishnan, S., “Guided wave based structural health monitoring: A review.,” Smart Materials and Structures, 25 (5), 053001 (2016). https://doi.org/10.1088/0964-1726/25/5/053001 Google Scholar

[12] 

Wu, X., Ghaboussi, J., & Garrett Jr, J. H., “Use of neural networks in detection of structural damage.,” Computers & structures, 42 (4), 649 –659 (1992). https://doi.org/10.1016/0045-7949(92)90132-J Google Scholar

[13] 

Worden, K., & Manson, G., “The application of machine learning to structural health monitoring.,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 365 (1851), 515 –537 (2007). https://doi.org/10.1098/rsta.2006.1938 Google Scholar

[14] 

Farrar, C. R., & Worden, K., Structural health monitoring: a machine learning perspective, John Wiley & Sons.2012). https://doi.org/10.1002/9781118443118 Google Scholar

[15] 

Rytter, A., Vibrational based inspection of civil engineering structures, (1993). Google Scholar

[16] 

Allemang, R. J., & Brown, D. L., “A correlation coefficient for modal vector analysis,” in Proceedings of the 1st international modal analysis conference, 110 –116 (1982). Google Scholar

[17] 

Allemang, R. J., “The modal assurance criterion–twenty years of use and abuse.,” Sound and vibration, 37 (8), 14 –23 (2003). Google Scholar

[18] 

Kim, J. T., Ryu, Y. S., Cho, H. M., & Stubbs, N., “Damage identification in beam-type structures: frequency-based method vs mode-shape-based method.,” Engineering structures, 25 (1), 57 –67 (2003). https://doi.org/10.1016/S0141-0296(02)00118-9 Google Scholar

[19] 

Shi, Z., Law, S. S., & Zhang, L., “Structural damage detection from modal strain energy change.,” Journal of engineering mechanics, 126 (12), 1216 –1223 (2000). https://doi.org/10.1061/(ASCE)0733-9399(2000)126:12(1216) Google Scholar

[20] 

Pandey, A. K., Biswas, M., & Samman, M. M., “Damage detection from changes in curvature mode shapes.,” Journal of sound and vibration, 145 (2), 321 –332 (1991). https://doi.org/10.1016/0022-460X(91)90595-B Google Scholar

[21] 

Wahab, M. A., & De Roeck, G., “Damage detection in bridges using modal curvatures: application to a real damage scenario.,” Journal of Sound and vibration, 226 (2), 217 –235 (1999). https://doi.org/10.1006/jsvi.1999.2295 Google Scholar

[22] 

Pandey, A. K., & Biswas, M., “Damage detection in structures using changes in flexibility.,” Journal of sound and vibration, 169 (1), 3 –17 (1994). https://doi.org/10.1006/jsvi.1994.1002 Google Scholar

[23] 

Pandey, A. K., & Biswas, M., “Damage diagnosis of truss structures by estimation of flexibility change.,” Modal Analysis-the International Journal of Analytical and Experimental Modal Analysis, 10 (2), 104 –117 (1995). Google Scholar

[24] 

Jaishi, B., & Ren, W. X., “Damage detection by finite element model updating using modal flexibility residual.,” Journal of sound and vibration, 290 (1-2), 369 –387 (2006). https://doi.org/10.1016/j.jsv.2005.04.006 Google Scholar

[25] 

Bernal, D., “Load vectors for damage localization.,” Journal of Engineering Mechanics, 128 (1), 7 –14 (2002). https://doi.org/10.1061/(ASCE)0733-9399(2002)128:1(7) Google Scholar

[26] 

Staszewski, W. J., Worden, K., & Tomlinson, G. R., “Time–frequency analysis in gearbox fault detection using the Wigner–Ville distribution and pattern recognition.,” Mechanical systems and signal processing, 11 (5), 673 –692 (1997). https://doi.org/10.1006/mssp.1997.0102 Google Scholar

[27] 

Sohn, H., Park, G., Wait, J. R., Limback, N. P., & Farrar, C. R., “Wavelet-based active sensing for delamination detection in composite structures.,” Smart Materials and structures, 13 (1), 153 (2003). https://doi.org/10.1088/0964-1726/13/1/017 Google Scholar

[28] 

Ovanesova, A. V., & Suarez, L. E., “Applications of wavelet transforms to damage detection in frame structures.,” Engineering structures, 26 (1), 39 –49 (2004). https://doi.org/10.1016/j.engstruct.2003.08.009 Google Scholar

[29] 

Kim, H., & Melhem. H., “Damage detection of structures by wavelet analysis,” Engineering structures, 26 (3), 347 –362 (2004). https://doi.org/10.1016/j.engstruct.2003.10.008 Google Scholar

[30] 

Taha, M. R., Noureldin, A., Lucero, J. L., & Baca, T. J., “Wavelet transform for structural health monitoring: a compendium of uses and features.,” Structural health monitoring, 5 (3), 267 –295 (2006). https://doi.org/10.1177/1475921706067741 Google Scholar

[31] 

Yang, J. N., Lei, Y., Lin, S., & Huang, N., “Hilbert-Huang based approach for structural damage detection.,” Journal of engineering mechanics, 130 (1), 85 –95 (2004). https://doi.org/10.1061/(ASCE)0733-9399(2004)130:1(85) Google Scholar

[32] 

Hay, T. R., Royer, R. L., Gao, H., Zhao, X., & Rose, J. L., “A comparison of embedded sensor Lamb wave ultrasonic tomography approaches for material loss detection.,” Smart materials and structures, 15 (4), 946 (2006). https://doi.org/10.1088/0964-1726/15/4/007 Google Scholar

[33] 

He, J., & Yuan, F. G., “Lamb-wave-based two-dimensional areal scan damage imaging using reverse-time migration with a normalized zero-lag cross-correlation imaging condition.,” Structural Health Monitoring, 16 (4), 444 –457 (2017). https://doi.org/10.1177/1475921716674373 Google Scholar

[34] 

Kudva, J. N., Munir, N., & Tan, P. W., “Damage detection in smart structures using neural networks and finite-element analyses.,” Smart Materials and Structures, 1 (2), 108 (1992). https://doi.org/10.1088/0964-1726/1/2/002 Google Scholar

[35] 

Teboub, Y., & Hajela, P., “A neural network based damage analysis of smart composite beams,” in 4th Symposium on Multidisciplinary Analysis and Optimization, 4685 (1992). Google Scholar

[36] 

Mehrjoo, M., Khaji, N., Moharrami, H., & Bahreininejad, A., “Damage detection of truss bridge joints using Artificial Neural Networks.,” Expert Systems with Applications, 35 (3), 1122 –1131 (2008). https://doi.org/10.1016/j.eswa.2007.08.008 Google Scholar

[37] 

Liu, Y. Y., Ju, Y. F., Duan, C. D., & Zhao, X. F., “Structure damage diagnosis using neural network and feature fusion.,” Engineering applications of artificial intelligence, 24 (1), 87 –92 (2011). https://doi.org/10.1016/j.engappai.2010.08.011 Google Scholar

[38] 

Hinton, G. E., Osindero, S., & Teh, Y. W., “A fast learning algorithm for deep belief nets,” Neural computation, 18 (7), 1527 –1554 (2006). https://doi.org/10.1162/neco.2006.18.7.1527 Google Scholar

[39] 

LeCun, Y., Bengio, Y., & Hinton, G., “Deep learning.,” Nature, 521 (7553), 436 –444 (2015). https://doi.org/10.1038/nature14539 Google Scholar

[40] 

Schmidhuber, J., “Deep learning in neural networks: An overview.,” Neural networks, 61 85 –117 (2015). https://doi.org/10.1016/j.neunet.2014.09.003 Google Scholar

[41] 

Sze, V., Chen, Y. H., Yang, T. Y., & Emer, J. S., “Efficient processing of deep neural networks: A tutorial and survey.,” Proceedings of the IEEE, 105 (12), 2295 –2329 (2017). https://doi.org/10.1109/JPROC.2017.2761740 Google Scholar

[42] 

Diez, A., Khoa, N. L. D., Alamdari, M. M., Wang, Y., Chen, F., & Runcie, P., “A clustering approach for structural health monitoring on bridges,” Journal of Civil Structural Health Monitoring, 6 (3), 429 –445 (2016). https://doi.org/10.1007/s13349-016-0160-0 Google Scholar

[43] 

Alves, V., Cury, A., Roitman, N., Magluta, C., & Cremona, C., “Novelty detection for SHM using raw acceleration measurements.,” Structural Control and Health Monitoring, 22 (9), 1193 –1207 (2015). https://doi.org/10.1002/stc.v22.9 Google Scholar

[44] 

Toivola, J., Prada, M. A., & Hollmén, J., “Novelty detection in projected spaces for structural health monitoring. In,” International Symposium on Intelligent Data Analysis, 208 –219 Springer, Berlin, Heidelberg (2010). Google Scholar

[45] 

Von Rueden, L., Mayer, S., Garcke, J., Bauckhage, C., & Schuecker, J., “Informed machine learning–towards a taxonomy of explicit integration of knowledge into machine learning.,” Learning, 18 (19), 20 (2019). Google Scholar

[46] 

Kramer, M. A., “Nonlinear principal component analysis using autoassociative neural networks,” AIChE journal, 37 (2), 233 –243 (1991). https://doi.org/10.1002/(ISSN)1547-5905 Google Scholar

[47] 

Mavrovouniotis, M. L., & Chang, S., “Hierarchical neural networks.,” Computers & chemical engineering, 16 (4), 347 –369 (1992). https://doi.org/10.1016/0098-1354(92)80053-C Google Scholar

[48] 

Tan, S., & Mayrovouniotis, M. L., “Reducing data dimensionality through optimizing neural network inputs.,” AIChE Journal, 41 (6), 1471 –1480 (1995). https://doi.org/10.1002/(ISSN)1547-5905 Google Scholar

[49] 

Yun, C. B., & Bahng, E. Y., “Substructural identification using neural networks.,” Computers & Structures, 77 (1), 41 –52 (2000). https://doi.org/10.1016/S0045-7949(99)00199-6 Google Scholar

[50] 

Marwala, T., “Damage identification using committee of neural networks.,” Journal of Engineering Mechanics, 126 (1), 43 –50 (2000). https://doi.org/10.1061/(ASCE)0733-9399(2000)126:1(43) Google Scholar

[51] 

Chen, S. L., & Jen, Y. W., “Data fusion neural network for tool condition monitoring in CNC milling machining.,” International journal of machine tools and manufacture, 40 (3), 381 –400 (2000). https://doi.org/10.1016/S0890-6955(99)00066-8 Google Scholar

[52] 

Zang, C., & Imregun, M., “Structural damage detection using artificial neural networks and measured FRF data reduced via principal component projection.,” Journal of sound and vibration, 242 (5), 813 –827 (2001). https://doi.org/10.1006/jsvi.2000.3390 Google Scholar

[53] 

Ghosh, N., Ravi, Y. B., Patra, A., Mukhopadhyay, S., Paul, S., Mohanty, A. R., & Chattopadhyay, A. B., “Estimation of tool wear during CNC milling using neural network-based sensor fusion.,” Mechanical Systems and Signal Processing, 21 (1), 466 –479 (2007). https://doi.org/10.1016/j.ymssp.2005.10.010 Google Scholar

[54] 

Kramer, M. A., “Autoassociative neural networks,” Computers & chemical engineering, 16 (4), 313 –328 (1992). https://doi.org/10.1016/0098-1354(92)80051-A Google Scholar

[55] 

Zhang, J., Walter, G. G., Miao, Y., & Lee, W. N. W., “Wavelet neural networks for function learning,” IEEE transactions on Signal Processing, 43 (6), 1485 –1497 (1995). https://doi.org/10.1109/78.388860 Google Scholar

[56] 

Wang, F., & Zhang, Q. J., “Knowledge-based neural models for microwave design,” IEEE Transactions on Microwave Theory and Techniques, 45 (12), 2333 –2343 (1997). https://doi.org/10.1109/22.643839 Google Scholar

[57] 

Jiang, X., & Adeli, H., “Pseudospectra, MUSIC, and dynamic wavelet neural network for damage detection of highrise buildings,” International Journal for Numerical Methods in Engineering, 71 (5), 606 –629 (2007). https://doi.org/10.1002/(ISSN)1097-0207 Google Scholar

[58] 

Psichogios, D. C., & Ungar, L. H., “A hybrid neural network-first principles approach to process modeling,” AIChE Journal, 38 (10), 1499 –1511 (1992). https://doi.org/10.1002/(ISSN)1547-5905 Google Scholar

[59] 

Kramer, M. A., Thompson, M. L., & Bhagat, P. M., “Embedding theoretical models in neural networks,” in In 1992 American Control Conference, 475 –479 (1992). Google Scholar

[60] 

Thompson, M. L., & Kramer, M. A., “Modeling chemical processes using prior knowledge and neural networks,” AIChE Journal, 40 (8), 1328 –1340 (1994). https://doi.org/10.1002/(ISSN)1547-5905 Google Scholar

[61] 

Jain, A. K., Mao, J., & Mohiuddin, K. M., “Artificial neural networks: A tutorial.,” Computer, 29 (3), 31 –44 (1996). https://doi.org/10.1109/2.485891 Google Scholar

[62] 

Hornik, K., Stinchcombe, M., & White, H., “Multilayer feedforward networks are universal approximators.,” Neural networks, 2 (5), 359 –366 (1989). https://doi.org/10.1016/0893-6080(89)90020-8 Google Scholar

[63] 

Dissanayake, M. W. M. G., & Phan-Thien, N., “Neural-network-based approximations for solving partial differential equations.,” communications in Numerical Methods in Engineering, 10 (3), 195 –201 (1994). https://doi.org/10.1002/cnm.1640100303 Google Scholar

[64] 

Lagaris, I. E., Likas, A., & Fotiadis, D. I., “Artificial neural networks for solving ordinary and partial differential equations.,” IEEE transactions on neural networks, 9 (5), 987 –1000 (1998). https://doi.org/10.1109/72.712178 Google Scholar

[65] 

Lagaris, I. E., Likas, A. C., & Papageorgiou, D. G., “Neural-network methods for boundary value problems with irregular boundaries.,” IEEE Transactions on Neural Networks, 11 (5), 1041 –1049 (2000). https://doi.org/10.1109/72.870037 Google Scholar

[66] 

He, S., Reif, K., & Unbehauen, R., “Multilayer neural networks for solving a class of partial differential equations.,” Neural networks, 13 (3), 385 –396 (2000). https://doi.org/10.1016/S0893-6080(00)00013-7 Google Scholar

[67] 

Aarts, L. P., & Van Der Veer, P., “Neural network method for solving partial differential equations.,” Neural Processing Letters, 14 (3), 261 –271 (2001). https://doi.org/10.1023/A:1012784129883 Google Scholar

[68] 

Alli, H., Uçar, A., & Demir, Y., “The solutions of vibration control problems using artificial neural networks.,” Journal of the Franklin Institute, 340 (5), 307 –325 (2003). https://doi.org/10.1016/S0016-0032(03)00036-X Google Scholar

[69] 

Malek, A., & Beidokhti, R. S., “Numerical solution for high order differential equations using a hybrid neural network—optimization method.,” Applied Mathematics and Computation, 183 (1), 260 –271 (2006). https://doi.org/10.1016/j.amc.2006.05.068 Google Scholar

[70] 

Beidokhti, R. S., & Malek, A., “Solving initial-boundary value problems for systems of partial differential equations using neural networks and optimization techniques.,” Journal of the Franklin Institute, 346 (9), 898 –913 (2009). https://doi.org/10.1016/j.jfranklin.2009.05.003 Google Scholar

[71] 

Shirvany, Y., Hayati, M., & Moradian, R., “Multilayer perceptron neural networks with novel unsupervised training method for numerical solution of the partial differential equations.,” Applied Soft Computing, 9 (1), 20 –29 (2009). https://doi.org/10.1016/j.asoc.2008.02.003 Google Scholar

[72] 

Raissi, M., Perdikaris, P., & Karniadakis, G. E., “Physics informed deep learning (part i): Data-driven solutions of nonlinear partial differential equations.,” arXiv preprint arXiv:1711.10561, (2017). Google Scholar

[73] 

Raissi, M., Perdikaris, P., & Karniadakis, G. E., “Physics informed deep learning (part II): Data-driven discovery of nonlinear partial differential equations. arXiv.,” arXiv preprint arXiv:1711.10561, (2017). Google Scholar

[74] 

Raissi, M., Perdikaris, P., & Karniadakis, G. E., “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.,” Journal of Computational Physics, 378 686 –707 (2019). https://doi.org/10.1016/j.jcp.2018.10.045 Google Scholar

[75] 

Sirignano, J., & Spiliopoulos, K., “DGM: A deep learning algorithm for solving partial differential equations.,” Journal of Computational Physics, 375 1339 –1364 (2018). https://doi.org/10.1016/j.jcp.2018.08.029 Google Scholar

[76] 

Anitescu, C., Atroshchenko, E., Alajlan, N., & Rabczuk, T., “Artificial neural network methods for the solution of second order boundary value problems.,” Computers, Materials & Continua, 59 (1), 345 –359 (2019). https://doi.org/10.32604/cmc.2019.06641 Google Scholar

[77] 

Berg, J., & Nyström, K., “A unified deep artificial neural network approach to partial differential equations in complex geometries.,” Neurocomputing, 317 28 –41 (2018). https://doi.org/10.1016/j.neucom.2018.06.056 Google Scholar

[78] 

Meng, X., & Karniadakis, G. E., “A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems.,” Journal of Computational Physics, 401 109020 (2020). https://doi.org/10.1016/j.jcp.2019.109020 Google Scholar

[79] 

Zhu, Y., Zabaras, N., Koutsourelakis, P. S., & Perdikaris, P., “Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data.,” Journal of Computational Physics, 394 56 –81 (2019). https://doi.org/10.1016/j.jcp.2019.05.024 Google Scholar

[80] 

Lutter, M., Ritter, C., & Peters, J., “Deep Lagrangian networks: Using physics as model prior for deep learning.,” arXiv preprint arXiv:1907.04490, (2019). Google Scholar

[81] 

Federal Aviation Administration, Visual inspection for aircraft, U.S. Department of Transportation, Advisory circular [online],” https://www.faa.gov/documentLibrary/media/Advisory_Circular/43–204.pdf Google Scholar

[82] 

Drury, C. G., & Watson, J., “Good practices in visual inspection.,” Human factors in aviation maintenance-phase nine, progress report, FAA/Human Factors in Aviation Maintenance.@, (2002) Google Scholar

[83] 

Swain, A. D., & Guttmann, H. E., Handbook of human-reliability analysis with emphasis on nuclear power plant applications. Final report, Sandia National Labs.1983). https://doi.org/10.2172/5752058 Google Scholar

[84] 

See, J. E., “Visual inspection reliability for precision manufactured parts.,” Human factors, 57 (8), 1427 –1442 (2015). https://doi.org/10.1177/0018720815602389 Google Scholar

[85] 

See, J. E., “Visual inspection: a review of the literature.,” Sandia National Laboratories, Albuquerque, New Mexico, (2012). Google Scholar

[86] 

Azuma, R. T., “A survey of augmented reality.,” Presence: Teleoperators & Virtual Environments, 6 (4), 355 –385 (1997). https://doi.org/10.1162/pres.1997.6.4.355 Google Scholar

[87] 

Wang, S., Zargar, S. A., Xu, C. & Yuan, F. G., “An Efficient Augmented Reality (AR) System for Enhanced Visual Inspection.,” 1575 –1582 (2019). Google Scholar

[88] 

Zargar, S. A. & Yuan, F. G., “Impact Diagnosis in Stiffened Structural Panels Using a Deep Learning Approach,” Structural Health Monitoring Journal, Accepted, (2020). Google Scholar

[89] 

Inoue, H., Harrigan, J. J., & Reid, S. R., “Review of inverse analysis for indirect measurement of impact force.,” Applied Mechanics Reviews, 54 (6), 503 –524 (2001). https://doi.org/10.1115/1.1420194 Google Scholar

[90] 

Chen, C., Li, Y., & Yuan, F. G., “Impact source identification in finite isotropic plates using a time-reversal method: experimental study.,” Smart Materials and Structures, 21 (10), 105025 (2012). https://doi.org/10.1088/0964-1726/21/10/105025 Google Scholar

[91] 

Worden, K., & Staszewski, W. J., “Impact location and quantification on a composite panel using neural networks and a genetic algorithm.,” Strain, 36 (2), 61 –68 (2000). https://doi.org/10.1111/str.2000.36.issue-2 Google Scholar

[92] 

Haywood, J., Coverley, P. T., Staszewski, W. J., & Worden, K., “An automatic impact monitor for a composite panel employing smart sensor technology.,” Smart Materials and Structures, 14 (1), 265 (2004). https://doi.org/10.1088/0964-1726/14/1/027 Google Scholar

[93] 

Ghajari, M., Sharif-Khodaei, Z., Aliabadi, M. H., & Apicella, A., “Identification of impact force for smart composite stiffened panels.,” Smart Materials and Structures, 22 (8), 085014 (2013). https://doi.org/10.1088/0964-1726/22/8/085014 Google Scholar

[94] 

Seno, A. H., & Aliabadi, M. H., “Impact localisation in composite plates of different stiffness impactors under simulated environmental and operational conditions.,” Sensors, 19 (17), 3659 (2019). https://doi.org/10.3390/s19173659 Google Scholar

[95] 

Fink, M., “Time reversal of ultrasonic fields. I. Basic principles.,” IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 39 (5), 555 –566 (1992). https://doi.org/10.1109/58.156174 Google Scholar

[96] 

Wu, F., Thomas, J. L., & Fink, M., “Time reversal of ultrasonic fields. Il. Experimental results.,” IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 39 (5), 567 –578 (1992). https://doi.org/10.1109/58.156175 Google Scholar

[97] 

Cassereau, D., & Fink, M., “Time-reversal of ultrasonic fields. III. Theory of the closed time-reversal cavity.,” IEEE transactions on ultrasonics, ferroelectrics, and frequency control, 39 (5), 579 –592 (1992). https://doi.org/10.1109/58.156176 Google Scholar

[98] 

Yeum, C. M., & Dyke, S. J., “Vision-based automated crack detection for bridge inspection,” Computer-Aided Civil and Infrastructure Engineering, 30 (10), 759 –770 (2015). https://doi.org/10.1111/mice.2015.30.issue-10 Google Scholar

[99] 

Feng, D., & Feng, M. Q., “Experimental validation of cost-effective vision-based structural health monitoring,” Mechanical Systems and Signal Processing, 88 199 –211 (2017). https://doi.org/10.1016/j.ymssp.2016.11.021 Google Scholar

[100] 

Hoskere, V., Narazaki, Y., Hoang, T., & Spencer Jr, B., “Vision-based structural inspection using multiscale deep convolutional neural networks,” arXiv preprint arXiv:1805.01055, (2018). Google Scholar

[101] 

Narazaki, Y., Hoskere, V., Hoang, T. A., & Spencer Jr, B. F., “Automated vision-based bridge component extraction using multiscale convolutional neural networks,” arXiv preprint arXiv:1805.06042, (2018). Google Scholar

[102] 

Spencer Jr, B. F., Hoskere, V., & Narazaki, Y., “Advances in computer vision-based civil infrastructure inspection and monitoring,” Engineering, (2019). https://doi.org/10.1016/j.eng.2018.11.030 Google Scholar

[103] 

Bao, Y., Chen, Z., Wei, S., Xu, Y., Tang, Z., & Li, H., “The State of the Art of Data Science and Engineering in Structural Health Monitoring,” Engineering, 5 (2), 234 –242 (2019). https://doi.org/10.1016/j.eng.2018.11.027 Google Scholar

[104] 

Xu, Y., Bao, Y., Chen, J., Zuo, W., & Li, H., “Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images,” Structural Health Monitoring, 18 (3), 653 –674 (2019). https://doi.org/10.1177/1475921718764873 Google Scholar

[105] 

Wei, S., Bao, Y., & Li, H., “Optimal policy for structure maintenance: A deep reinforcement learning framework,” Structural Safety, 83 101906 (2020). https://doi.org/10.1016/j.strusafe.2019.101906 Google Scholar

[106] 

V. Giurgiutiu, “Tuned Lamb Wave Excitation and Detection with Piezoelectric Wafer Active Sensors for Structural Health Monitoring,” Journal of Intelligent Material Systems and Structures, 16 (4), 291 –305 (2005). https://doi.org/10.1177/1045389X05050106 Google Scholar

[107] 

Z. Tian, L. Yu, and C. Leckey, “Delamination Detection and Quantification on Laminated Composite Structures with Lamb Waves and Wavenumber Analysis,” Journal of Intelligent Material Systems and Structures, 26 (13), 1723 –1738 (2015). https://doi.org/10.1177/1045389X14557506 Google Scholar

[108] 

B. Park, Y. K. An, and H. Sohn, “Visualization of Hidden Delamination and Debonding in Composites through Noncontact Laser Ultrasonic Scanning,” Composites Science and Technology, 100 10 –18 (2014). https://doi.org/10.1016/j.compscitech.2014.05.029 Google Scholar

[109] 

J. He and F. G. Yuan, “Lamb Wave-based Subwavelength Damage Imaging using the DORT-MUSIC Technique in Metallic Plates,” Structural Health Monitoring, 15 (1), 65 –80 (2016). https://doi.org/10.1177/1475921715623359 Google Scholar

[110] 

J. He and F. G. Yuan, “Damage Identification for Composite Structures using a Cross-correlation Reverse-time Migration Technique,” Structural Health Monitoring, 14 (6), 558 –570 (2015). https://doi.org/10.1177/1475921715602546 Google Scholar

[111] 

M. S. Harb and F. G. Yuan, “Damage Imaging using Non-contact Air-coupled Transducer/Laser Doppler Vibrometer System,” Structural Health Monitoring, 15 (2), 193 –203 (2016). https://doi.org/10.1177/1475921716636336 Google Scholar

[112] 

E. B. Flynn, S. Y. Chong, G. J. Jarmer, and J. R. Lee, “Structural Imaging through Local Wavenumber Estimation of Guided Waves,” Ndt & E International, 59 1 –10 (2013). https://doi.org/10.1016/j.ndteint.2013.04.003 Google Scholar

[113] 

Reu, P. L., Rohe, D. P., & Jacobs, L. D., “Comparison of DIC and LDV for practical vibration and modal measurements.,” Mechanical Systems and Signal Processing, 86 2 –16 (2017). https://doi.org/10.1016/j.ymssp.2016.02.006 Google Scholar

[114] 

Chu, T. C., Ranson, W. F., & Sutton, M. A., “Applications of digital-image-correlation techniques to experimental mechanics.,” Experimental mechanics, 25 (3), 232 –244 (1985). https://doi.org/10.1007/BF02325092 Google Scholar

[115] 

Luo, P. F., Chao, Y. J., Sutton, M. A., & Peters, W. H., “Accurate measurement of three-dimensional deformations in deformable and rigid bodies using computer vision.,” Experimental mechanics, 33 (2), 123 –132 (1993). https://doi.org/10.1007/BF02322488 Google Scholar

[116] 

Luo, P. F., Chao, Y. J., & Sutton, M. A., “Application of stereo vision to three-dimensional deformation analyses in fracture experiments.,” Optical Engineering, 33 (3), 981 –991 (1994). https://doi.org/10.1117/12.160877 Google Scholar

[117] 

Sutton, M. A., Matta, F., Rizos, D., Ghorbani, R., Rajan, S., Mollenhauer, D. H., … & Lasprilla, A. O., “Recent progress in digital image correlation: background and developments since the 2013 WM Murray Lecture.,” Experimental Mechanics, 57 (1), 1 –30 (2017). https://doi.org/10.1007/s11340-016-0233-3 Google Scholar

[118] 

Pan, B., “Digital image correlation for surface deformation measurement: historical developments, recent advances and future goals.,” Measurement Science and Technology, 29 (8), 082001 (2018). https://doi.org/10.1088/1361-6501/aac55b Google Scholar

[119] 

Helfrick, M. N., Niezrecki, C., Avitabile, P., & Schmidt, T., “3D digital image correlation methods for full-field vibration measurement.,” Mechanical systems and signal processing, 25 (3), 917 –927 (2011). https://doi.org/10.1016/j.ymssp.2010.08.013 Google Scholar

[120] 

Trebuňa, F., & Hagara, M., “Experimental modal analysis performed by high-speed digital image correlation system.,” Measurement, 50 78 –85 (2014). https://doi.org/10.1016/j.measurement.2013.12.038 Google Scholar

[121] 

Beberniss, T. J., & Ehrhardt, D. A., “High-speed 3D digital image correlation vibration measurement: Recent advancements and noted limitations.,” Mechanical Systems and Signal Processing, 86 35 –48 (2017). https://doi.org/10.1016/j.ymssp.2016.04.014 Google Scholar

[122] 

Chang, H. Y., Damage Visualization of Scattered Ultrasonic Wavefield via Integrated High-speed Camera System, (2019). Google Scholar

[123] 

H. Y. Chang and F. G. Yuan, “Damage Imaging in a Stiffened Curved Composite Sandwich Panel with Wavenumber Index via Riesz Transform,” Structural Health Monitoring, 1475921719858432 (2019). Google Scholar

[124] 

Tokognon, C. A., Gao, B., Tian, G. Y., & Yan, Y., “Structural health monitoring framework based on Internet of Things: A survey,” IEEE Internet of Things Journal, 4 (3), 619 –635 (2017). https://doi.org/10.1109/JIoT.6488907 Google Scholar

[125] 

Abdelgawad, A., & Yelamarthi, K., “Internet of things (IoT) platform for structure health monitoring,” Wireless Communications and Mobile Computing, (2017). https://doi.org/10.1155/2017/6560797 Google Scholar
© (2020) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.
Fuh-Gwo Yuan, Sakib Ashraf Zargar, Qiuyi Chen, and Shaohan Wang "Machine learning for structural health monitoring: challenges and opportunities", Proc. SPIE 11379, Sensors and Smart Structures Technologies for Civil, Mechanical, and Aerospace Systems 2020, 1137903 (23 April 2020); https://doi.org/10.1117/12.2561610
PROCEEDINGS
23 PAGES + PRESENTATION

SHARE
Advertisement
Advertisement
Back to Top