WO2025191082A1 - A computer implemented method and system for health monitoring of structures - Google Patents
A computer implemented method and system for health monitoring of structuresInfo
- Publication number
- WO2025191082A1 WO2025191082A1 PCT/EP2025/056915 EP2025056915W WO2025191082A1 WO 2025191082 A1 WO2025191082 A1 WO 2025191082A1 EP 2025056915 W EP2025056915 W EP 2025056915W WO 2025191082 A1 WO2025191082 A1 WO 2025191082A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- dynamic
- poles
- implemented method
- computer implemented
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M5/00—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings
- G01M5/0066—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings by exciting or detecting vibration or acceleration
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M5/00—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings
- G01M5/0008—Investigating the elasticity of structures, e.g. deflection of bridges or air-craft wings of bridges
Definitions
- the present invention belongs to the field of monitoring, surveillance, conservation and maintenance of civil engineering structures, such as bridges. More particularly, the solution herein disclosed refers to a computer implemented method and system for health monitoring structures.
- Inspection activities are normally done with frequencies measured in years, sometimes larger than 5 years.
- -Radar-based methods That detect cracking, anchor exploration, reinforcement location, detection of objects in the ground, buried installations and foundations.
- -Methods based on X-rays That define location of reinforcement diameter and distribution, inspection of prestressing cables, cracking and homogeneity of concrete.
- thermography That detect cracks, crevices and other defects in the concrete.
- -Optical fiber That measure temperatures, deformations, displacement acceleration.
- the state-of-the-art regarding electronic devices deployed in different structures is limited to data loggers and gateways that only perform data storage, data collection and data forwarding tasks, the cloud services being defined as the key enablers of the technology.
- the cloud computing approach assumes that all the structure locations have internet connection with enough broadband to send raw data into the cloud.
- Reality says that structures such as bridges are often located in remote areas where connectivity is limited.
- scalability challenges are created, as communication infrastructures and cloud services deployment generalization for many structures is expensive and not reliable enough.
- the method and system may need between 10 ms and 2 s to produce a key performance indicator (KPI) from the time instant an impulse is generated.
- KPI key performance indicator
- the KPI may represent a change of conditions over time, a health index (HI) and/or the remaining useful life (RUL).
- HI health index
- RUL remaining useful life
- the method and system operate with availability 24/7 and provide a set of KPIs.
- the HI or RUL obtained from the KPI can be used as a maintenance decision making insight by a nonexpert end user. They also reduce the level of subjectivity and dependence on the traditional experience of a human inspector, providing objective information as actionable insights.
- the produced KPIs are artificial intelligence-based values, which do not need previous knowledge from the structure, avoiding the need of training and improving scalability.
- the method learns from the current state of the structure, meaning that no failure situation needs to be modelled or tested.
- the KPI is obtained through the application of a model called concept drift approach.
- This model can be automatically updated using artificial intelligence technology to increase system availability.
- the method and system integrate data stream processing at the edge in terms of its preprocessing and processing. This approach helps to avoid large infrastructure or electronic instrument costs improving the system scalability to any infrastructure.
- Edge devices typically have limited processing power, memory, and storage capacity. Edge devices are responsible for collecting, processing, and analyzing data in real-time and transmitting only the relevant data to an external data integration and visualization system.
- a dynamic response of a structure represents the behaviour of dynamic variables when the structure is subjected to an excitation caused by external phenomena.
- a structure may be excited by different phenomena.
- the structure may be excited by different phenomena. Examples of phenomena that can cause the excitation of the structure are: the passage of vehicles (when the structure is, for example, a bridge or a viaduct), gusts of wind, a force exerted by a material when it is loaded into the structure (when the structure is, for example, a dam, a silo or a warehouse in which material is unloaded), a force or slapping of sea waves, or an earthquake.
- variables or parameters providing data related to the dynamic response of the structure to an external excitation are vibration, displacement, inclination and force, among others.
- Data (such as captured values) associated to dynamic variables is referred to as dynamics data.
- the dynamic response of a structure is usually expressed as a set of pairs of natural frequencies and damping factors.
- Contextual variables are usually environmental ones which may cause a temporal change in, for example, a characteristic of the material of which the structure is made or the weight of the structure, which in turn causes a change in the mechanical response of the structure, thus altering the dynamic response of the structure.
- Contextual variables related to the environment surrounding the structure are: surface and environment temperature, environment humidity, wind speed and wind direction, rain drops, snow, frost, ice or season of the year, among others.
- Data (such as captured values) associated to contextual variables is referred to as contextual data.
- Contextual variables can optionally (typically depending on the circumstances, such as type of structure, location and/or environmental situation) affect the structure and alter its dynamic response.
- the combination of dynamic and contextual variables causes a certain response of the structure when the structure is subjected to an excitation.
- the method is able to discriminate whether a change in the response of the structure to an impulse is due to a change in the health of the structure of simply to a change in an environmental variable, such as temperature.
- sensors refers broadly to sensors and any other devices that generate data from measurements associated or originated at the structure under supervision.
- the sensors are related to, or can be seen as, the structure instrumentation.
- instrumentation is sometimes used to refer to the sensors.
- the general concept of the proposed method is as follows: When external phenomena excite variables affecting a structure under supervision, creating an impulse, which typically causes an oscillation, a plurality of sensors capture the behaviour of these variables.
- the sensors typically capture values of physical parameters periodically (at a certain sampling frequency, which may be between about 0.5 and 1 ,000 Hz).
- Non-limiting examples of sensors that may be used are accelerometers, inclinometers, extensometers, strain gauges, force transducers and GPSs.
- a communication interface builds a data stream comprising instant values of these variables. For example, accelerometers measure vibration, inclinometers measure tilt, extensometers measure distance, strain gauges measure deformation, force transducers measure force, GPSs measure distance, and so on.
- the data stream comprises at least dynamics data. It can optionally comprise also contextual data when there are sensors capable of capturing values of contextual variables.
- the objective is to record impulses created by external phenomena (such as normal traffic over a bridge) through continuous monitoring and avoiding interruptions in the normal use of the structure (such as traffic interruptions).
- the method of this disclosure obtains a dynamic response of the structure from the captured measurements: An impulse caused by external factors creates an oscillation in the structure.
- the free load response of the oscillation (that is to say, the response when the impulse has ended) is an attenuated oscillation which is directly related to the dynamic characteristics of the structure, and therefore can be considered a dynamic response of the structure.
- the proposed method differentiates stable regimes and transient regimes in the dynamic response, and extracts from the transient regime the free load response of the structure to the excitation. This is done by applying a transient analysis technique. From the free load response, the dynamic response of the structure will be estimated.
- the dynamic response is a set of pairs of natural frequencies and damping factors (poles) of the structure called poles. With the poles and optionally with the contextual data (if there are contextual data), a baseline model of the structure is generated.
- a baseline model of the structure needs to be generated.
- a model which represents the current behaviour of the structure needs to be created.
- This model represents the actual dynamic behaviour of the structure.
- the employed model is a stochastic baseline model (in opposition to deterministic models, based for example on deep learning techniques or on neural networks).
- a probabilistic relationship between different natural frequencies and damping factors representing the dynamic response of the structure is learned. These pairs or poles represent the estimated dynamic variables of the structure, and therefore represent the dynamic response of the structure.
- contextual data preferably conditioned contextual data
- conditioned contextual data represents the available contextual variables affecting the structure, such as temperature, humidity, etc.
- the generation of the stochastic baseline model does not require any previous training, such as a specific training under real failure conditions or training with synthetic failure data created ad hoc for the structure under supervision.
- the technique used to learn the baseline model without previous training is called nonsupervised learning.
- the result of the probability-based (stochastic) ML baseline model is a set of parameters, which are conditional probability distributions associated with each features (natural frequencies, damping factors and (if any) contextual variables).
- a probability-based ML baseline model can have a structure, which is a directed acyclic graph (DAG) that expresses conditional independencies and dependencies among random variables associated with features.
- DAG directed acyclic graph
- Parameters and structure enable a specific measure of fitness which depends on the algorithm used to generate the baseline model.
- the KPI can be monitored instantaneously and/or, for example, by a concept drift detection algorithm to decide if a group of instances (online poles instantly obtained from recently obtained measurements captured by sensors in a structure) fits into the baseline model or the baseline model needs to be updated.
- the KPI can be sent to a supervision platform, such as a dashboard (usually at a remote location).
- the KPI can optionally be interpreted as a health index, where is evolution can be used to estimate the remaining useful life (RUL) of the structure under supervision.
- the RUL can be then sent to the dashboard for human assessment.
- a first aspect of this disclosure is a computer implemented method for on-line monitoring a structure.
- the method comprises the steps of: receiving at least dynamics data of dynamic variables representing a dynamic response of the structure from a plurality of sensors, the dynamic response of the structure being generated when the structure has been subjected to an excitation; detecting an impulse created by said excitation by identifying a free load response of the structure from said at least dynamic data; obtaining a set of poles comprising pairs of a natural frequency and a damping factor of the structure from the free load response; generating, using a non-supervised machine learning algorithm, a probability-based baseline model for the structure using a plurality of said poles obtained along a certain period of time, the probability-based baseline model associating in a probabilistic way the dynamic variables affecting the structure and providing a plurality of baseline parameters; every time a new impulse is detected and a new set of poles are obtained from the free load response of said impulse, feeding the new set of poles to a concept drift algorithm to obtain a KPI of the
- the method further comprises, together with receiving at least dynamics data from the plurality of sensors, receiving from the plurality of sensors contextual data of contextual variables representing the environment surrounding the structure; wherein the step of generating, using a machine learning algorithm, a probability-based baseline model for the structure, also uses said contextual data.
- the method further comprises, prior to identifying a free load response of the structure from said dynamic data, carrying out data conditioning of the dynamics data received from the plurality of sensors, said data conditioning comprising data cleaning and real-time data imputation.
- the method further comprises, prior to identifying a free load response of the structure from said dynamic data, carrying out data conditioning of the contextual data received from the plurality of sensors, said data conditioning comprising data cleaning and real-time data imputation.
- the method further comprises, once a KPI has been obtained, monitoring a fitness level to decide if the new online poles fit into the baseline model and, if a certain number of outliers are identified, providing concept drift feedback to the machine learning algorithm to update the probability-based baseline model.
- the method further comprises estimating a remaining useful life of the structure based on a health index interpreted from the KPI.
- the excitation the structure is subjected to is generated by at least one of the following phenomena: a passage of at least one vehicle on the structure, a gust of wind, a force exerted by a material when loaded into the structure, a force or slapping of sea waves, an earthquake and any combination thereof.
- the variables or parameters providing data related to the dynamic response of the structure to an external excitation are at least one of: vibration, displacement, inclination and force.
- the variables or parameters providing contextual data related to the environment surrounding the structure are at least one of: surface and environment temperature, environment humidity, wind speed and wind direction, rain drops, snow, frost, ice or season of the year.
- the method further comprises using a technique for carrying out the real-time data imputation of the data streams that is selected from a group comprising: last-value technique, mean value technique, interpolated value technique and any combination thereof.
- detecting an impulse created by said excitation by identifying a free load response of the structure from said at least dynamic data is done by implementing at least one classifier configured to detect transient regimes.
- two classifiers configured to detect transient regimes are implemented.
- detecting an impulse created by said excitation is done as follows: calculating from the dynamics data a sample moment-time representation curve rc(t) of the dynamical system in a previously selected time window th; computing the arc length l r (t) of the SMTR curve rc(t); generating a piecewise function hTsi(t) using a threshold l r o and the arc length l r (t); classifying the system’s time periods as either transitory or stationary; using the SMTR curve rc(t) of the dynamical system, computing its matrix of derivatives Dc(t); calculating time segments where the determinant of the matrix of derivatives Dc(t) is zero; if the determinant obtained is zero, segmenting the curve representing the dynamic system into flat curves r y j(t) and calculating their flat curvature kn(t); if the determinant obtained is not zero, computing generalized curvatures ki(t) of the
- obtaining a set of natural frequencies and a damping factor of the structure from the free load response is carried out by a system identification algorithm implementing system identification techniques selected from a group comprising system identification techniques based on deterministic algorithms, system identification techniques based on probabilistic algorithms and any combination thereof.
- the machine learning baseline model algorithm comprises at least one of gaussian mixture models, gaussian Bayesian networks, semiparametric Bayesian networks, and any combination thereof.
- the method further comprises triggering an alarm when the trend in a health evolution of the structure is above a certain threshold.
- the structure to be supervised is a bridge, a viaduct, a dam, a warehouse, a silo, a marine platform, or any other building or infrastructure requiring structural health surveillance. It is in particular suitable to monitor structures of difficult access for human operators.
- a second aspect of this disclosure is a device for on-line monitoring a structure.
- the device comprises: means for receiving at least dynamics data of dynamic variables representing a dynamic response of the structure from a plurality of sensors, the dynamic response of the structure being generated when the structure has been subjected to an excitation; means for detecting an impulse created by said excitation by identifying a free load response of the structure from said at least dynamic data; means for obtaining a set of poles comprising pairs of a natural frequency and a damping factor of the structure from the free load response; means for generating, using a machine learning algorithm, a probability-based baseline model for the structure using a plurality of said poles obtained along a certain period of time, the probability-based baseline model associating in a probabilistic way the dynamic variables affecting the structure and providing a plurality of baseline parameters; means for feeding, every time a new impulse is detected and a new set of poles are obtained from the free load response of said impulse, the new set of poles to a concept drift algorithm to obtain a key
- the device is an edge device.
- the hardware of the edge device includes: one or more central processing units (CPU) or at least one core thereof, a multi-core central processing unit (multi-core CPU), an embedded circuit (e.g. a system-on-chip, a multiprocessor system-on-chip) -e.g. Zynq, MPSoC by Xilinx, a graphics processing unit (GPU) or any combination thereof.
- CPU central processing units
- multi-core CPU multi-core central processing unit
- embedded circuit e.g. a system-on-chip, a multiprocessor system-on-chip
- MPSoC by Xilinx
- GPU graphics processing unit
- the edge device further includes a data storage device including at least a non-volatile memory, such as a hard disk drive (HDD) or preferably a solid-state drive (SSD).
- a non-volatile memory such as a hard disk drive (HDD) or preferably a solid-state drive (SSD).
- a third aspect of the disclosure is a system for on-line monitoring a structure.
- the system comprises: a plurality of sensors attachable to the structure to be supervised, the sensors being capable of capturing data related to a dynamic response of the structure when it is subjected to an excitation; a device according to the second aspect of this disclosure, wherein the device is an edge device situated in the vicinity of the structure to be supervised; and a remote system configured to receive from the edge device information about the health of the structure to be supervised.
- the system comprises a plurality of sensors attachable to the structure to be supervised, the sensors being capable of capturing data related to a dynamic response of the structure when it is subjected to an excitation, and optionally contextual data that can alter the dynamic response of the structure; a device according to the second aspect of the invention, wherein the device is an edge device situated in the vicinity of the structure to be supervised; and a remote system configured to receive from the edge device KPI information about the structure to be supervised.
- the sensors are in communication with the edge device. This communication may be implemented either through a wired connection or through a wireless connection, such as WiFi, Bluetooth, 5G or based on any other suitable wireless communication protocol, typically a short-distance communication protocol.
- Figure 1 shows a block diagram of a system for monitoring a structure according to a particular embodiment of the invention.
- Figure 2 schematically shows a portion of a structure including a plurality of sensors according to a particular embodiment of the invention.
- Figure 3 shows a block diagram of a data stream conditioning stage performed at an edge device according to an embodiment of the invention.
- Figure 4 shows a block diagram of a preprocessing stage performed at an edge device according to an embodiment of the invention.
- Figure 5A shows a scheme of a proposed methodology for transient regimes and stationary regimes classification according to an embodiment of the invention.
- Figure 5B shows impulses occurring on a bridge over time as a result of passing traffic according to an embodiment of the invention.
- the square signals depict the impulse detection classifier, responsible for identifying and extracting the corresponding response.
- Figure 6A shows a block diagram scheme of an implementation of the stage of impulse detection (classification of transient regimes and stationary regimes) according to a preferred embodiment of the invention.
- Figure 6B shows an exemplary validation assessment of the proposed classifiers for a studied dynamical system under switching step input functions.
- Figure 7 shows the response of a single impulse obtained at the stage of impulse detection. From this response, the relevant natural frequencies and damping factors of the structure can be calculated.
- Figures 8A-8B show estimated natural periods (inverse of frequencies) and damping factors, respectively, obtained for an exemplary bridge between October 2019 and May 2022.
- Figures 9A-9D show the estimation distribution for first, second and third frequencies of an exemplary bridge, showing estimation algorithm stability.
- Figure 10 shows a block diagram of a processing stage performed at an edge device according to an embodiment of the invention.
- Figure 11 shows exemplary graphics representing the impulse detection stage, the new features extraction stage and the stage for generation of probability baseline model.
- Figure 12 shows an example of probability-based baseline model generated in an embodiment of the invention.
- Figure 13 shows the graphics of Figure 11 , plus the result of the concept drift detection stage according to embodiments of the invention.
- Figure 14 shows an example of KPI obtained after comparing new observations (impulses) against the baseline model: if a new impulse fits the model, the KPI is low value (100% fitting indicates KPI » 0), while if a new impulse fits poorly, the KPI is a large value (0% fitting indicates KPI » «).
- Figure 15 shows a flow diagram of the method for monitoring a dynamic response of a structure, according to a particular embodiment of the invention.
- Figure 16 shows an example of a structure (bridge) being monitored by means of the disclosed method and system.
- Figure 17A-E show real data and results obtained for span 1 of the bridge of Figure 16.
- FIGS 18A-18H show an online dashboard for span 1 of the bridge of Figure 16.
- FIG. 1 A block diagram representing an architecture of a system 1 for monitoring a structure according to a particular embodiment of the invention is depicted in Figure 1.
- the shown architecture is defined by three architectural levels: sensors 10 for capturing data from the structure under supervision and its environment, an edge device 20 (also referred to as edge node) and a remote system or platform 30.
- the system 1 can monitor a dynamic response of a structure under supervision and provide an indication of the healthiness of the structure.
- the dynamic response of the structure is generated when the structure is subject to an excitation caused by external phenomena.
- Contextual variables can optionally affect the structure and alter its dynamic response.
- the system 1 is able to collect two types of data: dynamic data provided by dynamic variables (such as vibration, displacement, inclination and force, also generally referred to as data related to the structure dynamics) and contextual data provided by contextual variables (such as temperature, humidity and wind speed and direction, also generally referred to as data related to the environment surrounding the structure).
- dynamic data such as vibration, displacement, inclination and force
- contextual data such as temperature, humidity and wind speed and direction, also generally referred to as data related to the environment surrounding the structure.
- the main requirement for the sensors 10 is to be able to capture the required variables with the right sensitivity. In other words, the instrumentation must capture the response of the structure (for example, its oscillation, in which case sensors are typically accelerometers) caused by any impulse or excitation.
- the sensors 10 must be suitable to work outdoors to avoid measuring deviations because of environmental changes.
- Figure 2 schematically shows a portion of a structure (in this case, a viaduct) in which a plurality of accelerometers and inclinometers have been installed to continuously (i.e. at a certain sampling frequency) measure the vibration (oscillation) and tilt, respectively, suffered by the structure.
- a structure in this case, a viaduct
- accelerometers and inclinometers have been installed to continuously (i.e. at a certain sampling frequency) measure the vibration (oscillation) and tilt, respectively, suffered by the structure.
- the sensors are in communication (wired or wireless) with the edge device 20.
- the data captured by the sensors 10 is provided to the edge device 20 through a communication interface 201 implementing a communication layer.
- Communication interface 201 is software-based and is implemented to enable working with different types of sensors or systems of sensors 10.
- Communication interface 21 provides an interface module for every communication protocol needed by the sensors. Non-limiting examples of these interfaces are Integrated Electronics Piezo-Electric (IEPE) standard, MicroElectromechanical Systems (MEMS) standard, User Datagram Protocol (UDP) and Transfer Control Protocol (TCP), among others. These interfaces guarantee the quality of the data provided by the sensors 10 to build a useful data stream 202 that can be consumed by subsequent stages. Each interface module needs to maintain data integrity, avoiding losing data packages (i.e. sanity checks) and performing a proper time stamping.
- IEPE Integrated Electronics Piezo-Electric
- MEMS MicroElectromechanical Systems
- UDP User Datagram Protocol
- TCP Transfer Control Protocol
- the data stream 202 outputted by the communication interface 201 can comprise two types of data captured by the sensors 10: dynamics data related to dynamic variables and optionally contextual data related to contextual variables.
- dynamics data related to dynamic variables
- they can be delivered in a single data stream or in two separate data streams (a dynamics data stream and a contextual data stream). They represent the dynamic response of the structure to an excitation. In other words, they represent the response (for example, oscillation) to an impulse generated in the structure as a consequence of the excitation to which it is subjected.
- a single data stream 202 is depicted, as explained, it can be implemented with two separate data streams: a dynamic data stream and a contextual data stream.
- the data of the two data streams must be synchronized so that they can be used in subsequent stages.
- all the relevant signals provided by sensors 10 are synchronized into a single or multiple data streams, depending on the subsequent analysis to be performed.
- the nature of the data forming the data stream(s) 202 can be very divers (for example, the sampling frequency at which different variables are captured by sensors is very divers and may vary between 0.5 Hz and 1 ,000 Hz depending on the sensor and type of variable), this data is preferably conditioned. Therefore, the data stream(s) 202 is(are) preferably taken to a data stream conditioning module 203, in which sanity checks are applied. For example, it is important to verify that the jitter (difference between timestamps) is constant because artificial intelligence (IA) algorithms used later assume constant time, and therefore variable jitter could imply precision problems. Or data contents should be checked to verify that they are coherent.
- IA artificial intelligence
- conditioning techniques are data cleaning and real-time imputation applied to the data comprised in the data stream 202.
- a conditioned data stream may take the form of a table including a timestamp and a string of values of different parameters (vibration, temperature, etc.) coming from the different sensors 10.
- a block diagram of the data stream conditioning stage 203 is shown in Figure 3, in which the inputs 202 and outputs 204, 205 of the stage are depicted.
- the conditioned data stream has been split into two data streams, a conditioned dynamics data stream 204 and a conditioned contextual data stream 205, because in the shown implementation of the edge device 20 the data coming from the two types of variables (dynamic ones and environment ones, if any) are processed separately.
- stage 66 the time segments where the determinant of the matrix is zero are calculated.
- stage 67 If the determinant obtained in stage 66 is zero, the curve representing the dynamic system is segmented into flat curves r y j(t) and their flat curvature kn(t) is calculated (stage 67). This is done as follows:
- the computation of a matrix determinant is a common procedure that can be done by several methods. For instance, one of the most optimal methods is based on performing the LU decomposition to the matrix T>g and computing the product of the elements of the diagonals of the matrices L and U as follows: where TMis the dimension of the square matrix whereas s is the number of row exchanges during the decomposition. Further information of this computation can be surveilled in Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2001). Introduction to Algorithms, MIT Press and McGraw-Hill, 2001. ISBN, 262032937, 636- 640.
- the SMTR curve of a dynamical system can be written as 1,2, ... y are the samp
- the segmentation of the curve into plane curve will consist of a set of plane curves written as -
- e curvature of each plane curve can be written as
- the generalized curvatures ki(t) of the SMTR curve rc(t) are computed (stage 68), preferably using the Frenet-Serret formulas.
- stage 69 a piecewise function k c (t) is calculated, which stores the magnitude of the generalized and flat curvatures of the studied system.
- a piecewise function hTsii(t) is generated using a threshold k c o and the magnitude of the curvatures ki(t).
- the piecewise function hTsii(t) enables the identification of time periods when the outputs of the dynamic system behaves as weakly stationary stochastic processes.
- stage 71 the piecewise function hTsii(t) is used to classify the system’s time periods as either transitory or stationary.
- This approach enables the development of a classifier based on the curvatures of the sample moment-time representation curve rc(t).
- This methodology has been validated with linear, non-linear, and discontinuous simulated systems and compared with existing methodologies in the literature.
- the proposed classifiers (stages 63 and 70) outperform existing methodologies when identifying transient regimes in systems with differing behaviors.
- the algorithm is able to work with a real time data stream 204 integrated into the edge node 20.
- one of the tests performed during the validation assessment of the proposed classifiers consisted in simulating a dynamical system written as
- the three sensors y(t) of the dynamic system were used to validate the proposed method by varying the two sources of excitations
- the method was validated with switching step functions as inputs.
- An example of the algorithm performance is shown in Figure 6B.
- the sensors behavior and the geometrical properties of the STMR curve are shown in Figure 6B.
- the classification of the transient and stationary regimes are depicted in this plot.
- the free load response 208 of the structure under supervision has been obtained (subblock 207 in preprocessing block 206).
- the free load response of the structure provides information of the dynamic features of the structure
- the free load response represents the signal(s) captured by the sensor(s) 10. These signals are typically not directly interpreted in the sense that they do not enable to directly infer from these signals the dynamic response of the structure.
- a set of poles can be estimated, and from these poles the natural frequencies and damping factors can be obtained. Therefore, once the impulse (for example represented in Figure 7 (left)) has been extracted at stage 207, the free load response 208 is consumed by a stage 209 of new features extraction. These new features are the poles from which the natural frequencies and damping factors can be obtained.
- This package or stage 209 is to produce a new data stream formed by only explainable variables (versus the free load response representing the data captured by the sensors, which is not directly interpreted by a human operator).
- Figure 7 shows an exemplary single impulse response from which the natural frequencies and damping factors of the structure can be obtained.
- the stage of new features extraction 209 replaces conventional analysis, such as FEM analysis, which only works offline, and enables a real-time analysis of the dynamic response of the structure. Additionally, it produces variables, of which their actual value can be inspected by field experts, such as structural engineers. This distinctive characteristic is called explainability of the variables. The importance of explainability is intimately related to the interpretation done by an expert, as it guarantees that the Machine Learning analysis in the next step only consumes relevant and non-redundant data. This improves the system robustness and critically reduces the amount of false positives during the next phase.
- system identification The technique implemented to perform the new feature extraction preprocessing package or stage 209 is called system identification.
- system identification iPARSIMonious-based Subspace Identification [2], [3], ARX, ARIMAX, ARIMA, OE, BOX Jenkins, Spectral Identification [4], HAVOK [5], among others
- system identification iPARSIMonious-based Subspace Identification [2], [3], ARX, ARIMAX, ARIMA, OE, BOX Jenkins, Spectral Identification [4], HAVOK [5], among others
- system identification iPARSIMonious-based Subspace Identification [2], [3], ARX, ARIMAX, ARIMA, OE, BOX Jenkins, Spectral Identification [4], HAVOK [5], among others
- stochastic system identification developed to work in real time. This new approach provides a related probability of dynamic response variable estimation, exponentially increasing the explainability of the result.
- Block 209 for extraction of new features (poles) encompasses novel system identification techniques to robustly identify and characterize the behavior of the structure’s modal parameters over time.
- the poles are extracted using HAVOK.
- poles can be extracted using a stochastic approach.
- the proposed scheme identifies the modal parameters (poles) employing HAVOK (the Hankel alternative view of the Koopman operator) for pulses exhibiting similar linear behavior in bridge responses.
- HAVOK provides a mathematical framework for investigating the behavior of nonlinear systems by approximating the Koopman operator. This approach enables the identification of linear and non-linear behavior in dynamic systems. After extracting frequency features over time, they can be characterized as stochastic processes. These processes can be fitted to various probabilistic distributions.
- Bayesian inferences approaches are used, which treat model parameters as random variables and incorporates prior knowledge to provide a more robust estimate, essentially self-regularizing the model.
- Bayesian inference techniques it is possible to characterize the probabilistic parameters of these features by treating them as regularizable random variables. This approach accommodates changes that might arise due to varying operating conditions, ensuring consistent and robust monitoring.
- Figure 7 shows an example of an impulse response from which 3 poles have been obtained by applying the former approach.
- Figures 8A-8B show the evolution of pole n° 1 (the most relevant in time).
- Figure 9A shows instant natural frequencies of the three obtained poles as captured for a certain time period.
- the histograms of Figures 9B-9D represent the distribution of the values of the three poles (in this case, unimodal Gaussian distribution).
- the processing block or module 212 is the block which follows the preprocessing module 206, It is depicted in Figures 1 and 10.
- the first substage or package 213 of the processing module 212 implements the generation of a probability-based model which represents the actual dynamic behavior of the structure under supervision during a specific time span, such as week, month, year or several years. It is a probability-based Machine Learning (ML) model which models baseline parameters. This model associates in a probabilistic way the variables affecting the structure (dynamic variables represented by poles (natural frequencies and damping factors) and optionally contextual variables), thus describing how the variables behave and relate to each other. The number of used poles to build the probability-based baseline model determines the dimensions of the model.
- ML Machine Learning
- This probability-based ML baseline model 213 describes the structural behavior of the structure at the time of measurement, as it captures all the possible states from the variables and stores them in probability distributions, which guarantee the generalization and scalability of the model(s).
- there may be more than one baseline model For example, when the structure under supervision is located in an environment having extreme weather conditions (i.e. temperatures), there may be a summer baseline model and a winter baseline model. It can also happen that the first time a baseline model is created, the structure is in good condition, but after several years it has suffered certain degradation, in which case it may be necessary to have a second model reflecting the current degradation of the structure.
- the probability distributions and the relationship between variables are defined by the baseline parameters.
- the type of ML model used determines the type of baseline parameters which will represent the actual behaviour of the structure.
- the baseline parameters are a Gaussian mixture comprising the following parameters: a number of components, a means vector and a covariance matrix.
- Figure 11 shows graphically the results of the impulse detection stage 207, new features extraction 209 and generation of probability baseline model 213.
- the impulse detection stage 207 is represented by a plurality of free load responses 208 obtained for a certain structure along a certain time period.
- the new features extraction stage 209 is represented by the baseline poles (natural frequencies and damping factors) extracted for each free load response 208.
- the generation of the baseline model 213 is represented as a data clustering graph showing two clusters based on the distribution of natural periods (or natural frequencies) and damping factors.
- two probability-based baseline models have been obtained (for example one representing the summer season and the other one representing the winter season).
- the baseline model 213 is also fed with an additional input, which is the concept drift feedback 215, explained in detail later.
- the concept drift feedback 215 indicates that the baseline model 213 needs to be updated with a specific subset of the data stream 202. This information helps to launch the baseline model updated with previous knowledge.
- the result of the probability-based ML baseline model 213 is a set of model parameters 214 of reference that enable a specific measure of fitness which depends on the algorithm used to generate the baseline model 213. These model parameters 214 describe everything needed from the baseline (print of the structure) to evaluate said fitness in a next step of the analysis.
- the generation of the baseline model 213 can be done using different types of algorithms, as described below. All these algorithms associate the variables affecting the structure in a probabilistic way, thus enabling to obtain probabilistic distributions of the possible states of each variable.
- the structural baseline is built using the information available from the data stream that is consumed. As described before, the main goal of this baseline model is to capture all specific multidimensional behaviours (i.e. behaviours defined by more than one variable) at a measuring time interval. As in real life these behaviors have natural variability, the main characteristic of the algorithms is their ability to store these behaviours into probability distributions.
- GMM Gaussian mixture model
- a baseline model obtained using a GMM has the following parameters: a vector containing the number of components (number of normal distributions of a mixture, a mixture being a combination of two or more Gaussian distributions), a vector containing the means per each component and a matrix of variance-covariances.
- the GMM is made of a finite number of components, each of which follows a multivariate normal distribution, and will be learned from the data (natural frequencies and damping factors) using an Expectation- Maximization (EM) algorithm.
- the EM algorithm is iterative.
- the EM algorithm is a statistical algorithm used to estimate the parameters of a probability distribution when some of the variables are missing or unobserved.
- the algorithm iteratively computes two steps, an expectation step (E-step) and a maximization step (M-step), to update the estimates of the parameters until convergence.
- E-step the algorithm computes the probability distribution of the unobserved variables given the observed data and the current estimates of the parameters.
- This step involves calculating the conditional expectation of the complete data log-likelihood, which is a measure of how well the observed data fits the proposed probability distribution.
- the algorithm updates the estimates of the parameters based on the probability distribution computed in the E-step.
- This step involves maximizing the expected value of the complete data loglikelihood with respect to the parameters.
- the EM algorithm tries to find the parameters (weight of each component of the mixture, vector of means and variancecovariance matrix for each component) that maximize the likelihood of the data given the GMM.
- the use of this algorithm is oriented to specific structure analysis, as it is the simplest way to deploy the system in terms of implementation and computational requirements. Analysis such as, couple bridge analysis and single beam uniaxial analysis, among others, are good examples of a viable GMM application.
- a Gaussian Bayesian network [7] is used.
- a continuous Bayesian network is a Bayesian network in which a conditional probability distribution (CPD) of each variable (inputs, such as the baseline poles 210) is represented using a continuous probability distribution.
- a GBN is a Bayesian network in which all CPDs are defined using a linear Gaussian CPD. For this model, it will be assumed that the inputs coming from the previous steps follow Gaussian behaviors.
- a GBN model assumes that each variable conditional on its parent variables follows a Gaussian density with an expectation that turns out to be a linear combination of the states of its parent variables, and with a variance that is independent of its parents.
- the learning of the GBN model from different input data is carried out by means of a heuristic approach, based on the hill climb algorithm, which tries to maximize the likelihood of the data given the model.
- the operators used by the heuristic is based on adding, deleting or changing the direction of an arc of the current structure that the model represents.
- Each candidate structure is evaluated with the likelihood, without any type of penalty for its complexity.
- a bootstrap sampling is carried out (based on n samplings with replacement of the base of original data and with its same size). The most frequent arcs among all the induced structures will be those that constitute the GBN structure.
- This algorithm is oriented to specific structure analysis in which it is important to evaluate the relationships between the variables.
- This algorithm is more complex to implement and requires low-to-medium computational power.
- Non-limiting examples of analysis of structures to which it can be applied are: temperature detrend couple structure analysis and multi-girder uniaxial analysis, among others.
- a Semiparametric Bayesian network [8] is used.
- SPBN Semiparametric Bayesian network
- CPDs do not fit well into multivariate Gaussian distributions (for example, bimodal distributions)
- GBNs perform poorly when modeling such distributions.
- linear Gaussian CPDs are linear in nature, they are inapplicable to represent nonlinear interactions between random variables.
- GBNs usually performs poorly to model the damping factors as damping factors usually follow a Gamma distribution.
- An alternative approach to avoid assuming data normality is to combine Bayesian network models with other nonparametric estimation models that are more flexible compared with parametric estimation models as they do not assume any type of parametric distribution.
- Figure 12 shows an example of probability-based baseline model generated using GMM (Gaussian mixture model).
- the training and calibration time of the model is of outmost importance.
- the creation of the probabilistic baseline model 213 can start while the structure is under supervision (in other words, from the very moment the structure starts being supervised), without requiring any previous training.
- the Machine Learning algorithms previously described need a certain set of data instances or examples to generate the baseline model, system calibration time depends on the size of this set. Therefore, to prevent the system from starting from scratch during new implementations, and thus reduce to time-to-operation, a consensus approach can be optionally used.
- Consensus learning is a type of machine learning algorithm that combines the predictions of multiple models or experts to arrive at a final prediction or decision.
- the goal of consensus learning is to improve the accuracy, robustness, and generalization of the learned model by reducing the variance and bias of individual models.
- a federated learning approach is selected.
- the federated learning approach is a type of consensus learning where the models are trained locally on distributed data sources, and the updated models are periodically aggregated and averaged, for example over a central server. Therefore, a consensus baseline model is preferably used in the generation of the baseline model 213, as it directly affects the preliminary calibration of the system.
- the consensus baseline model contributes to the creation of a baseline model 213 starting from previous knowledge. Because it is based on a model obtained from previous knowledge, it accelerates the generation of a robust baseline model 213.
- a robust baseline model 213 can be achieved with only about 500 impulses (less than one day of traffic circulation).
- Consensus typically refers to types of structures, meaning that consensus models can be established and used for structures of the same type. Structures can be grouped attending to their different types, such as bridges, viaducts and buildings, and with different levels of granularity, such as made of reinforced concrete, made of iron, etc. Therefore, different baselines obtained for different types of structures can be classified in terms of the type of the structure. For example, a consensus model can be established (and later used to generate a baseline model 213) for concrete bridges using predictions of several models of different concrete bridges, another consensus model can be established for steel bridges using predictions of several models of different steel bridges, or another consensus model of a bridge bay can be established using predictions of several models of bays of a bridge (bays having a similar structure).
- the consensus learning also enables to find out the classification of the structure when there is available a collection of models (such as GMM, GBN or SPBN) for different types of structures, such as several consensus models for concrete brides or several consensus models for steel bridges.
- a collection of models has in common that they all store fundamental and common structural information from all the similar analyzed structures.
- transfer learning is a machine learning technique where a model developed for a specific task is reused as the starting point for a model on a second task. Rather than training a model from scratch, transfer learning leverages knowledge gained from solving a related problem and applies it to a new, but similar, task.
- Transfer learning helps improve the performance of models on new tasks, accelerates training, and can lead to more effective and efficient learning across a range of applications.
- consensus models permit to have a starting baseline model in terms of parameters that can be fed into the system to start working as soon as possible. Then, local behaviors of the structure are also learned and updated while the system is working, as explained next. In sum, a consensus approach permits the system to start from scratch during new implementations, reducing the time-to-operation.
- the result of the probability-based ML baseline model 213 (together with the consensus model if this has been used) is a set of model parameters, referred to as ML baseline parameters 214, that enable a specific measure of fitness which depends on the algorithm used to generate the baseline model.
- ML baseline parameters 214 Once the baseline parameters 214 are generated, the next step is to evaluate their fitness to new instances or examples (online poles 211) that are coming from the new features extraction stage 209.
- a probability-based ML baseline model 213 representing the actual dynamic behaviour of the structure has already been generated (as represented for example in Figure 12, in which Euclidean distances are represented in both axis for different groups 0-5).
- a concept drift is defined as a change over time in the relation between input data (a set of online poles 211) and a target variable (the baseline model 213 representing the dynamic response of the structure).
- drift detection stage 216 is based on an algorithm that has the aim of reducing computational costs and avoiding the use of thresholds. [9]— [11].
- the concept drift algorithm evaluates a fitness of the new poles (currently obtained from on-line sensor measurements) to the probability-based baseline model.
- the fitness is typically represented by a distance between each pole falling outside the baseline model and the baseline model itself, the distance being indicative of a dynamic response degradation of the structure.
- the fitness can be, for example, a likelihood value.
- each new pole is assessed in terms of likelihood to the baseline model. When this likelihood is below a certain threshold, the pole is considered an outlier. When a certain number of outliers show up within a certain time period, it is considered that a concept drift has occurred.
- the concept drift algorithm 216 evaluates the fitness of a current instance (online poles 211) to a baseline model 213, as disclosed for example in [15] using actual data streams which are discarded once consumed.
- the algorithm provides a fitness (KPI 217) that can be evaluated using likelihood analysis, or penalized likelihood such as Bayesian Information Criteria (BIC), Akaike Information Criteria (AIC) or any other probabilistic based approach.
- This fitness measure (KPI 217) is sent to a next block 218 of the processing module 212.
- a KPI 217 can for example be a unidimensional value or a temporal series that may express positive or negative tendencies which provide information of the degradation of the structure.
- a KPI 217 is schematically represented in Figure 13 as the result of the comparison between the online poles 211 (represented as “Future data”) and an exemplary baseline model parameters 214.
- the KPI is in Figure 13 represented as an “Indicator of Change in Response”.
- FIG. 15 shows a flow diagram of the already described method for on-line monitoring a dynamic response of a structure, according to a particular embodiment of the invention.
- the fitness level can optionally be monitored, for example by an outlier detection algorithm such as Page- Hinkley [12], [13], to decide if an instance (online poles 211 instantly obtained from recently obtained (real time) measurements captured by sensors 10 in a structure) fits into the baseline model generated at block 213 or if the instance is an outlier.
- the number of outliers detected can be monitored and accumulated as input information for a next concept drift detection step (for future online poles 211). This is referred to as concept drift feedback 215.
- a final decision if the new instance is no longer represented by the baseline model can be taken by a probability-based outlier evaluator algorithm which evaluates a probability to find more outliers [14], When a concept drift is detected, several physical interpretations are possible:
- the dynamic response of the structure has change from the baseline model, i.e., potential degradation or damage is detected. This situation can trigger an alarm along with the KPI information 217.
- the baseline model is no longer valid and it needs to be updated.
- feedback 215 is sent to the probability-based Machine Learning baseline stage 213. This feedback 215 contains information about the subset of data to be used during the updating.
- drift detection block 216 ultimately evaluates the fitness of the baseline model, it is also useful to determine controlled changes of the model in terms of external factors, such as seasonal conditions and triggering different consensus models depending on the season.
- the dynamic response pattern of the structure is generated by comparing distances obtained from different dynamic responses of the structure at different times.
- the dynamic response pattern is indicative of the evolution of the dynamic response degradation of the structure over time.
- a remaining useful time of the structure can be estimated.
- the detection of the concept drift is also useful to update consensus baselines models that may being used.
- a consensus baseline model is taken as the starting point, where the concept drift algorithm 216 detects that the baseline model is not completely fitted. Therefore, an update is launched with the specific set of data increasing the fitness of the consensus-based new baseline model.
- the KPI 217 obtained in the concept drift detection stage 216 can be sent to a supervision platform, such as a dashboard 310 (usually at a remote location).
- the KPI 217 obtained in the concept drift detection stage 216 can optionally be used to estimate the remaining useful life (RUL) of the structure under supervision.
- the processing module 212 may include blocks 218 and 220, as schematically shown in Figures 1 and 10.
- RUL estimation of structures is a common field of development within the structural engineering. Methods of RUL estimation 220 are often statistics-based depending on the behavior distribution of the measured variable.
- a KPI 217 is a unidimensional value where its distribution can be modelled through traditional Weibull, Gaussian, or Gamma distribution to estimate the expected useful life.
- a new methodology can be applied to obtain at module 218 a health index (HI) 219 by calculating multiple exponential regressions at each sampling time, considering only incremental samples.
- the output from this module 218 is a health index 219, such as a curve, containing information about the dynamic response of the structure and its performance along time. For example, assuming that a curve of useful life of a certain structure is well-known, the health index 219 may indicate actual deviations from the original curve.
- the KPI 217, or the health index 219 in the event this information is available, is taken to a software-based block 220 for obtaining the remaining useful life (RUL) of the structure.
- Block 220 takes the information (KPI 217 or health index 219).
- Health assessment 221 provided by the structural RUL block 220 for example comprises at least one of a health index, a RUL indicator, which are typically float values, and a concept drift detection alarm, which is typically a Boolean value. Therefore, the output of the RUL block 220 is provided in the form of a vector of several values. For example, it is vector of four values: time stamp, health index, RUL and alarm.
- Figure 14 represents the concept of “fitness” and shows an example of KPI obtained after comparing new observations (impulses) against the baseline model: if a new impulse fits the model, the KPI is low value (100% fitting indicates KPI » 0), while if a new impulse fits poorly, the KPI is a large value (0% fitting indicates KPI » «).
- the health assessment 221 is the output of the processing module 212.
- the last module of the edge device 20 is a communication interface 222 implementing a communication layer ( Figure 1).
- Communication interface 222 is software-based and is aimed at integrating the health assessment information 221 into platform layers such as cloud services, in-premise clouds, servers and control rooms, among others. Therefore, the objective is to format the health assessment information 221 in the required communication protocols: Unified Architecture of Open Protocol Communications (OPC- UA), Message Queuing Telemetry Transport (MQTT), TCP and UDP, among others, understandable by the corresponding packages in the platform layer. This formatted information is referred to as 223 in Figure 1 .
- the communication interface 222 can also be used to send information into a hard drive of the edge device 20 to temporally store the data and send it when communications are available.
- the communication interface 222 can be implemented following a batch-mode configuration to send data in the required time windows, for example, when satellite link is available, reducing its use and the cost.
- the third architectural level of the system 1 is the remote system or platform 30, also referred to as platform layer.
- the remote system 30 is typically physically located far away from the structure under supervision and, therefore, from the edge device 20.
- Nonlimiting examples of remote systems 30 are cloud services, in-premise clouds, servers and control rooms, among others.
- the formatted information 223, formatted in the suitable communication protocol is received in the remote system 30, the contained fields are represented differently in a dashboard 310 depending on the nature of each individual value. For example, in the case of health index, as a unidimensional value along time, it can be represented as a time series plot. This is particularly useful in order to visually analyze the evolution of the KPI.
- the information represented by the dashboard 310 is objective information extracted from the data produced by the structure under supervision. Therefore, no subjective approaches are provided, improving the evaluation of the structural responses.
- each structure is highly condensed, i.e. , it only produces formatted information 223 when an impulse is detected, the scalability of the solution is improved. For example, having condensed and critical information from the bridges helps a bridge maintainer, who typically has more than 1 ,000 of bridges to manage, to perform a fleet assessment over the entire number of them. Moreover, this information can help take decisions, as explained next.
- Temporal change in the Health Index If there are important changes of the Health Index at the dashboard, such as presence of outliers or presence of a trend in its values, early warning signs can be detected. This information can be useful to increase traditional inspection protocols in the specific structure.
- RUL As the remaining useful life is a value of time expected until failure of the structure, preventive, predictive and prescriptive maintenance activities can be performed accordingly. This value can also be used as a ranking value when a fleet of structure is managed. This is also important for maintenance budget allocation and inspection activities.
- the main decision to be taken when an alarm is triggered is to inspect the structure to validate the reason of the change in the dynamic response.
- a change in the dynamic response of the structure is an early indicator of structural degradation. This alarm is also useful for corrective maintenance planning if used with other indicators that can help to diagnose the potential structure issue.
- Figure 16 shows an example of a structure being monitored by means of the disclosed method and system. It is a bridge located in Bilbao (Spain) (location 43.264697, - 2.961540). Figure 16 shows the location of the monitoring device and instrumentation 160. The bride has 6 spans. There is one accelerometer per span. Figure 17A-E show real data and results obtained at the illustrate bridge. Figure 17A shows the impulse detection for the first span of the bridge. Figures 17B and 17C show the extraction of natural frequencies and damping factors for the first span, respectively. Figure 17D shows the model baseline for the first span. It has been done with SPBN obtained algorithm. All variables are non-Gaussian so it is modelled by kernels (grey nodes).
- Figure 17E represents an KPI based on likelihood for the first span of the bridge.
- Figures 18A-18H show an online dashboard for the first span including results of one-year monitoring (358,095 cars detected).
- the first graph (Figure 18A) represents detected natural frequencies.
- the second one ( Figure 18B) represents the KPI.
- Figures 18C-18H represent histograms with the distribution of the values of the three main poles (in Figures 18C-18E the natural frequencies for poles 1 , 2 and 3, respectively; in Figures 18F-18H the damping factors for poles 1 , 2 and 3, respectively).
- the term “comprises” and its derivations (such as “comprising”, etc.) should not be understood in an excluding sense, that is, these terms should not be interpreted as excluding the possibility that what is described and defined may include further elements, steps, etc.
- the term “another,” as used herein, is defined as at least a second or more.
- the term “coupled,” as used herein, is defined as connected, whether directly without any intervening elements or indirectly with at least one intervening elements, unless otherwise indicated. Two elements can be coupled mechanically, electrically, or communicatively linked through a communication channel, pathway, network, or system.
Landscapes
- Engineering & Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Testing Of Devices, Machine Parts, Or Other Structures Thereof (AREA)
Abstract
A computer implemented method for on-line monitoring a structure, the method comprising: receiving (151) at least dynamics data (204) of dynamic variables representing a dynamic response of the structure from a plurality of sensors (10), the dynamic response of the structure being generated when the structure has been subjected to an excitation; detecting an impulse (152) created by said excitation by identifying a free load response (208) of the structure from said at least dynamic data (204); obtaining (153) a set of poles (210) comprising pairs of a natural frequency and a damping factor of the structure from the free load response (208); generating (154), using a non-supervised machine learning algorithm, a probability-based baseline model for the structure using a plurality of said poles (210) obtained along a certain period of time, the probability-based baseline model associating in a probabilistic way the dynamic variables affecting the structure and providing a plurality of baseline parameters (214); every time a new impulse is detected and a new set of poles (211) are obtained from the free load response of said impulse, feeding (155) the new set of poles (211) to a concept drift algorithm to obtain a KPI (217) of the structure.
Description
A COMPUTER IMPLEMENTED METHOD AND SYSTEM FOR HEALTH MONITORING OF STRUCTURES
TECHNICAL FIELD
The present invention belongs to the field of monitoring, surveillance, conservation and maintenance of civil engineering structures, such as bridges. More particularly, the solution herein disclosed refers to a computer implemented method and system for health monitoring structures.
STATE OF THE ART
In civil engineering, the conservation and maintenance of structures are critical important activities regarding safety. Nowadays, maintenance work on almost all large structures is visual and is based on subjective inspections carried out by experts. In economic terms, it is essential to decide maintenance intervention, since costs increase exponentially when going from merely preventive to corrective maintenance. In terms of security, the detection of anomalous behaviors that could lead to collapses would prevent the loss of human lives.
Inspection activities are normally done with frequencies measured in years, sometimes larger than 5 years. As the number of structures to inspect is normally high, for example, in bridges, where a structure maintainer can have more than 1.000 in its portfolio, comprehensive and continuous inspection is a challenge. That is, traditional inspection routines have an intrinsic scalability issue.
Currently, solutions to this problem are being searched for in technologies such as Internet of Things (loT) and Artificial Intelligence (Al). However, in terms of technology available to solve inspection needs, the use of electronic devices for the inspection of bridges has increased considerably in the last 20 years, but are oriented to singular structures, such as iconic bridges around the world. Therefore, the use of electronic devices in standard structures is limited by the cost and technical challenges that affects the scalability.
From the point of view of application, electronic devices deployed can be divided into instrumentation (sensors and interfaces), computation and communication. Therefore, in the field of structure instrumentation there is a continuous evolution within the standard methods such as:
-Topographic methods: That measure deformations of the order of millimeters.
-Radar-based methods: That detect cracking, anchor exploration, reinforcement location, detection of objects in the ground, buried installations and foundations.
-Methods based on X-rays: That define location of reinforcement diameter and distribution, inspection of prestressing cables, cracking and homogeneity of concrete.
-Acoustic methods: That detect cracks in the concrete.
-Photogrammetric methods: That measure displacements, deformations and cracks.
-Methods based on thermography: That detect cracks, crevices and other defects in the concrete.
-Methods based on magnetic fluxes: That diagnose the state of the reinforcement in a reinforced concrete structure and, in general, of those materials that can be magnetized by the action of a magnetic field (such as steel).
-Elastomagnetic sensors: That measure the force in metallic linear elements.
-GPS methods: That measure large displacements in structures.
-Electrical gauges: That measure deformations.
-Optical fiber: That measure temperatures, deformations, displacement acceleration.
-Accelerometers and inclinometers: That measure deviations in vibration patterns.
There are new technologies approaches regarding structure inspection using drones. However, these technologies are only oriented to surface visual inspection and not to dynamic structural response. Additionally, the classification techniques that have capabilities to correlate an image with a crack need strong training procedures from damage catalogues, prepared by expert civil engineers.
Therefore, the state-of-the-art regarding electronic devices deployed in different structures is limited to data loggers and gateways that only perform data storage, data collection and data forwarding tasks, the cloud services being defined as the key enablers of the technology. However, the cloud computing approach assumes that all the structure locations have internet connection with enough broadband to send raw data into the cloud. Reality says that structures such as bridges are often located in remote areas where connectivity is limited. Moreover, scalability challenges are created, as communication infrastructures and cloud services deployment generalization for many structures is expensive and not reliable enough.
From the point of view of data analysis, there are plenty of Al-based approaches, having strong scalability and application issues. On one hand, the use of common classification
algorithms such as Artificial Neural Networks (ANN), Support Vector Machines (SVM), decision trees, among others, must have training with balanced datasets, meaning that algorithms need to be trained with abnormal structure behaviour. Although there are plenty of experiments in the literature based on finite element models, structure miniatures and even real size structures to analyze the behaviour of the structure under failure situations, they are not cost-effective. Moreover, algorithms trained with these failures are overfitted to that particular failure mode, meaning that these algorithms are only suitable for the specific structure for which they have been developed.
On the other hand, many types of these algorithms require high amount of computing power, creating an important implementation issue as analytics for only one bridge using classification algorithms needs large and expensive computing power, for example, cloud computing. An overview of Big Data and Artificial Intelligence Techniques in bridge structural health monitoring is disclosed in [1],
Moreover, many of these Al algorithms are black-box based, where it is not possible to assess internal algorithm relationships with a real physical phenomenon. In other words, lack of explainability is an important issue with these algorithms as it is not possible to evaluate the outcome insights of it. This shortcoming of explainability is critical when the quality and the type of data fed to the algorithm is not good enough. It is also critical when the robustness of the system must be proved to be considered reproducible over time. For example, if a raw data stream from an accelerometer with a dynamic range between 0.5 to 1 ,000 Hz is consumed by an algorithm, it will learn the behavior of every single frequency. As a structure will have only some specific frequencies of interest, typically its natural frequencies, the rest of the data stream is irrelevant, so, the algorithm will be learning noise.
As explained above, it is critical to feed the selected Machine Learning algorithm with a high-quality data stream, but also, work in real time (so that it does not need to rely on historical databases). In this case, literature approaches (see for example [16]) use finite element models (FEM) of the structure to get the dynamic variables, such as, natural frequencies and damping factors. However, doing FEM analysis for every structure is not feasible, affecting again the system scalability. As an example, the state of Massachusetts in the United States manages 4,768 bridges, where nearly 500 are in urgent need of repair. This means to have 500 FEM models developed and running in a short period of time, which is not viable as previous high-quality digitalization of the bridge is needed.
Other approaches use traditional civil engineering tools to estimate the relevant variables from the bridge. Nevertheless, these approaches are performed offline, requiring high amount of effort.
In summary, there is need of methods and systems that can provide decision making insights about structural health with high availability, scalability and explainability, to improve maintenance efficiency and safety of structures.
DESCRIPTION OF THE INVENTION
The solution for structural health monitoring described in the present disclosure overcomes the drawbacks of prior art technologies therefor.
The present disclosure provides a method and system for monitoring a structure, in particular the dynamic response of a structure, such as a bridge, which takes into account different excitation sources.
In the context of the present disclosure, the structure may be a bridge, a viaduct, a dam, a warehouse, a silo, a marine platform, or any other building or infrastructure requiring structural health surveillance.
The method and system work on-line (i.e. in real time). In other words, they work with data streams in real time, meaning that data is consumed as it is obtained (and eliminated once consumed, thus reducing storage requirements). In the context of the present disclosure, the term “real time” mainly depends on the sampling frequency of the sensors used to capture dynamics data related to the dynamic response of the structure. Typical sampling frequencies may vary between about 0.5 and 1 ,000 Hz. Therefore, the term “real time” depends on the structure under analysis and on the sensing equipment used in the analysis. In this context, “real time” refers to a time comprised within a range varying between a minimum value Vmin and an upper value of 30 seconds, such as a range varying between a minimum value Vmin and an upper value of 10 seconds, or a range varying between a minimum value Vmin and an upper value of 2 seconds. Taking into account current technology, the minimum value Vmin may be, for example but without limitation, 10 ms (milliseconds). Nevertheless, one skilled in the art will understand that the evolution of technology may enable to reduce the minimum value Vmin of the range to a minimum value smaller than 10 ms. As a matter of example, the method and system may need between 10 ms and 2 s to produce a key performance indicator (KPI) from the time instant an impulse is generated.
The KPI may represent a change of conditions over time, a health index (HI) and/or the remaining useful life (RUL). The method and system operate with availability 24/7 and provide a set of KPIs. The HI or RUL obtained from the KPI can be used as a maintenance decision making insight by a nonexpert end user. They also reduce the level of subjectivity and dependence on the traditional experience of a human inspector, providing objective information as actionable insights.
The produced KPIs are artificial intelligence-based values, which do not need previous knowledge from the structure, avoiding the need of training and improving scalability. In other words, the method learns from the current state of the structure, meaning that no failure situation needs to be modelled or tested. The KPI is obtained through the application of a model called concept drift approach. This model can be automatically updated using artificial intelligence technology to increase system availability. The method and system integrate data stream processing at the edge in terms of its preprocessing and processing. This approach helps to avoid large infrastructure or electronic instrument costs improving the system scalability to any infrastructure.
In the context of the present disclosure, data preprocessing and processing is performed at the edge when the computing devices preprocessing or processing data are situated at or near the physical location where data is generated. Edge devices typically have limited processing power, memory, and storage capacity. Edge devices are responsible for collecting, processing, and analyzing data in real-time and transmitting only the relevant data to an external data integration and visualization system.
In the context of the present disclosure, a dynamic response of a structure represents the behaviour of dynamic variables when the structure is subjected to an excitation caused by external phenomena. A structure may be excited by different phenomena. Depending on the type of structure and other causes, such as location and exposure to atmospheric phenomena, the structure may be excited by different phenomena. Examples of phenomena that can cause the excitation of the structure are: the passage of vehicles (when the structure is, for example, a bridge or a viaduct), gusts of wind, a force exerted by a material when it is loaded into the structure (when the structure is, for example, a dam, a silo or a warehouse in which material is unloaded), a force or slapping of sea waves, or an earthquake. This excitation has an impact on the structure, for example causing an impulse. Non-limiting examples of variables or parameters (dynamic variables) providing data related to the dynamic response of the structure to an external excitation are vibration, displacement, inclination and force, among others. Data (such as captured values) associated to dynamic variables is referred to as dynamics data.
The dynamic response of a structure is usually expressed as a set of pairs of natural frequencies and damping factors.
The dynamic response of the structure is sometimes affected by contextual variables. Contextual variables are usually environmental ones which may cause a temporal change in, for example, a characteristic of the material of which the structure is made or the weight of the structure, which in turn causes a change in the mechanical response of the structure, thus altering the dynamic response of the structure. Non-limiting examples of contextual variables related to the environment surrounding the structure, are: surface and environment temperature, environment humidity, wind speed and wind direction, rain drops, snow, frost, ice or season of the year, among others. Data (such as captured values) associated to contextual variables is referred to as contextual data. Contextual variables can optionally (typically depending on the circumstances, such as type of structure, location and/or environmental situation) affect the structure and alter its dynamic response. The combination of dynamic and contextual variables causes a certain response of the structure when the structure is subjected to an excitation. By taking into account both dynamic and contextual variables, the method is able to discriminate whether a change in the response of the structure to an impulse is due to a change in the health of the structure of simply to a change in an environmental variable, such as temperature.
Dynamics data and contextual data are typically captured by sensors. In the context of the present disclosure, the general term “sensors” refers broadly to sensors and any other devices that generate data from measurements associated or originated at the structure under supervision. In other words, the sensors are related to, or can be seen as, the structure instrumentation. In this text, the term “instrumentation” is sometimes used to refer to the sensors.
The general concept of the proposed method is as follows: When external phenomena excite variables affecting a structure under supervision, creating an impulse, which typically causes an oscillation, a plurality of sensors capture the behaviour of these variables. The sensors typically capture values of physical parameters periodically (at a certain sampling frequency, which may be between about 0.5 and 1 ,000 Hz). Non-limiting examples of sensors that may be used are accelerometers, inclinometers, extensometers, strain gauges, force transducers and GPSs. At a monitoring system (edge device), a communication interface builds a data stream comprising instant values of these variables. For example, accelerometers measure vibration, inclinometers measure tilt, extensometers measure distance, strain gauges measure deformation, force transducers measure force, GPSs measure distance, and so on. The data stream
comprises at least dynamics data. It can optionally comprise also contextual data when there are sensors capable of capturing values of contextual variables. The objective is to record impulses created by external phenomena (such as normal traffic over a bridge) through continuous monitoring and avoiding interruptions in the normal use of the structure (such as traffic interruptions). The method of this disclosure obtains a dynamic response of the structure from the captured measurements: An impulse caused by external factors creates an oscillation in the structure. The free load response of the oscillation (that is to say, the response when the impulse has ended) is an attenuated oscillation which is directly related to the dynamic characteristics of the structure, and therefore can be considered a dynamic response of the structure. The proposed method differentiates stable regimes and transient regimes in the dynamic response, and extracts from the transient regime the free load response of the structure to the excitation. This is done by applying a transient analysis technique. From the free load response, the dynamic response of the structure will be estimated. The dynamic response is a set of pairs of natural frequencies and damping factors (poles) of the structure called poles. With the poles and optionally with the contextual data (if there are contextual data), a baseline model of the structure is generated. The proposed approach for the extraction of these features (natural frequencies and damping factors) is based on system identification techniques that take advantage to the fact that a linear system satisfies the properties of superposition and homogeneity, and therefore it is possible to obtain linear approximations of the structure and model it as a coupled spring-mass- damper system. From control theory, it is known that this system of various coupled spring-mass-dampers can be defined by its characteristic equation, which roots determine the character of the time response. These roots are the poles, and for the structural application, a pole is a pair of a natural frequency and a damping factor. Each pole represents a different vibration mode of the structure.
Next, a baseline model of the structure needs to be generated. When the proposed method and system are applied to monitor a structure for the first time, a model which represents the current behaviour of the structure needs to be created. This model represents the actual dynamic behaviour of the structure. The employed model is a stochastic baseline model (in opposition to deterministic models, based for example on deep learning techniques or on neural networks). To build the stochastic baseline model of the structure, a probabilistic relationship between different natural frequencies and damping factors representing the dynamic response of the structure, that is to say, its recent behaviour, is learned. These pairs or poles represent the estimated dynamic variables of the structure, and therefore represent the dynamic response of the structure.
If there are also contextual data (preferably conditioned contextual data), which represents the available contextual variables affecting the structure, such as temperature, humidity, etc., they are also used in the generation of the stochastic baseline model of the structure. The generation of the stochastic baseline model does not require any previous training, such as a specific training under real failure conditions or training with synthetic failure data created ad hoc for the structure under supervision. The technique used to learn the baseline model without previous training is called nonsupervised learning.
The result of the probability-based (stochastic) ML baseline model is a set of parameters, which are conditional probability distributions associated with each features (natural frequencies, damping factors and (if any) contextual variables). Additionally, a probability-based ML baseline model can have a structure, which is a directed acyclic graph (DAG) that expresses conditional independencies and dependencies among random variables associated with features. Parameters and structure enable a specific measure of fitness which depends on the algorithm used to generate the baseline model. Once the baseline parameters and structure are learned, the next step is to evaluate their fitness to new instances or examples (online poles and (if any) contextual variables) that are coming from the new features extraction stage. The result of this stage is a measure of fitness called key performance indicator (KPI). Once the KPI is obtained, it can be monitored instantaneously and/or, for example, by a concept drift detection algorithm to decide if a group of instances (online poles instantly obtained from recently obtained measurements captured by sensors in a structure) fits into the baseline model or the baseline model needs to be updated. The KPI can be sent to a supervision platform, such as a dashboard (usually at a remote location). Alternatively, the KPI can optionally be interpreted as a health index, where is evolution can be used to estimate the remaining useful life (RUL) of the structure under supervision. The RUL can be then sent to the dashboard for human assessment.
A first aspect of this disclosure is a computer implemented method for on-line monitoring a structure. The method comprises the steps of: receiving at least dynamics data of dynamic variables representing a dynamic response of the structure from a plurality of sensors, the dynamic response of the structure being generated when the structure has been subjected to an excitation; detecting an impulse created by said excitation by identifying a free load response of the structure from said at least dynamic data; obtaining a set of poles comprising pairs of a natural frequency and a damping factor of the structure from the free load response; generating, using a non-supervised machine learning algorithm, a probability-based baseline model for the structure using a plurality
of said poles obtained along a certain period of time, the probability-based baseline model associating in a probabilistic way the dynamic variables affecting the structure and providing a plurality of baseline parameters; every time a new impulse is detected and a new set of poles are obtained from the free load response of said impulse, feeding the new set of poles to a concept drift algorithm to obtain a KPI of the structure.
In embodiments of this disclosure, the method further comprises, together with receiving at least dynamics data from the plurality of sensors, receiving from the plurality of sensors contextual data of contextual variables representing the environment surrounding the structure; wherein the step of generating, using a machine learning algorithm, a probability-based baseline model for the structure, also uses said contextual data.
In embodiments of this disclosure, the method further comprises, prior to identifying a free load response of the structure from said dynamic data, carrying out data conditioning of the dynamics data received from the plurality of sensors, said data conditioning comprising data cleaning and real-time data imputation.
In embodiments of this disclosure, the method further comprises, prior to identifying a free load response of the structure from said dynamic data, carrying out data conditioning of the contextual data received from the plurality of sensors, said data conditioning comprising data cleaning and real-time data imputation.
In embodiments of this disclosure, the method further comprises, once a KPI has been obtained, monitoring a fitness level to decide if the new online poles fit into the baseline model and, if a certain number of outliers are identified, providing concept drift feedback to the machine learning algorithm to update the probability-based baseline model.
In embodiments of this disclosure, the method further comprises estimating a remaining useful life of the structure based on a health index interpreted from the KPI.
In embodiments of this disclosure, the excitation the structure is subjected to is generated by at least one of the following phenomena: a passage of at least one vehicle on the structure, a gust of wind, a force exerted by a material when loaded into the structure, a force or slapping of sea waves, an earthquake and any combination thereof.
In embodiments of this disclosure, the variables or parameters providing data related to the dynamic response of the structure to an external excitation are at least one of: vibration, displacement, inclination and force.
In embodiments of this disclosure, the variables or parameters providing contextual data related to the environment surrounding the structure, are at least one of: surface and
environment temperature, environment humidity, wind speed and wind direction, rain drops, snow, frost, ice or season of the year.
In embodiments of this disclosure, the method further comprises using a technique for carrying out the real-time data imputation of the data streams that is selected from a group comprising: last-value technique, mean value technique, interpolated value technique and any combination thereof.
In embodiments of this disclosure, detecting an impulse created by said excitation by identifying a free load response of the structure from said at least dynamic data is done by implementing at least one classifier configured to detect transient regimes.
In embodiments of this disclosure, two classifiers configured to detect transient regimes are implemented.
In embodiments of this disclosure, detecting an impulse created by said excitation is done as follows: calculating from the dynamics data a sample moment-time representation curve rc(t) of the dynamical system in a previously selected time window th; computing the arc length lr(t) of the SMTR curve rc(t); generating a piecewise function hTsi(t) using a threshold lro and the arc length lr(t); classifying the system’s time periods as either transitory or stationary; using the SMTR curve rc(t) of the dynamical system, computing its matrix of derivatives Dc(t); calculating time segments where the determinant of the matrix of derivatives Dc(t) is zero; if the determinant obtained is zero, segmenting the curve representing the dynamic system into flat curves ryj(t) and calculating their flat curvature kn(t); if the determinant obtained is not zero, computing generalized curvatures ki(t) of the SMTR curve rc(t); calculating a piecewise function kc(t) which stores the magnitude of the generalized and flat curvatures of the studied system; generating a piecewise function hTsii(t) using a threshold kco and the magnitude of the curvatures ki(t); using the piecewise function hTsii(t) to classify the system’s time periods as either transitory or stationary.
In embodiments of this disclosure, obtaining a set of natural frequencies and a damping factor of the structure from the free load response is carried out by a system identification algorithm implementing system identification techniques selected from a group comprising system identification techniques based on deterministic algorithms, system identification techniques based on probabilistic algorithms and any combination thereof.
In embodiments of this disclosure, the machine learning baseline model algorithm comprises at least one of gaussian mixture models, gaussian Bayesian networks, semiparametric Bayesian networks, and any combination thereof.
In embodiments of this disclosure, the method further comprises triggering an alarm when the trend in a health evolution of the structure is above a certain threshold.
In embodiments of this disclosure, the structure to be supervised is a bridge, a viaduct, a dam, a warehouse, a silo, a marine platform, or any other building or infrastructure requiring structural health surveillance. It is in particular suitable to monitor structures of difficult access for human operators.
A second aspect of this disclosure is a device for on-line monitoring a structure. The device comprises: means for receiving at least dynamics data of dynamic variables representing a dynamic response of the structure from a plurality of sensors, the dynamic response of the structure being generated when the structure has been subjected to an excitation; means for detecting an impulse created by said excitation by identifying a free load response of the structure from said at least dynamic data; means for obtaining a set of poles comprising pairs of a natural frequency and a damping factor of the structure from the free load response; means for generating, using a machine learning algorithm, a probability-based baseline model for the structure using a plurality of said poles obtained along a certain period of time, the probability-based baseline model associating in a probabilistic way the dynamic variables affecting the structure and providing a plurality of baseline parameters; means for feeding, every time a new impulse is detected and a new set of poles are obtained from the free load response of said impulse, the new set of poles to a concept drift algorithm to obtain a key performance indicator of the structure.
The same advantages of the first aspect of the disclosure are applicable to the second aspect of the disclosure.
The device is an edge device. In some embodiments of the invention, the hardware of the edge device includes: one or more central processing units (CPU) or at least one core thereof, a multi-core central processing unit (multi-core CPU), an embedded circuit (e.g. a system-on-chip, a multiprocessor system-on-chip) -e.g. Zynq, MPSoC by Xilinx, a graphics processing unit (GPU) or any combination thereof.
In some embodiments of the invention, the edge device further includes a data storage device including at least a non-volatile memory, such as a hard disk drive (HDD) or preferably a solid-state drive (SSD).
A third aspect of the disclosure is a system for on-line monitoring a structure. The system comprises: a plurality of sensors attachable to the structure to be supervised, the sensors being capable of capturing data related to a dynamic response of the structure when it is subjected to an excitation; a device according to the second aspect of this disclosure,
wherein the device is an edge device situated in the vicinity of the structure to be supervised; and a remote system configured to receive from the edge device information about the health of the structure to be supervised.
The system comprises a plurality of sensors attachable to the structure to be supervised, the sensors being capable of capturing data related to a dynamic response of the structure when it is subjected to an excitation, and optionally contextual data that can alter the dynamic response of the structure; a device according to the second aspect of the invention, wherein the device is an edge device situated in the vicinity of the structure to be supervised; and a remote system configured to receive from the edge device KPI information about the structure to be supervised. The sensors are in communication with the edge device. This communication may be implemented either through a wired connection or through a wireless connection, such as WiFi, Bluetooth, 5G or based on any other suitable wireless communication protocol, typically a short-distance communication protocol.
Additional advantages and features of the disclosure will become apparent from the detailed description that follows and will be particularly pointed out in the appended claims.
BRIEF DESCRIPTION OF THE DRAWINGS
To complete the description and in order to provide for a better understanding of the invention, a set of drawings is provided. Said drawings form an integral part of the description and illustrate an embodiment of the invention, which should not be interpreted as restricting the scope of the invention, but just as an example of how the invention can be carried out.
The drawings comprise the following figures:
Figure 1 shows a block diagram of a system for monitoring a structure according to a particular embodiment of the invention.
Figure 2 schematically shows a portion of a structure including a plurality of sensors according to a particular embodiment of the invention.
Figure 3 shows a block diagram of a data stream conditioning stage performed at an edge device according to an embodiment of the invention.
Figure 4 shows a block diagram of a preprocessing stage performed at an edge device according to an embodiment of the invention.
Figure 5A shows a scheme of a proposed methodology for transient regimes and stationary regimes classification according to an embodiment of the invention.
Figure 5B shows impulses occurring on a bridge over time as a result of passing traffic according to an embodiment of the invention. The square signals depict the impulse detection classifier, responsible for identifying and extracting the corresponding response.
Figure 6A shows a block diagram scheme of an implementation of the stage of impulse detection (classification of transient regimes and stationary regimes) according to a preferred embodiment of the invention. Figure 6B shows an exemplary validation assessment of the proposed classifiers for a studied dynamical system under switching step input functions.
Figure 7 shows the response of a single impulse obtained at the stage of impulse detection. From this response, the relevant natural frequencies and damping factors of the structure can be calculated.
Figures 8A-8B show estimated natural periods (inverse of frequencies) and damping factors, respectively, obtained for an exemplary bridge between October 2019 and May 2022.
Figures 9A-9D show the estimation distribution for first, second and third frequencies of an exemplary bridge, showing estimation algorithm stability.
Figure 10 shows a block diagram of a processing stage performed at an edge device according to an embodiment of the invention.
Figure 11 shows exemplary graphics representing the impulse detection stage, the new features extraction stage and the stage for generation of probability baseline model.
Figure 12 shows an example of probability-based baseline model generated in an embodiment of the invention.
Figure 13 shows the graphics of Figure 11 , plus the result of the concept drift detection stage according to embodiments of the invention.
Figure 14 shows an example of KPI obtained after comparing new observations (impulses) against the baseline model: if a new impulse fits the model, the KPI is low value (100% fitting indicates KPI » 0), while if a new impulse fits poorly, the KPI is a large value (0% fitting indicates KPI » «).
Figure 15 shows a flow diagram of the method for monitoring a dynamic response of a structure, according to a particular embodiment of the invention.
Figure 16 shows an example of a structure (bridge) being monitored by means of the disclosed method and system.
Figure 17A-E show real data and results obtained for span 1 of the bridge of Figure 16.
Figures 18A-18H show an online dashboard for span 1 of the bridge of Figure 16.
DESCRIPTION OF A WAY OF CARRYING OUT THE INVENTION
A block diagram representing an architecture of a system 1 for monitoring a structure according to a particular embodiment of the invention is depicted in Figure 1. The shown architecture is defined by three architectural levels: sensors 10 for capturing data from the structure under supervision and its environment, an edge device 20 (also referred to as edge node) and a remote system or platform 30.
In particular, the system 1 can monitor a dynamic response of a structure under supervision and provide an indication of the healthiness of the structure. The dynamic response of the structure is generated when the structure is subject to an excitation caused by external phenomena. Contextual variables can optionally affect the structure and alter its dynamic response.
Through the sensors 10, the system 1 is able to collect two types of data: dynamic data provided by dynamic variables (such as vibration, displacement, inclination and force, also generally referred to as data related to the structure dynamics) and contextual data provided by contextual variables (such as temperature, humidity and wind speed and direction, also generally referred to as data related to the environment surrounding the structure). The main requirement for the sensors 10 (instrumentation, in general) is to be able to capture the required variables with the right sensitivity. In other words, the instrumentation must capture the response of the structure (for example, its oscillation, in which case sensors are typically accelerometers) caused by any impulse or excitation. The sensors 10 must be suitable to work outdoors to avoid measuring deviations because of environmental changes. For example, Figure 2 schematically shows a portion of a structure (in this case, a viaduct) in which a plurality of accelerometers and inclinometers have been installed to continuously (i.e. at a certain sampling frequency) measure the vibration (oscillation) and tilt, respectively, suffered by the structure.
The sensors are in communication (wired or wireless) with the edge device 20. The data captured by the sensors 10 is provided to the edge device 20 through a communication interface 201 implementing a communication layer. Communication interface 201 is software-based and is implemented to enable working with different types of sensors or systems of sensors 10. Communication interface 21 provides an interface module for
every communication protocol needed by the sensors. Non-limiting examples of these interfaces are Integrated Electronics Piezo-Electric (IEPE) standard, MicroElectromechanical Systems (MEMS) standard, User Datagram Protocol (UDP) and Transfer Control Protocol (TCP), among others. These interfaces guarantee the quality of the data provided by the sensors 10 to build a useful data stream 202 that can be consumed by subsequent stages. Each interface module needs to maintain data integrity, avoiding losing data packages (i.e. sanity checks) and performing a proper time stamping.
So, when external phenomena excite variables affecting the structure, the sensors 10 capture the behaviour of these variables and the communication interface 21 builds a data stream 202 comprising instant values of these variables. The data stream 202 outputted by the communication interface 201 can comprise two types of data captured by the sensors 10: dynamics data related to dynamic variables and optionally contextual data related to contextual variables. When there are two types of data, they can be delivered in a single data stream or in two separate data streams (a dynamics data stream and a contextual data stream). They represent the dynamic response of the structure to an excitation. In other words, they represent the response (for example, oscillation) to an impulse generated in the structure as a consequence of the excitation to which it is subjected. Although in Figures 1 and 2 a single data stream 202 is depicted, as explained, it can be implemented with two separate data streams: a dynamic data stream and a contextual data stream. The data of the two data streams must be synchronized so that they can be used in subsequent stages. In sum, all the relevant signals provided by sensors 10 are synchronized into a single or multiple data streams, depending on the subsequent analysis to be performed.
Because the nature of the data forming the data stream(s) 202 can be very divers (for example, the sampling frequency at which different variables are captured by sensors is very divers and may vary between 0.5 Hz and 1 ,000 Hz depending on the sensor and type of variable), this data is preferably conditioned. Therefore, the data stream(s) 202 is(are) preferably taken to a data stream conditioning module 203, in which sanity checks are applied. For example, it is important to verify that the jitter (difference between timestamps) is constant because artificial intelligence (IA) algorithms used later assume constant time, and therefore variable jitter could imply precision problems. Or data contents should be checked to verify that they are coherent. Non-limiting examples of conditioning techniques are data cleaning and real-time imputation applied to the data comprised in the data stream 202. A conditioned data stream may take the form of a
table including a timestamp and a string of values of different parameters (vibration, temperature, etc.) coming from the different sensors 10.
A block diagram of the data stream conditioning stage 203 is shown in Figure 3, in which the inputs 202 and outputs 204, 205 of the stage are depicted. The conditioned data stream has been split into two data streams, a conditioned dynamics data stream 204 and a conditioned contextual data stream 205, because in the shown implementation of the edge device 20 the data coming from the two types of variables (dynamic ones and environment ones, if any) are processed separately.
Other operations that can be performed in this stage of data stream conditioning 203 are filtering and offset removal. These operations are necessary to eliminate potential contents of the data stream that are not related to the structure under analysis, but to other external factors affecting the sensors, such as, electrical frequency (interfering signal at 50 Hz) or gravity. These operations are especially necessary when certain sensors are involved, such as MEMS-based accelerometers. A calibration stage prior to use in operation, performed on each sensor 10 to be used in the structure, enables to identify the type of contents that should be eliminated with these operations.
Other operations that can be performed in this stage of data stream conditioning 203 are removal of not available data (for example marked as NaN, -9999, or in any other way, depending on the sensor 10) and/or imputation process. Not available data are sometimes generated during the synchronization process to indicate that there has been a problem (missing capture) with the data acquisition by the sensor. Regarding imputation, as the data stream might contain variables with different sampling frequencies, the instances created could be empty for the slowest variables (variables with lower sampling frequencies). Therefore, a real-time imputation process needs to be applied to fill empty positions in the data stream. The imputation strategy is directly related to the nature of the signal (data stream), such as if it is binomial, discrete or continuous. Depending on the nature of the data stream, the following imputation strategies can be used:
Last value: In this case, the imputation is performed by filling the missing data with the last available value. This strategy is particularly useful with slow variables such as environment temperature.
Mean value: In this case, the imputation is performed by filling the missing data with the mean value of the last known n-values of the variable. This strategy is particularly useful with slow-to-medium variables such as surface temperature.
Interpolated value: in this case, the imputation is performed by filling the missing data with the result of interpolation fed with the last known values. The interpolation type could be done with simple linear interpolation, splines or bicubic interpolation. This strategy is particularly useful with medium-to-fast variables such as vibration or inclination.
The different operations comprised in the data stream conditioning stage 203 can be integrated as software-based conditioning packages which can be configurated to any potential structure as needed during the system deployment.
The dynamics data stream (preferably conditioned at stage 203) enters a preprocessing module 206, as shown in the block diagram of Figure 4. This preprocessing module 206 entails two stages. First, an impulse detection stage 207 is applied. An impulse is created every time the structure is excited. The impulse causes an oscillation which, when the excitation ends, becomes attenuated until it tends to disappear. This attenuated oscillation represents the dynamic response of the structure. In this stage, a transient analysis technique is applied to the -preferably conditioned- dynamics data stream 204 to differentiate a stable regime from a transient regime in the dynamic response of the structure and to extract, from the transient regime, a free load response 208 of the structure. Second, a stage 209 for new features extraction is applied. In this stage, from the free load response 208, a set of poles (natural frequencies and damping factors) of the structure are obtained. Each pole comprises a natural frequency and a damping factor. These poles 210 will be later used, together with the conditioned contextual data stream 205 (if any), to generate a baseline model of the structure. This will be described later.
The impulse detection stage 207 is represented in Figure 5A. For every excitation source, such as a vehicle passing on the structure, a wind gust on the structure or an earthquake in the ground in which the structure is located, an impulse is generated. This impulse excites the structure. The response to this excitation is a free oscillation after the impulse, also called transient regime, until the response ends when the structure recovers its stability, also called stable regime. This free oscillation or free response of the structure provides information of the dynamic features of the structure. The excitation is captured by the one or more sensors 10, from which the dynamics data stream 204 (preferably conditioned dynamics data stream) is built. Figure 5A refers to a particular impulse caused by the passing of a vehicle on a bridge. However, any other impulse caused by a different external excitation can be detected in a similar way. In particular, the signal shown in Figure 5A represents the response captured by an accelerometer sensor disposed on the bridge when a vehicle passes on the bridge. At time instant 51
the accelerometer detects the vehicle passing. Between time instants 51 and 52 the signal captured by the accelerometer represents the vehicle moving on the bridge. From time instant 52, a transient regime is shown, which represents the dynamic response of the bridge once the vehicle (source of excitation) has left the bridge. At certain time after the vehicle has left the bridge, the transient regime becomes stable regime. Sj(t) represents the signal registered by the accelerometer. oSj(t) represents the algorithm used to detect the impulse. Figure 5A additionally shows other registered excitations, such as the excitation caused by another vehicle and the excitation caused by two vehicles simultaneously passing in opposite directions.
In stage 207 a transient analysis technique is applied to the dynamics data stream 204 to identify the transient regime after the impulse and to extract, from the transient regime, a free load response 208 of the structure. From the free load response 208 of the structure, the dynamic components of the structure (natural frequencies and damping factors) can be later obtained.
There exist different algorithms to detect impulses offline. For example, there are algorithms based on the energy of the signal, on signal statistics, based on wavelets and based on measurement of correlation and sparsity. In conventional structural health monitoring systems, the detection of the response to the excitation is normally done through threshold analysis, but it lacks in the ability to correctly identify the useful part of the impulse response within the signal. Moreover, the extraction is usually not correctly done, and this affects negatively the quality of the data stream to be fed into the next step. The main challenge of the threshold-based impulse detection is the need of precise calibration from the specific application deployment, which is not feasible, as the structures can be excited with different types of impulses (for example, a bridge can be excited by one car driving along, or by two cars crossing at the same time in opposite directions, or by a long track, or by a train having different cars, etc.) and have many types of invalid impulse response for the analysis. For example, if there are two cars crossing at the same time over a bridge, with a slight delay between them, the resulting impulse response will be affected in a random way. This type of impulse response must be rejected to avoid introduction of noise into the analysis.
To overcome the main challenge of conventional structural health monitoring systems, the proposed method and system relate the impulse and its response to a transient analysis. Therefore, the software-based impulse detection preprocessing package (stage 207) is able to differentiate stable regimes and transient regimes from a signal (a conditioned dynamics data stream) in real-time. Transient regimes in a data stream represent an excitation (for example, the pass of a vehicle on a structure) and stable
regimes in the data stream represent the absence of the excitation (for example, no vehicle passing on the structure). Through the detection of stable regimes and transient regimes, the impulse causing a free load response is detected (the impulse representing the pass of the vehicle on the structure) and the free load response 208 of the structure is extracted from the transient regime.
The preprocessing package or module 206, and in particular stage 207 thereof, implements an algorithm for online impulse detection and classification of transient regimes. The proposed algorithm uses stochastic process definitions and differential geometry tools to generate a spatial curve representation of the dynamical systems output signals. In other words, the algorithm considers the geometric representation of a dynamic system. The algorithm is based on the fact that the response of a structure to an impulse contains information about its structural dynamics. Because the structure response can be represented as a spring-mass-damper system, it can be expressed through a characteristic equation whose roots determine the character of the time response. The roots of this characteristic equation are the dynamic components of the structure (the so-called poles of the structure, i.e., natural frequencies and damping factors). The algorithm implements one or more classifiers to detect in real time the occurrence of an impulse and its transient regime.
The implemented at least one classifier detects transient regimes for multivariate asymptotic, marginally stable, and cyclo-stationary behaviors. Stable regimes are discarded because they mean that the structure has not been excited and therefore no dynamic information can be extracted from the structure.
A preferred methodology for stationary and transient regimes classification that can be implemented at stage 207 is described next in connection with the scheme of Figure 6. The proposed transient and stationary regimen classification methodology is oriented to dynamical systems monitored by a group of sensors 10 that capture the dynamics (signal y(t), conditioned data stream 204) of their states. Figure 6 shows a block diagram of a preferred methodology for transient regimes and stationary regimes classification.
In a first stage 61 , using the preferably conditioned dynamics data stream 204 (y(t) in Figure 6), which represents the systems output captured by the available sensors 10, a sample moment-time representation (SMTR) curve rc(t) of the dynamical system G is calculated in a previously selected time window th. This curve represents the sampling moment of each measured output 204 and serve to monitor each output as a stochastic process.
In stage 62, after constructing the curve rc(t), its arc length lr(t) is computed. The arc length summarizes the behavior of the sample moments of the signals that monitor the system.
Next, in stage 63, a piecewise function hTsi(t) is generated using a threshold lro and the arc length lr(t). The piecewise function stores the transient portions of the signal. The piecewise function hTsi(t) enables the identification of the time periods when the outputs of the dynamic system behave as a weakly stationary stochastic process.
Finally, in stage 64, the piecewise function hTsi(t) is used to classify the system’s time periods as either transitory or stationary. This approach enables the development of a classifier based on the arc length of the curve that represents the dynamic system under study. When the output of stage 64 is "yes", it is determined that an impulse caused by an excitation has occurred in the structure. If it is “not”, it is determined that no excitation causing an impulse has occurred. T denotes the set of times where the state-space is mapped. Ts represents the periods of time where the system is in stationary regime, while Tt represents the transient periods of time. Therefore, the subindex ‘s’ means stationary and ‘t’ refers to transient.
The classifier obtained at stage 64 performs correctly in certain circumstances. For example, when the structure is a bridge or a viaduct, it performs correctly when the traffic on the structure is relatively low. However, in other circumstances, such as situations involving many impulses occurring at close time instants (for example, in situations of load traffic) or situations susceptible to the influence of random external loads and noise, it has been observed that the classifier is not robust enough to correctly detect and isolate the required transient responses. For this reason, an additional classifier has been implemented, which is preferably used, together with stages 61-64, to improve the accuracy of the algorithm. It is explained next.
In stage 65, the SMTR curve rc(t) of the dynamical system G is used to compute its matrix of derivatives Dc(t).
Then, in stage 66, the time segments where the determinant of the matrix is zero are calculated.
If the determinant obtained in stage 66 is zero, the curve representing the dynamic system is segmented into flat curves ryj(t) and their flat curvature kn(t) is calculated (stage 67). This is done as follows:
On one hand, the computation of a matrix determinant is a common procedure that can be done by several methods. For instance, one of the most optimal methods is based on
performing the LU decomposition to the matrix T>g and computing the product of the elements of the diagonals of the matrices L and U as follows:
where ™is the dimension of the square matrix
whereas s is the number of row exchanges during the decomposition. Further information of this computation can be surveilled in Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2001). Introduction to Algorithms, MIT Press and McGraw-Hill, 2001. ISBN, 262032937, 636- 640.
Regarding the segmentation of the SMTR curve, the procedure for computing the flat curvatures is next shown:
The SMTR curve of a dynamical system can be written as
1,2, ... y are the samp|e moments of each sensor 31 ! r Thus, the segmentation of the curve into plane curve will consist of a set of plane curves written as
-|-|-|e curvature of each plane curve
can be written as
Otherwise (if the determinant obtained in stage 66 is not zero), the generalized curvatures ki(t) of the SMTR curve rc(t) are computed (stage 68), preferably using the Frenet-Serret formulas.
Next, in stage 69, a piecewise function kc(t) is calculated, which stores the magnitude of the generalized and flat curvatures of the studied system.
The, in stage 70, a piecewise function hTsii(t) is generated using a threshold kco and the magnitude of the curvatures ki(t). The piecewise function hTsii(t) enables the identification of time periods when the outputs of the dynamic system behaves as weakly stationary stochastic processes.
Finally, in stage 71 , the piecewise function hTsii(t) is used to classify the system’s time periods as either transitory or stationary. This approach enables the development of a classifier based on the curvatures of the sample moment-time representation curve rc(t).
This methodology has been validated with linear, non-linear, and discontinuous simulated systems and compared with existing methodologies in the literature. Based on the validation assessment, the proposed classifiers (stages 63 and 70) outperform existing methodologies when identifying transient regimes in systems with differing behaviors. The algorithm is able to work with a real time data stream 204 integrated into the edge node 20. As a matter of example, one of the tests performed during the validation assessment of the proposed classifiers consisted in simulating a dynamical system written as
The three sensors y(t) of the dynamic system were used to validate the proposed method by varying the two sources of excitations
The method was validated with switching step functions as inputs. An example of the algorithm performance is shown in Figure 6B. The sensors behavior and the geometrical properties of the STMR curve are shown in Figure 6B. Moreover, the classification of the transient and stationary regimes are depicted in this plot.
So far, the free load response 208 of the structure under supervision has been obtained (subblock 207 in preprocessing block 206). However, although the free load response of the structure provides information of the dynamic features of the structure, the free load response represents the signal(s) captured by the sensor(s) 10. These signals are typically not directly interpreted in the sense that they do not enable to directly infer from these signals the dynamic response of the structure. However, from each signal a set of poles can be estimated, and from these poles the natural frequencies and damping factors can be obtained. Therefore, once the impulse (for example represented in Figure 7 (left)) has been extracted at stage 207, the free load response 208 is consumed by a stage 209 of new features extraction. These new features are the poles from which the natural frequencies and damping factors can be obtained. The objective of this package or stage 209 is to produce a new data stream formed by only explainable variables (versus the free load response representing the data captured by the sensors, which is not directly interpreted by a human operator). Figure 7 shows an exemplary single
impulse response from which the natural frequencies and damping factors of the structure can be obtained.
The stage of new features extraction 209 replaces conventional analysis, such as FEM analysis, which only works offline, and enables a real-time analysis of the dynamic response of the structure. Additionally, it produces variables, of which their actual value can be inspected by field experts, such as structural engineers. This distinctive characteristic is called explainability of the variables. The importance of explainability is intimately related to the interpretation done by an expert, as it guarantees that the Machine Learning analysis in the next step only consumes relevant and non-redundant data. This improves the system robustness and critically reduces the amount of false positives during the next phase.
The technique implemented to perform the new feature extraction preprocessing package or stage 209 is called system identification. In comparison to conventional ways of performing system identification (iPARSIMonious-based Subspace Identification [2], [3], ARX, ARIMAX, ARIMA, OE, BOX Jenkins, Spectral Identification [4], HAVOK [5], among others), which are offline solutions based on finite element models, the proposed implementation is based on stochastic system identification developed to work in real time. This new approach provides a related probability of dynamic response variable estimation, exponentially increasing the explainability of the result.
Block 209 for extraction of new features (poles) encompasses novel system identification techniques to robustly identify and characterize the behavior of the structure’s modal parameters over time. In a particular embodiment, the poles are extracted using HAVOK. Alternatively, poles can be extracted using a stochastic approach. The proposed scheme identifies the modal parameters (poles) employing HAVOK (the Hankel alternative view of the Koopman operator) for pulses exhibiting similar linear behavior in bridge responses. HAVOK provides a mathematical framework for investigating the behavior of nonlinear systems by approximating the Koopman operator. This approach enables the identification of linear and non-linear behavior in dynamic systems. After extracting frequency features over time, they can be characterized as stochastic processes. These processes can be fitted to various probabilistic distributions. In preferred embodiments of this stage, Bayesian inferences approaches are used, which treat model parameters as random variables and incorporates prior knowledge to provide a more robust estimate, essentially self-regularizing the model. Thus, by using Bayesian inference techniques, it is possible to characterize the probabilistic parameters of these features by treating them as regularizable random variables. This approach accommodates changes that might arise due to varying operating conditions, ensuring consistent and
robust monitoring. In sum, for each extracted pole (natural frequency and damping factor), a value of estimation probability is also obtained, which provides information of the estimation precision of the algorithm. Figure 7 shows an example of an impulse response from which 3 poles have been obtained by applying the former approach. Figures 8A-8B show the evolution of pole n° 1 (the most relevant in time). Figure 9A shows instant natural frequencies of the three obtained poles as captured for a certain time period. The histograms of Figures 9B-9D represent the distribution of the values of the three poles (in this case, unimodal Gaussian distribution).
The next block 212 in the edge node 20 is the processing block. The processing block or module 212 is the block which follows the preprocessing module 206, It is depicted in Figures 1 and 10.
The first substage or package 213 of the processing module 212 implements the generation of a probability-based model which represents the actual dynamic behavior of the structure under supervision during a specific time span, such as week, month, year or several years. It is a probability-based Machine Learning (ML) model which models baseline parameters. This model associates in a probabilistic way the variables affecting the structure (dynamic variables represented by poles (natural frequencies and damping factors) and optionally contextual variables), thus describing how the variables behave and relate to each other. The number of used poles to build the probability-based baseline model determines the dimensions of the model. This probability-based ML baseline model 213 describes the structural behavior of the structure at the time of measurement, as it captures all the possible states from the variables and stores them in probability distributions, which guarantee the generalization and scalability of the model(s). Depending on the circumstances, there may be more than one baseline model. For example, when the structure under supervision is located in an environment having extreme weather conditions (i.e. temperatures), there may be a summer baseline model and a winter baseline model. It can also happen that the first time a baseline model is created, the structure is in good condition, but after several years it has suffered certain degradation, in which case it may be necessary to have a second model reflecting the current degradation of the structure. The probability distributions and the relationship between variables are defined by the baseline parameters.
The type of ML model used determines the type of baseline parameters which will represent the actual behaviour of the structure. In embodiments of this disclosure, the baseline parameters are a Gaussian mixture comprising the following parameters: a number of components, a means vector and a covariance matrix. Figure 11 shows graphically the results of the impulse detection stage 207, new features extraction 209
and generation of probability baseline model 213. The impulse detection stage 207 is represented by a plurality of free load responses 208 obtained for a certain structure along a certain time period. The new features extraction stage 209 is represented by the baseline poles (natural frequencies and damping factors) extracted for each free load response 208. The generation of the baseline model 213 is represented as a data clustering graph showing two clusters based on the distribution of natural periods (or natural frequencies) and damping factors. In this idealized case, two probability-based baseline models have been obtained (for example one representing the summer season and the other one representing the winter season).
In an embodiment, as shown in Figure 10, in which there are contextual variables captured by the sensors 10, baseline poles 210 and contextual variables (preferably conditioned contextual data stream 205) are taken into account to build the baseline model 213. This enables to guarantee abstraction from normal variations created by the environment, for example, an increase on the structure rigidity caused by a weather temperature decrease. As the baseline model is Machine Learning-based, some algorithms automatically infer the relationship between dynamical and contextual variables, improving the detrending of the result.
The baseline model 213 is also fed with an additional input, which is the concept drift feedback 215, explained in detail later. The concept drift feedback 215 indicates that the baseline model 213 needs to be updated with a specific subset of the data stream 202. This information helps to launch the baseline model updated with previous knowledge.
The result of the probability-based ML baseline model 213 is a set of model parameters 214 of reference that enable a specific measure of fitness which depends on the algorithm used to generate the baseline model 213. These model parameters 214 describe everything needed from the baseline (print of the structure) to evaluate said fitness in a next step of the analysis.
Depending on the application (type of structure, characteristics thereof, relevant environmental conditions, etc.), the generation of the baseline model 213 can be done using different types of algorithms, as described below. All these algorithms associate the variables affecting the structure in a probabilistic way, thus enabling to obtain probabilistic distributions of the possible states of each variable. The structural baseline is built using the information available from the data stream that is consumed. As described before, the main goal of this baseline model is to capture all specific multidimensional behaviours (i.e. behaviours defined by more than one variable) at a measuring time interval. As in real life these behaviors have natural variability, the main
characteristic of the algorithms is their ability to store these behaviours into probability distributions. Some examples of Machine Learning algorithms used to generate the baseline model 213 are mentioned next.
In a possible implementation of the package 213 of the processing module 212, a Gaussian mixture model (GMM) [6] is used. In this case, it is assumed that the inputs coming from previous steps follow Gaussian behaviors. GMM models are probabilistic models that are able to work within multidimensional scenarios. A baseline model obtained using a GMM has the following parameters: a vector containing the number of components (number of normal distributions of a mixture, a mixture being a combination of two or more Gaussian distributions), a vector containing the means per each component and a matrix of variance-covariances. The GMM is made of a finite number of components, each of which follows a multivariate normal distribution, and will be learned from the data (natural frequencies and damping factors) using an Expectation- Maximization (EM) algorithm. The EM algorithm is iterative. The EM algorithm is a statistical algorithm used to estimate the parameters of a probability distribution when some of the variables are missing or unobserved. The algorithm iteratively computes two steps, an expectation step (E-step) and a maximization step (M-step), to update the estimates of the parameters until convergence. In the E-step, the algorithm computes the probability distribution of the unobserved variables given the observed data and the current estimates of the parameters. This step involves calculating the conditional expectation of the complete data log-likelihood, which is a measure of how well the observed data fits the proposed probability distribution. In the M-step, the algorithm updates the estimates of the parameters based on the probability distribution computed in the E-step. This step involves maximizing the expected value of the complete data loglikelihood with respect to the parameters. Thus, the EM algorithm tries to find the parameters (weight of each component of the mixture, vector of means and variancecovariance matrix for each component) that maximize the likelihood of the data given the GMM. The use of this algorithm is oriented to specific structure analysis, as it is the simplest way to deploy the system in terms of implementation and computational requirements. Analysis such as, couple bridge analysis and single beam uniaxial analysis, among others, are good examples of a viable GMM application.
In another possible implementation of the package 213 of the processing module 212, a Gaussian Bayesian network (GBN) [7] is used. A continuous Bayesian network is a Bayesian network in which a conditional probability distribution (CPD) of each variable (inputs, such as the baseline poles 210) is represented using a continuous probability distribution. A GBN is a Bayesian network in which all CPDs are defined using a linear
Gaussian CPD. For this model, it will be assumed that the inputs coming from the previous steps follow Gaussian behaviors. A GBN model assumes that each variable conditional on its parent variables follows a Gaussian density with an expectation that turns out to be a linear combination of the states of its parent variables, and with a variance that is independent of its parents. The learning of the GBN model from different input data is carried out by means of a heuristic approach, based on the hill climb algorithm, which tries to maximize the likelihood of the data given the model. The operators used by the heuristic is based on adding, deleting or changing the direction of an arc of the current structure that the model represents. Each candidate structure is evaluated with the likelihood, without any type of penalty for its complexity. To prevent the properties of monotonicity of the likelihood with respect to the model structure (higher likelihoods are always associated with totally dense structures) from leading to complete structures, a bootstrap sampling is carried out (based on n samplings with replacement of the base of original data and with its same size). The most frequent arcs among all the induced structures will be those that constitute the GBN structure. The use of this algorithm is oriented to specific structure analysis in which it is important to evaluate the relationships between the variables. This algorithm is more complex to implement and requires low-to-medium computational power. Non-limiting examples of analysis of structures to which it can be applied are: temperature detrend couple structure analysis and multi-girder uniaxial analysis, among others.
In another possible implementation of the package 213 of the processing module 212, a Semiparametric Bayesian network (SPBN) [8] is used. As particular CPDs do not fit well into multivariate Gaussian distributions (for example, bimodal distributions), GBNs perform poorly when modeling such distributions. Furthermore, as linear Gaussian CPDs are linear in nature, they are inapplicable to represent nonlinear interactions between random variables. For example, GBNs usually performs poorly to model the damping factors as damping factors usually follow a Gamma distribution. An alternative approach to avoid assuming data normality is to combine Bayesian network models with other nonparametric estimation models that are more flexible compared with parametric estimation models as they do not assume any type of parametric distribution. In this case, the use of kernel density estimation (KDE) with multivariate Gaussian kernels enables dealing with non-Gaussian data. Bayesian networks integrating this approach are named as KDEBNs (kernel density estimation Bayesian networks). Therefore, Bayesian networks joining GBNs and KDEBNs are defined as a particular case of SPBN. The use of this algorithm is oriented to specific structure analysis, where there is now clear understanding of Gaussianity of the variables or their CPD change in an unknown pattern
along time. This algorithm also has the ability to evaluate the relationships between the variables. Some examples of applications are related to the introduction of contextual variables such as humidity and wind speed together with multiaxial analysis of decoupled structural models (multi-girder and/or multi-axial).
Figure 12 shows an example of probability-based baseline model generated using GMM (Gaussian mixture model).
In structural applications, the training and calibration time of the model is of outmost importance. In the present case, the creation of the probabilistic baseline model 213 can start while the structure is under supervision (in other words, from the very moment the structure starts being supervised), without requiring any previous training. As the Machine Learning algorithms previously described need a certain set of data instances or examples to generate the baseline model, system calibration time depends on the size of this set. Therefore, to prevent the system from starting from scratch during new implementations, and thus reduce to time-to-operation, a consensus approach can be optionally used. Consensus learning is a type of machine learning algorithm that combines the predictions of multiple models or experts to arrive at a final prediction or decision. The goal of consensus learning is to improve the accuracy, robustness, and generalization of the learned model by reducing the variance and bias of individual models. In particular, a federated learning approach is selected. The federated learning approach is a type of consensus learning where the models are trained locally on distributed data sources, and the updated models are periodically aggregated and averaged, for example over a central server. Therefore, a consensus baseline model is preferably used in the generation of the baseline model 213, as it directly affects the preliminary calibration of the system. The consensus baseline model contributes to the creation of a baseline model 213 starting from previous knowledge. Because it is based on a model obtained from previous knowledge, it accelerates the generation of a robust baseline model 213. For example, supposing that a bridge needs about 90,000 impulses (equivalent to about one month of traffic circulation) to obtain a robust baseline model 213, if a consensus model is additionally used, a robust baseline model 213 can be achieved with only about 500 impulses (less than one day of traffic circulation).
Consensus typically refers to types of structures, meaning that consensus models can be established and used for structures of the same type. Structures can be grouped attending to their different types, such as bridges, viaducts and buildings, and with different levels of granularity, such as made of reinforced concrete, made of iron, etc. Therefore, different baselines obtained for different types of structures can be classified in terms of the type of the structure. For example, a consensus model can be established
(and later used to generate a baseline model 213) for concrete bridges using predictions of several models of different concrete bridges, another consensus model can be established for steel bridges using predictions of several models of different steel bridges, or another consensus model of a bridge bay can be established using predictions of several models of bays of a bridge (bays having a similar structure).
The consensus learning also enables to find out the classification of the structure when there is available a collection of models (such as GMM, GBN or SPBN) for different types of structures, such as several consensus models for concrete brides or several consensus models for steel bridges. A collection of models has in common that they all store fundamental and common structural information from all the similar analyzed structures. This is also called transfer learning, which is a machine learning technique where a model developed for a specific task is reused as the starting point for a model on a second task. Rather than training a model from scratch, transfer learning leverages knowledge gained from solving a related problem and applies it to a new, but similar, task. This approach is particularly valuable when labeled data for the target task is limited, as it allows the model to benefit from knowledge gained during the training process on the source task. Transfer learning helps improve the performance of models on new tasks, accelerates training, and can lead to more effective and efficient learning across a range of applications.
In sum, consensus models permit to have a starting baseline model in terms of parameters that can be fed into the system to start working as soon as possible. Then, local behaviors of the structure are also learned and updated while the system is working, as explained next. In sum, a consensus approach permits the system to start from scratch during new implementations, reducing the time-to-operation.
The result of the probability-based ML baseline model 213 (together with the consensus model if this has been used) is a set of model parameters, referred to as ML baseline parameters 214, that enable a specific measure of fitness which depends on the algorithm used to generate the baseline model. Once the baseline parameters 214 are generated, the next step is to evaluate their fitness to new instances or examples (online poles 211) that are coming from the new features extraction stage 209. We turn now back to Figures 4 and 10 and consider that a probability-based ML baseline model 213 representing the actual dynamic behaviour of the structure has already been generated (as represented for example in Figure 12, in which Euclidean distances are represented in both axis for different groups 0-5). Now, every time new poles 211 (composed of pair of natural frequency and damping factor) are estimated from a (preferably conditioned) dynamics data stream 204 (in turn obtained from the capture of the sensors 10), these
new poles (referred to as online poles 211 to reflect the real-time character of these parameters) 211 , together with the baseline parameters 214, are fed to a new stage 216 for fitness evaluation. This stage (software-based package) of fitness evaluation is called concept drift detection 216. A concept drift is defined as a change over time in the relation between input data (a set of online poles 211) and a target variable (the baseline model 213 representing the dynamic response of the structure). This change can be triggered by specific changes in behavior, such as degradation or deterioration, common in many structural environments and helpful for early failure detection. The concept drift detection stage 216 is based on an algorithm that has the aim of reducing computational costs and avoiding the use of thresholds. [9]— [11].
The concept drift algorithm evaluates a fitness of the new poles (currently obtained from on-line sensor measurements) to the probability-based baseline model. The fitness is typically represented by a distance between each pole falling outside the baseline model and the baseline model itself, the distance being indicative of a dynamic response degradation of the structure. The fitness can be, for example, a likelihood value. In other words, each new pole is assessed in terms of likelihood to the baseline model. When this likelihood is below a certain threshold, the pole is considered an outlier. When a certain number of outliers show up within a certain time period, it is considered that a concept drift has occurred.
In the present case, unlike conventional techniques for supervising structures, the concept drift algorithm 216 evaluates the fitness of a current instance (online poles 211) to a baseline model 213, as disclosed for example in [15] using actual data streams which are discarded once consumed. The algorithm provides a fitness (KPI 217) that can be evaluated using likelihood analysis, or penalized likelihood such as Bayesian Information Criteria (BIC), Akaike Information Criteria (AIC) or any other probabilistic based approach. This fitness measure (KPI 217) is sent to a next block 218 of the processing module 212. A KPI 217 can for example be a unidimensional value or a temporal series that may express positive or negative tendencies which provide information of the degradation of the structure. A KPI 217 is schematically represented in Figure 13 as the result of the comparison between the online poles 211 (represented as “Future data”) and an exemplary baseline model parameters 214. The KPI is in Figure 13 represented as an “Indicator of Change in Response”.
Figure 15 shows a flow diagram of the already described method for on-line monitoring a dynamic response of a structure, according to a particular embodiment of the invention.
Once the fitness is evaluated and a KPI 217 has been obtained, the fitness level can optionally be monitored, for example by an outlier detection algorithm such as Page- Hinkley [12], [13], to decide if an instance (online poles 211 instantly obtained from recently obtained (real time) measurements captured by sensors 10 in a structure) fits into the baseline model generated at block 213 or if the instance is an outlier. The number of outliers detected can be monitored and accumulated as input information for a next concept drift detection step (for future online poles 211). This is referred to as concept drift feedback 215. A final decision if the new instance is no longer represented by the baseline model can be taken by a probability-based outlier evaluator algorithm which evaluates a probability to find more outliers [14], When a concept drift is detected, several physical interpretations are possible:
1.- The dynamic response of the structure has change from the baseline model, i.e., potential degradation or damage is detected. This situation can trigger an alarm along with the KPI information 217.
2.- The baseline model is no longer valid and it needs to be updated. In this case, feedback 215 is sent to the probability-based Machine Learning baseline stage 213. This feedback 215 contains information about the subset of data to be used during the updating.
As the concept drift detection block 216 ultimately evaluates the fitness of the baseline model, it is also useful to determine controlled changes of the model in terms of external factors, such as seasonal conditions and triggering different consensus models depending on the season.
These changes in the model represent a dynamic response pattern of the structure. The dynamic response pattern of the structure is generated by comparing distances obtained from different dynamic responses of the structure at different times. The dynamic response pattern is indicative of the evolution of the dynamic response degradation of the structure over time. When the dynamic response pattern is projected over time, a remaining useful time of the structure can be estimated.
Additionally, the detection of the concept drift is also useful to update consensus baselines models that may being used. In this case, a consensus baseline model is taken as the starting point, where the concept drift algorithm 216 detects that the baseline model is not completely fitted. Therefore, an update is launched with the specific set of data increasing the fitness of the consensus-based new baseline model.
The KPI 217 obtained in the concept drift detection stage 216 can be sent to a supervision platform, such as a dashboard 310 (usually at a remote location).
Alternatively, the KPI 217 obtained in the concept drift detection stage 216 can optionally be used to estimate the remaining useful life (RUL) of the structure under supervision. To achieve this, the processing module 212 may include blocks 218 and 220, as schematically shown in Figures 1 and 10. RUL estimation of structures is a common field of development within the structural engineering. Methods of RUL estimation 220 are often statistics-based depending on the behavior distribution of the measured variable.
In a possible embodiment, a KPI 217 is a unidimensional value where its distribution can be modelled through traditional Weibull, Gaussian, or Gamma distribution to estimate the expected useful life. Additionally, a new methodology can be applied to obtain at module 218 a health index (HI) 219 by calculating multiple exponential regressions at each sampling time, considering only incremental samples. The output from this module 218 is a health index 219, such as a curve, containing information about the dynamic response of the structure and its performance along time. For example, assuming that a curve of useful life of a certain structure is well-known, the health index 219 may indicate actual deviations from the original curve.
Finally, the KPI 217, or the health index 219 in the event this information is available, is taken to a software-based block 220 for obtaining the remaining useful life (RUL) of the structure. Block 220 takes the information (KPI 217 or health index 219). Health assessment 221 provided by the structural RUL block 220 for example comprises at least one of a health index, a RUL indicator, which are typically float values, and a concept drift detection alarm, which is typically a Boolean value. Therefore, the output of the RUL block 220 is provided in the form of a vector of several values. For example, it is vector of four values: time stamp, health index, RUL and alarm. Figure 14 represents the concept of “fitness” and shows an example of KPI obtained after comparing new observations (impulses) against the baseline model: if a new impulse fits the model, the KPI is low value (100% fitting indicates KPI » 0), while if a new impulse fits poorly, the KPI is a large value (0% fitting indicates KPI » «).
The health assessment 221 is the output of the processing module 212. The last module of the edge device 20 is a communication interface 222 implementing a communication layer (Figure 1). Communication interface 222 is software-based and is aimed at integrating the health assessment information 221 into platform layers such as cloud services, in-premise clouds, servers and control rooms, among others. Therefore, the objective is to format the health assessment information 221 in the required communication protocols: Unified Architecture of Open Protocol Communications (OPC- UA), Message Queuing Telemetry Transport (MQTT), TCP and UDP, among others, understandable by the corresponding packages in the platform layer. This formatted
information is referred to as 223 in Figure 1 . The communication interface 222 can also be used to send information into a hard drive of the edge device 20 to temporally store the data and send it when communications are available. In embodiments in which the location of the structure under supervision is limited by energy-consumption because of its limited connectivity, the communication interface 222 can be implemented following a batch-mode configuration to send data in the required time windows, for example, when satellite link is available, reducing its use and the cost.
The third architectural level of the system 1 is the remote system or platform 30, also referred to as platform layer. The remote system 30 is typically physically located far away from the structure under supervision and, therefore, from the edge device 20. Nonlimiting examples of remote systems 30 are cloud services, in-premise clouds, servers and control rooms, among others. When the formatted information 223, formatted in the suitable communication protocol, is received in the remote system 30, the contained fields are represented differently in a dashboard 310 depending on the nature of each individual value. For example, in the case of health index, as a unidimensional value along time, it can be represented as a time series plot. This is particularly useful in order to visually analyze the evolution of the KPI. Other values such as RUL can be represented in terms of remaining time (hours, years, etc.), or the alarm as a light signpost. The dashboard 310 enables the end user to capture the structure behavior without a deep understanding of structural dynamics or to visually assess a multidimensional analysis with all the data available from the structure.
It is remarked that the information represented by the dashboard 310 is objective information extracted from the data produced by the structure under supervision. Therefore, no subjective approaches are provided, improving the evaluation of the structural responses.
Additionally, as the information transmitted by each structure is highly condensed, i.e. , it only produces formatted information 223 when an impulse is detected, the scalability of the solution is improved. For example, having condensed and critical information from the bridges helps a bridge maintainer, who typically has more than 1 ,000 of bridges to manage, to perform a fleet assessment over the entire number of them. Moreover, this information can help take decisions, as explained next.
The remote system 30 further includes a decision-making module 320 following the dashboard 310. From the structural information 315 provided by the dashboard 310, different actionable insights can be extracted:
Health index value: When performing fleet analysis, the Health Index value of different managed structures can be used to rank them in terms of its health. This information is useful to prioritize maintenance activities and budget allocation.
Temporal change in the Health Index: If there are important changes of the Health Index at the dashboard, such as presence of outliers or presence of a trend in its values, early warning signs can be detected. This information can be useful to increase traditional inspection protocols in the specific structure.
RUL: As the remaining useful life is a value of time expected until failure of the structure, preventive, predictive and prescriptive maintenance activities can be performed accordingly. This value can also be used as a ranking value when a fleet of structure is managed. This is also important for maintenance budget allocation and inspection activities.
Alarm: The main decision to be taken when an alarm is triggered is to inspect the structure to validate the reason of the change in the dynamic response. A change in the dynamic response of the structure is an early indicator of structural degradation. This alarm is also useful for corrective maintenance planning if used with other indicators that can help to diagnose the potential structure issue.
Figure 16 shows an example of a structure being monitored by means of the disclosed method and system. It is a bridge located in Bilbao (Spain) (location 43.264697, - 2.961540). Figure 16 shows the location of the monitoring device and instrumentation 160. The bride has 6 spans. There is one accelerometer per span. Figure 17A-E show real data and results obtained at the illustrate bridge. Figure 17A shows the impulse detection for the first span of the bridge. Figures 17B and 17C show the extraction of natural frequencies and damping factors for the first span, respectively. Figure 17D shows the model baseline for the first span. It has been done with SPBN obtained algorithm. All variables are non-Gaussian so it is modelled by kernels (grey nodes). Figure 17E represents an KPI based on likelihood for the first span of the bridge. Figures 18A-18H show an online dashboard for the first span including results of one-year monitoring (358,095 cars detected). The first graph (Figure 18A) represents detected natural frequencies. The second one (Figure 18B) represents the KPI. Figures 18C-18H represent histograms with the distribution of the values of the three main poles (in Figures 18C-18E the natural frequencies for poles 1 , 2 and 3, respectively; in Figures 18F-18H the damping factors for poles 1 , 2 and 3, respectively).
In this text, the term “comprises” and its derivations (such as “comprising”, etc.) should not be understood in an excluding sense, that is, these terms should not be interpreted
as excluding the possibility that what is described and defined may include further elements, steps, etc. The term “another,” as used herein, is defined as at least a second or more. The term “coupled,” as used herein, is defined as connected, whether directly without any intervening elements or indirectly with at least one intervening elements, unless otherwise indicated. Two elements can be coupled mechanically, electrically, or communicatively linked through a communication channel, pathway, network, or system.
The invention is obviously not limited to the specific embodiments described herein, but also encompasses any variations that may be considered by any person skilled in the art (for example, as regards the choice of materials, dimensions, components, configuration, etc.), within the general scope of the invention as defined in the claims.
BIBLIOGRAPHIC DATA
[1] Limin Sun, P.E. et al, Review of Bridge Structural Health Monitoring Aided by Big Data and Artificial Intelligence: From Condition Assessment to Damage Detection, J. Struct. Eng., 2020, 146(5): 04020073.
[2] G. Pannocchia y M. Calosi, «A predictor form PARSIMonious algorithm for closed-loop subspace identification®, J. Process Control, vol. 20, n.° 4, pp. 517-524, abr. 2010, doi: 10.1016/j.jprocont.2010.01 .004.
[3] G. Pannocchia y M. Calosi, «Closed-Loop PARSIMonious Subspace Identification: Theory and Application to MPC», IFAC Proc. Vol., vol. 43, n.° 5, pp. 361- 366, ene. 2010, doi: 10.3182/20100705-3-BE-2011.00060.
[4] L. Ljung, «System Identification®, en Signal Analysis and Prediction, A. Prochazka, J. Uhlfr, P. W. J. Rayner, y N. G. Kingsbury, Eds. Boston, MA: Birkhauser, 1998, pp. 163-173. doi: 10.1007/978- 1-4612- 1768-8_11.
[5] S. L. Brunton, B. W. Brunton, J. L. Proctor, E. Kaiser, y J. N. Kutz, «Chaos as an intermittently forced linear system», Nat. Commun., vol. 8, n.° 1 , Art. n.° 1 , may 2017, doi: 10.1038/S41467-017-00030-8.
[6] G. McLachlan y D. Peel, Finite Mixture Models. John Wiley & Sons, 2004
[7] D. Geiger y D. Heckerman, «Learning Gaussian Networks®, en Uncertainty Proceedings 1994, R. L. de Mantaras y D. Poole, Eds. San Francisco (CA): Morgan Kaufmann, 1994, pp. 235-243. doi: 10.1016/B978-1-55860-332-5.50035-3.
[8] D. Atienza, C. Bielza, y P. Larrahaga, «Semiparametric Bayesian networks®, Inf. Sci., vol. 584, pp. 564-582, ene. 2022, doi: 10.1016/j.ins.2021.10.074.
[9] J. Diaz-Rozo, C. Bielza, y P. Larranaga, «Machine-tool condition monitoring with Gaussian mixture models-based dynamic probabilistic clustering)), J. Eng. Appl. Artif. Intell., 2019.
[10] J. Diaz-Rozo, C. Bielza, y P. Larranaga, «Machine-tool condition monitoring with Gaussian mixture models-based dynamic probabilistic clustering)), Eng. Appl. Artif. Intell., vol. 89, p. 103434, 2020.
[11] J. Diaz-Rozo, «Clustering probabilistico dinamico para la busqueda de patrones de degradaton de elementos de maquina en el ambito del Industrie 4.0», phd, E.T.S. de Ingenieros Informaticos (UPM), 2019. Accedido: 27 de febrero de 2020. [En linea], Disponible en: http://oa.upm.es/56472/
[12] E. S. Page, «Continuous inspection schemes)), Biometrika, vol. 41, n.° 1/2, pp. 100-115, 1954.
[13] H. Mouss, D. Mouss, N. Mouss, y L. Sefouhi, «Test of Page-Hinckley, an approach for fault detection in an agro-alimentary production system», en Proceedings of the 5th Asian Control Conference, 2004, vol. 2, pp. 815-818.
[14] H. Chernoff, «A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations)), Ann. Math. Stat., pp. 493-507, 1952.
[15] Diaz-Rozo, Javier et al (Clustering of Data Streams With Dynamic Gaussian Mixture Models: An loT Application in Industrial Processes, IEEE Internet of Things Journal, Vol. 5, N° 5, October 2018)
[16] Li Dai et. Health-Monitoring-Oriented Finite Element Model for a Specially Shaped Steel Arch Bridge and Its Application, Mathematical and Computational Applications, 2023, 28, 33.
Claims
1 A computer implemented method for on-line monitoring a structure, the method comprising: receiving (151) at least dynamics data (204) of dynamic variables representing a dynamic response of the structure from a plurality of sensors (10), the dynamic response of the structure being generated when the structure has been subjected to an excitation; detecting an impulse (152) created by said excitation by identifying a free load response (208) of the structure from said at least dynamic data (204); obtaining (153) a set of poles (210) comprising pairs of a natural frequency and a damping factor of the structure from the free load response (208); generating (154), using a non-supervised machine learning algorithm, a probabilitybased baseline model for the structure using a plurality of said poles (210) obtained along a certain period of time, the probability-based baseline model associating in a probabilistic way the dynamic variables affecting the structure and providing a plurality of baseline parameters (214); every time a new impulse is detected and a new set of poles (211) are obtained from the free load response of said impulse, feeding (155) the new set of poles (211) to a concept drift algorithm to obtain a key performance indicator, KPI (217) of the structure.
2.- The computer implemented method according to claim 1 , further comprising, together with receiving at least dynamics data (204) from the plurality of sensors (10), receiving from the plurality of sensors (10) contextual data (205) of contextual variables representing the environment surrounding the structure; wherein the generating (154), using a machine learning algorithm, a probabilitybased baseline model for the structure, also uses said contextual data (205).
3.- The computer implemented method according to either claim 1 or 2, further comprising, prior to identifying a free load response of the structure from said dynamic data (204), carrying out data conditioning of the dynamics data received from the plurality of sensors (10), said data conditioning comprising data cleaning and real-time data imputation.
4.- The computer implemented method according to claims 3 and 4, further comprising, prior to identifying a free load response of the structure from said dynamic data (204),
carrying out data conditioning of the contextual data received from the plurality of sensors (10), said data conditioning comprising data cleaning and real-time data imputation.
5.- The computer implemented method according to any one of the preceding claims, further comprising, once a KPI (217) has been obtained, monitoring a fitness level to decide if the new online poles (211) fit into the baseline model (213) and, if a certain number of outliers are identified, providing concept drift feedback (215) to the machine learning algorithm to update the probability-based baseline model.
6.- The computer implemented method according to any one of the preceding claims, further comprising estimating a remaining useful life of the structure based on a health index interpreted from the KPI.
7.- The computer implemented method according to any one of the preceding claims, wherein the excitation the structure is subjected to is generated by at least one of the following phenomena: a passage of at least one vehicle on the structure, a gust of wind, a force exerted by a material when loaded into the structure, a force or slapping of sea waves, an earthquake and any combination thereof.
8.- The computer implemented method according to any one of the preceding claims, wherein the variables or parameters providing data related to the dynamic response of the structure to an external excitation are at least one of: vibration, displacement, inclination and force.
9.- The computer implemented method according to any one of the preceding claims, wherein the variables or parameters providing contextual data related to the environment surrounding the structure, are at least one of: surface and environment temperature, environment humidity, wind speed and wind direction, rain drops, snow, frost, ice or season of the year.
10. -The computer implemented method according to any one of the preceding claims, comprising using a technique for carrying out the real-time data imputation of the data streams that is selected from a group comprising: last-value technique, mean value technique, interpolated value technique and any combination thereof.
11.- The computer implemented method according to any one of the preceding claims, wherein detecting an impulse (152) created by said excitation by identifying a free load
response (208) of the structure from said at least dynamic data (204) is done by implementing at least one classifier configured to detect transient regimes.
12.- The computer implemented method according to claim 11 , wherein two classifiers configured to detect transient regimes are implemented.
13.- The computer implemented method according to claim 12, wherein detecting an impulse (152) created by said excitation is done as follows: calculating (61) from the dynamics data (204) a sample moment-time representation (SMTR) curve rc(t) of the dynamical system in a previously selected time window th; computing (62) the arc length lr(t) of the SMTR curve rc(t); generating (63) a piecewise function hTsi(t) using a threshold lroand the arc length lr(t); classifying (64) the system’s time periods as either transitory or stationary; using the SMTR curve rc(t) of the dynamical system, computing (65) its matrix of derivatives Dc(t); calculating (66) time segments where the determinant of the matrix of derivatives Dc(t) is zero; if the determinant obtained is zero, segmenting (67) the curve representing the dynamic system into flat curves ryj(t) and calculating their flat curvature kn(t); if the determinant obtained is not zero, computing (68) generalized curvatures ki(t) of the SMTR curve rc(t); calculating (69) a piecewise function kc(t) which stores the magnitude of the generalized and flat curvatures of the studied system; generating (70) a piecewise function hTsii(t) using a threshold kco and the magnitude of the curvatures ki(t); using the piecewise function hTsii(t) to classify (71) the system’s time periods as either transitory or stationary.
14.- The computer implemented method according to any one of the preceding claims, wherein obtaining a set of natural frequencies and a damping factor of the structure from the free load response is carried out by a system identification algorithm implementing system identification techniques selected from a group comprising system identification
techniques based on deterministic algorithms, system identification techniques based on probabilistic algorithms and any combination thereof.
15.- The computer implemented method according to any one of the preceding claims, wherein the machine learning baseline model algorithm comprises at least one of gaussian mixture models, gaussian Bayesian networks, semiparametric Bayesian networks, and any combination thereof.
16.- The computer implemented method according to any one of the preceding claims, comprising triggering an alarm when the trend in a health evolution of the structure is above a certain threshold.
17.- The computer implemented method according to any one of the preceding claims, wherein the structure is a bridge, a viaduct, a dam, a warehouse, a silo, a marine platform, or any other building or infrastructure requiring structural health surveillance.
18.- A device (20) for on-line monitoring a structure, comprising: means for receiving at least dynamics data (204) of dynamic variables representing a dynamic response of the structure from a plurality of sensors (10), the dynamic response of the structure being generated when the structure has been subjected to an excitation; means for detecting an impulse created by said excitation by identifying a free load response (208) of the structure from said at least dynamic data (204); means for obtaining a set of poles (210) comprising pairs of a natural frequency and a damping factor of the structure from the free load response (208); means for generating, using a machine learning algorithm, a probability-based baseline model for the structure using a plurality of said poles (210) obtained along a certain period of time, the probability-based baseline model associating in a probabilistic way the dynamic variables affecting the structure and providing a plurality of baseline parameters (214); means for feeding, every time a new impulse is detected and a new set of poles (211) are obtained from the free load response of said impulse, the new set of poles (211) to a concept drift algorithm to obtain a KPI (217) of the structure.
19.- A system (1) for monitoring a structure, comprising:
a plurality of sensors (10) attachable to the structure to be supervised, the sensors being capable of capturing data related to a dynamic response of the structure when it is subjected to an excitation; a device (20) according to claim 18, wherein the device is an edge device situated in the vicinity of the structure to be supervised; and a remote system (30) configured to receive from the edge device information about the health of the structure to be supervised.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| EP24382271.5 | 2024-03-13 | ||
| EP24382271 | 2024-03-13 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2025191082A1 true WO2025191082A1 (en) | 2025-09-18 |
Family
ID=90365630
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/EP2025/056915 Pending WO2025191082A1 (en) | 2024-03-13 | 2025-03-13 | A computer implemented method and system for health monitoring of structures |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2025191082A1 (en) |
-
2025
- 2025-03-13 WO PCT/EP2025/056915 patent/WO2025191082A1/en active Pending
Non-Patent Citations (20)
| Title |
|---|
| CORMEN, T. H.LEISERSON, C. E.RIVEST, R. L.STEIN, C.: "Introduction to Algorithms", vol. 262032937, 2001, MIT PRESS AND MCGRAW-HILL, pages: 636 - 640 |
| D. ATIENZAC. BIELZAP. LARRANAGA: "Semiparametric Bayesian networks", INF. SCI., vol. 584, 2022, pages 564 - 582 |
| D. GEIGERD. HECKERMAN: "Uncertainty Proceedings", 1994, MORGAN KAUFMANN, article "Learning Gaussian Networks", pages: 235 - 243 |
| DIAZ-ROZO, JAVIER ET AL.: "Clustering of Data Streams With Dynamic Gaussian Mixture Models: An loT Application in Industrial Processes", IEEE INTERNET OF THINGS JOURNAL, vol. 5, no. 5, October 2018 (2018-10-01), XP011704947, DOI: 10.1109/JIOT.2018.2840129 |
| E. S. PAGE: "Continuous inspection schemes", BIOMETRIKA, vol. 41, no. 1-2, 1954, pages 100 - 115 |
| G. PANNOCCHIAM. CALOSI: "A predictor form PARSIMonious algorithm for closed-loop subspace identification", J. PROCESS CONTROL, vol. 20, no. 4, 2010, pages 517 - 524, XP026976903 |
| G. PANNOCCHIAM. CALOSI: "Closed-Loop PARSIMonious Subspace Identification: Theory and Application to MPC", IFAC PROC., vol. 43, no. 5, 2010, pages 361 - 366 |
| H. CHERNOFF: "A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations", ANN. MATH. STAT., 1952, pages 493 - 507 |
| H. MOUSSD. MOUSSN. MOUSSL. SEFOUHI: "Test of Page-Hinckley, an approach for fault detection in an agro-alimentary production system", PROCEEDINGS OF THE 5TH ASIAN CONTROL CONFERENCE, vol. 2, 2004, pages 815 - 818 |
| J. DIAZ-ROZO: "Clustering probabilistico dinámico para la búsqueda de patrones de degradación de elementos de maquina en el ámbito del Industrie 4.0", E.T.S. DE INGENIEROS INFORMATICOS (UPM, 2019, Retrieved from the Internet <URL:http://oa.upm.es/56472> |
| J. DIAZ-ROZOC. BIELZAP. LARRAFIAGA: "Machine-tool condition monitoring with Gaussian mixture models-based dynamic probabilistic clustering", ENG. APPL. ARTIF. INTELL., vol. 89, 2020, pages 103434, XP085987212, DOI: 10.1016/j.engappai.2019.103434 |
| J. DIAZ-ROZOC. BIELZAP. LARRAFIAGA: "Machine-tool condition monitoring with Gaussian mixture models-based dynamic probabilistic clustering", J. ENG. APPL. ARTIF. INTELL., 2019 |
| KIMPARA MARCIO ET AL: "On the Cross Coupling Effects in Structural Response of Switched Reluctance Motor Drives", IEEE TRANSACTIONS ON ENERGY CONVERSION, IEEE SERVICE CENTER, PISCATAWAY, NJ, US, vol. 34, no. 2, 1 June 2019 (2019-06-01), pages 620 - 630, XP011724579, ISSN: 0885-8969, [retrieved on 20190514], DOI: 10.1109/TEC.2018.2863199 * |
| L. LJUNG, SYSTEM IDENTIFICATION», EN SIGNAL ANALYSIS AND PREDICTION, 1998, pages 163 - 173 |
| LI DAI: "Health-Monitoring-Oriented Finite Element Model for a Specially Shaped Steel Arch Bridge and Its Application", MATHEMATICAL AND COMPUTATIONAL APPLICATIONS, vol. 28, 2023, pages 33 |
| LIMIN SUN, P.E. ET AL.: "Review of Bridge Structural Health Monitoring Aided by Big Data and Artificial Intelligence: From Condition Assessment to Damage Detection", J. STRUCT. ENG., vol. 146, no. 5, 2020, pages 04020073 |
| MOHD AKIL TAN M H ET AL: "Classification of materials by modal analysis and neural network", INFORMATION TECHNOLOGY AND MULTIMEDIA (ICIM), 2011 INTERNATIONAL CONFERENCE ON, IEEE, 14 November 2011 (2011-11-14), pages 1 - 5, XP032087950, ISBN: 978-1-4577-0988-3, DOI: 10.1109/ICIMU.2011.6122753 * |
| S. L. BRUNTONB. W. BRUNTONJ. L. PROCTORE. KAISERJ. N. KUTZ: "«Chaos as an intermittently forced linear system", NAT. COMMUN., vol. 8, no. 1, May 2017 (2017-05-01) |
| TOMONORI NAGAYAMA ET AL: "Structural Health Monitoring Using Smart Sensors", NEWMARK STRUCTURAL ENGINEERING LABORATORY REPORT SERIES, 1 November 2007 (2007-11-01), pages 1 - 186, XP055368504, Retrieved from the Internet <URL:https://core.ac.uk/download/pdf/4813669.pdf> [retrieved on 20170502] * |
| WHELAN M J ET AL: "Real-time wireless vibration monitoring for operational modal analysis of an integral abutment highway bridge", ENGINEERING STRUCTURES, ELSEVIER, AMSTERDAM, NL, vol. 31, no. 10, 1 October 2009 (2009-10-01), pages 2224 - 2235, XP026625008, ISSN: 0141-0296, [retrieved on 20090507], DOI: 10.1016/J.ENGSTRUCT.2009.03.022 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11835030B2 (en) | Methods and systems for predicting risk of observable damage in wind turbine gearbox components | |
| Dervilis et al. | On robust regression analysis as a means of exploring environmental and operational conditions for SHM data | |
| US20170220718A1 (en) | Motion Sensing Wi-Fi Sensor Networks for Continuous 3D Modeling and Prediction of Facility Responses to Disturbances | |
| CN119025946A (en) | A dam safety monitoring method and system based on big data | |
| Fritz et al. | Fault diagnosis in structural health monitoring systems using signal processing and machine learning techniques | |
| CN119091594A (en) | A slope stability early warning method based on changes in multi-source monitoring data | |
| Catbas | Structural health monitoring: applications and data analysis | |
| Liu et al. | Anomaly detection of high-frequency sensing data in transportation infrastructure monitoring system based on fine-tuned model | |
| Singh et al. | Structural health monitoring and damage detection through machine learning approaches | |
| Liu et al. | Traffic dynamics exploration and incident detection using spatiotemporal graphical modeling | |
| Guéguen et al. | Consideration of the effects of air temperature on structural health monitoring through traffic light‐based decision‐making tools | |
| Mollineaux et al. | Structural health monitoring of progressive damage | |
| Sargiotis | Transforming civil engineering with AI and machine learning: innovations, applications, and future directions | |
| Ye et al. | Adaptive MAGNN-TCN: An Innovative Approach for Bearings Remaining Useful Life Prediction | |
| Ghiasi et al. | Generalization of anomaly detection in bridge structures using a vibration‐based Siamese convolutional neural network | |
| Silva et al. | Damage detection for structural health monitoring of bridges as a knowledge discovery in databases process | |
| CN119941084B (en) | Multi-dimensional logistics information supervision and early warning system and method based on artificial intelligence | |
| EP4577950A1 (en) | Real time detection, prediction and remediation of machine learning model drift in asset hierarchy based on time-series data | |
| Shaik et al. | Revolutionizing infrastructure resilience: AI-driven predictive maintenance and structural health monitoring | |
| Aftabi et al. | A variational autoencoder framework for robust, physics-informed cyberattack recognition in industrial cyber-physical systems | |
| WO2025191082A1 (en) | A computer implemented method and system for health monitoring of structures | |
| Vagnoli et al. | A machine learning classifier for condition monitoring and damage detection of bridge infrastructure | |
| Sun et al. | A data-driven framework for tunnel infrastructure maintenance | |
| Kamariotis et al. | Quantifying the value of vibration-based structural health monitoring considering environmental variability | |
| Muneshwara et al. | Advance Infratructure Maintenance Using Parallel Data Processing with Visual Analytics and IoT |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 25712853 Country of ref document: EP Kind code of ref document: A1 |