CN112131794A - Hydraulic structure multi-effect optimization prediction and visualization method based on LSTM network - Google Patents
Hydraulic structure multi-effect optimization prediction and visualization method based on LSTM network Download PDFInfo
- Publication number
- CN112131794A CN112131794A CN202011020208.1A CN202011020208A CN112131794A CN 112131794 A CN112131794 A CN 112131794A CN 202011020208 A CN202011020208 A CN 202011020208A CN 112131794 A CN112131794 A CN 112131794A
- Authority
- CN
- China
- Prior art keywords
- lstm
- prediction
- lstm network
- data
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000005457 optimization Methods 0.000 title claims abstract description 26
- 238000007794 visualization technique Methods 0.000 title claims abstract description 14
- 238000000034 method Methods 0.000 claims abstract description 37
- 238000012544 monitoring process Methods 0.000 claims abstract description 24
- 238000012549 training Methods 0.000 claims abstract description 22
- 238000012795 verification Methods 0.000 claims abstract description 12
- 238000012360 testing method Methods 0.000 claims abstract description 10
- 238000005516 engineering process Methods 0.000 claims abstract description 9
- 238000012800 visualization Methods 0.000 claims abstract description 9
- 238000007781 pre-processing Methods 0.000 claims abstract description 5
- 230000000007 visual effect Effects 0.000 claims abstract description 5
- 239000002245 particle Substances 0.000 claims description 42
- 230000000694 effects Effects 0.000 claims description 19
- 239000013598 vector Substances 0.000 claims description 15
- 230000006870 function Effects 0.000 claims description 11
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 230000015654 memory Effects 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 7
- 238000004458 analytical method Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 claims description 6
- 238000009499 grossing Methods 0.000 claims description 6
- 210000002569 neuron Anatomy 0.000 claims description 6
- 230000010354 integration Effects 0.000 claims description 5
- 239000000126 substance Substances 0.000 claims description 5
- 210000004027 cell Anatomy 0.000 claims description 4
- 230000008859 change Effects 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 4
- 238000013501 data transformation Methods 0.000 claims description 4
- 238000000265 homogenisation Methods 0.000 claims description 4
- 230000009467 reduction Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000011156 evaluation Methods 0.000 claims description 3
- 238000012417 linear regression Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 238000012935 Averaging Methods 0.000 claims description 2
- 238000010200 validation analysis Methods 0.000 claims description 2
- 238000013135 deep learning Methods 0.000 abstract description 7
- 238000011160 research Methods 0.000 abstract description 6
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000012706 support-vector machine Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000000737 periodic effect Effects 0.000 description 2
- 238000013179 statistical model Methods 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009933 burial Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 238000013213 extrapolation Methods 0.000 description 1
- 238000003973 irrigation Methods 0.000 description 1
- 230000002262 irrigation Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000035772 mutation Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 238000010248 power generation Methods 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000009418 renovation Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 238000005728 strengthening Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/27—Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/10—Geometric CAD
- G06F30/13—Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
- G06N3/006—Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2111/00—Details relating to CAD techniques
- G06F2111/06—Multi-objective optimisation, e.g. Pareto optimisation using simulated annealing [SA], ant colony algorithms or genetic algorithms [GA]
Abstract
A hydraulic structure multi-effect optimization prediction and visualization method based on an LSTM network comprises the following steps: obtaining prototype observation data; preprocessing the prototype observation data, and dividing a preprocessed data set into a training set, a verification set and a test set; constructing an LSTM prediction model, namely an LSTM network, by utilizing the training set, the verification set and the test set; carrying out hyper-parameter debugging on the LSTM network so as to optimize an LSTM algorithm; carrying out extension prediction on the LSTM prediction model by utilizing a direct multi-step prediction method; and (3) performing visual representation on the LSTM network correlation learning by adopting a t-SNE technology, namely completing the multi-effect optimization prediction and visualization of the hydraulic structure based on the LSTM network. The invention provides a new idea for the safety monitoring method of the hydraulic structure and lays a research foundation for deep learning, popularization and application.
Description
Technical Field
The invention relates to a hydraulic structure safety monitoring method, in particular to a hydraulic structure multi-effect optimization prediction and visualization method based on an LSTM network.
Background
In order to achieve the purposes of flood control, power generation, irrigation, water supply and the like, different types of hydraulic buildings are required to be built to control and allocate water flow, such as water retaining buildings, water delivery buildings, renovation buildings and the like. The structural safety is the premise that the building plays a regulating function, and the safety management provides guarantee for the normal operation of the building, particularly for long-term safety monitoring. Various instruments are arranged at key positions of a hydraulic structure, and the working performance of the hydraulic structure is comprehensively reflected from different dimensions by monitoring deformation, seepage and other effect quantities. According to prototype observation data, a multi-effect quantity mathematical monitoring model is constructed by using methods such as statistics, machine learning and the like, and important changes of the structural performance of the building can be mastered and predicted in time, so that scientific basis is provided for evaluating the safety condition of the building and finding out abnormal signs of the building.
The conventional hydraulic building safety monitoring model is roughly classified into a statistical model, a deterministic model and a hybrid model according to different construction methods. At the end of the last century, artificial intelligence technology has been involved and the development of industrial applications has been rapid. Wu, for example, firstly applies modeling methods such as an Artificial Neural Network (ANN) and a Support Vector Machine (SVM) to the safety state analysis of the hydraulic buildings. Then, modeling and predicting different effect quantities by utilizing a machine learning algorithm is always a research hotspot in the field of safety monitoring of hydraulic buildings. At present, water conservancy informatization construction focuses on upgrading and transforming monitoring instruments and information integration systems, and research and innovation of data analysis methods are less concerned. Although shallow learning algorithms such as ANN and SVM are greatly improved in nonlinear information extraction compared with statistical models, monitoring requirements are still difficult to meet in some scenarios. Deep learning is a breakthrough in ANN development, and the advantages of the deep learning in the aspect of implicit information mining are obvious because the deep learning utilizes a complex structure or multiple nonlinear transformation processing layers to perform high abstraction on data. The Long Short-Term Memory network (LSTM) can fully mine the time dependence in time sequence data to increase information dimensionality, and therefore high-precision prediction of monitoring quantities such as landslide displacement and underground water burial depth is achieved. The LSTM network still has deficiencies and can be optimized by methods such as integration, parameter debugging and the like. In the field of hydraulic building safety monitoring, application exploration of deep learning algorithms such as LSTM and the like is rarely reported, only a small amount of research is carried out on the application exploration for dam deformation prediction, and the application exploration is not extended to other hydraulic buildings at present and optimization processing is not carried out on characteristics of monitoring data.
Therefore, there is a need for an optimized method for monitoring the safety of hydraulic buildings.
Disclosure of Invention
In view of the above, the main objective of the present invention is to provide a hydraulic building multi-effect optimization prediction and visualization method based on LSTM network, so as to partially solve at least one of the above technical problems.
In order to achieve the above object, as an aspect of the present invention, there is provided a hydraulic structure multi-effect optimization prediction and visualization method based on an LSTM network, including the following steps:
obtaining prototype observation data;
preprocessing the prototype observation data, and dividing a preprocessed data set into a training set, a verification set and a test set;
constructing an LSTM prediction model, namely an LSTM network, by utilizing the training set, the verification set and the test set;
carrying out hyper-parameter debugging on the LSTM network so as to optimize an LSTM algorithm;
carrying out extension prediction on the LSTM prediction model by utilizing a direct multi-step prediction method; and
and (3) performing visual representation on the LSTM network correlation learning by adopting a t-SNE technology, namely completing the multi-effect optimization prediction and visualization of the hydraulic structure based on the LSTM network.
Wherein, the preprocessing of the prototype observation data comprises data cleaning, noise reduction smoothing, data transformation and data homogenization.
The LSTM prediction model in the step of constructing the LSTM prediction model by utilizing the training set, the verification set and the test set is composed of an input layer, a plurality of hidden layers and an output layer, the output of the previous hidden layer is used as the input of the next hidden layer, the input layer and the hidden layers jointly realize the extraction of input effect quantity data characteristics, the output of the last hidden layer is a one-dimensional column vector, and the predicted value of the processed effect quantity data is obtained through linear regression.
Wherein, the memory module of the LSTM network is composed of a forgetting gate (f)t l) Input gate
And output gate
And (4) forming. Forgetting to control internal state at last moment
The input door controls the candidate state of the current time
The amount of information to be stored, and the output gate controls the internal state at the current time
Need to be output to external state
The amount of information of (2). The corresponding calculation process is as follows:
(1) using external state at last moment
And current time input
Calculate ft l、
And
as shown in formulas (1) - (3);
(2) combination ft lAnd
updating memory cell states
As shown in formula (4);
(3) by passing
Will be provided with
Information delivery to
As shown in formulas (5) - (6);
wherein, σ (·) and tanh (·) are respectively a Sigmoid function and a hyperbolic tangent function; w and b are respectively a weight matrix and an offset vector; an indicator indicates a scalar product of two vectors.
Wherein, the step of performing hyper-parameter debugging on the LSTM network so as to optimize the LSTM algorithm specifically comprises:
constructing a deep LSTM network, overlapping and combining a plurality of LSTM networks by adopting a stacking integration method to obtain an integrated LSTM network, carrying out unitary time sequence prediction, and carrying out epitaxial analysis on future change trend of effect quantity directly according to historical monitoring data; and
and carrying out hyper-parameter debugging.
Wherein, the super-parameter debugging is realized by a particle swarm optimization algorithm, and the length w of the time window is calculatedtNumber n of hidden layershThe number of nodes n in each hidden layernLearning rate lrAnd optimizing the number of iterations niCarrying out iterative optimization on equal hyper-parameters, and selecting a hyper-parameter combination with the minimum corresponding fitness value
Wherein the performing hyper-parameter debugging further comprises:
initializing parameters; determining a population scale, iteration times, learning factors and limited intervals of position and speed values;
initializing the position and speed of the particles; randomly generating a population of particles Xi，0(wt，n1，n2，lr，ni) (ii) a Wherein n is1Number of neurons representing the first hidden layer, n2Representing the number of neurons in the second hidden layer;
determining an evaluation function of the particle; the particles X obtained in the last step are treatedi，0For LSTM netAssigning parameters of the network; dividing data into training samples, verifying samples and predicting samples; inputting the training sample to train the neural network, and obtaining the training sample output value of the neural network after the iteration times are limited
And verifying the sample output value
Then the individual XiFitness value fit ofiIs defined as:
wherein the content of the first and second substances,
and
respectively obtaining an expected output value of a training sample and an expected output value of a verification sample;
calculating the position X of each particleiDetermining individual extremum and group extremum according to the initial particle fitness value and taking the best position of each particle as the historical best position;
during each iteration, updating the speed and position of the particle itself through the individual extremum and the global extremum according to equations (7) and (8); calculating a new particle fitness value, and updating the individual extreme value and the population extreme value of the particles according to the new population particle fitness value;
Vi，t+1＝w×Vi，t+c1×rand×(pbesti-Xi，t)+c2×rand×(gbesti-Xi，t) (7)
Xi，t+1＝Xi，t+λ×Vi，t+1 (8)
wherein, Xi，tAnd Vi，tFor the position and velocity of the ith particle in the t-th iterationDegree; pbestiIs an individual extremum; gbestiIs a global optimal solution; w is the inertial weight; c. C1And c2Is a learning factor; rand is [0, 1 ]]A random number in between; λ is a speed coefficient, λ ═ 1;
and obtaining the super parameter with the minimum corresponding fitness value of the LSTM network after the maximum iteration times of the particle swarm optimization algorithm are met.
Wherein, the direct multi-step prediction in the extension prediction of the LSTM prediction model by using the direct multi-step prediction method returns the predicted values of a plurality of moments at one time
The learning and estimation processes are as follows:
yj(t+1)＝f1(x1，j(t+1)，x2，j(t+1)，…，xI，j(t+1))+；
wherein f is1(. to) a predictive model, referred to herein as LSTM; is a learning error; h is the number of epitaxial steps.
And averaging the operation results of the LSTM prediction model for more than or equal to 10 times to obtain a final output result.
The step of visually characterizing the LSTM network relevance learning by adopting the t-SNE technology comprises the following steps of:
extracting LSTM unit hidden layer vector under all time steps of second layer of network
And projecting the original high-dimensional space onto a two-dimensional plane by adopting a t-SNE visualization technology.
Based on the technical scheme, compared with the prior art, the hydraulic building multi-effect optimization prediction and visualization method based on the LSTM network has at least one or part of the following beneficial effects:
aiming at complex nonlinear safety monitoring big data, the invention provides a multi-effect optimization prediction and visualization method suitable for different types of hydraulic buildings by improving and strengthening an LSTM deep network from four aspects of front-end processing, network structure, extension prediction and visualization. The invention provides a new idea for the safety monitoring method of the hydraulic structure and lays a research foundation for deep learning, popularization and application.
Drawings
FIG. 1 is a schematic diagram of an internal structure of an LSTM unit provided in an embodiment of the present invention;
FIG. 2 is a diagram of data for monitoring a prototype of a concrete dam IP4_01_ X deformed according to an embodiment of the present invention;
FIG. 3 is a diagram of the prediction result of the deformation of the IP4_01_ X measurement point according to the embodiment of the present invention;
FIG. 4 is a visual representation of an LSTM hidden layer provided by an embodiment of the present invention;
FIG. 5 is a thermodynamic diagram of correlation coefficients of an LSTM hidden layer provided by an embodiment of the present invention;
fig. 6 is a flow chart of a hydraulic structure multi-effect optimization prediction and visualization method based on a deep LSTM network according to an embodiment of the present invention.
Detailed Description
The invention discloses a hydraulic building multi-effect quantity optimization prediction and visualization method based on a deep LSTM network.
In order that the objects, technical solutions and advantages of the present invention will become more apparent, the present invention will be further described in detail with reference to the accompanying drawings in conjunction with the following specific embodiments.
As shown in fig. 6, the invention discloses a hydraulic structure multi-effect optimization prediction and visualization method based on a deep LSTM network, which can predict monitoring effects of different types of hydraulic structures, and specifically comprises the following steps:
and step A, carrying out data cleaning, noise reduction smoothing, data transformation and data homogenization on the prototype observation data, and dividing the preprocessed data set into a training set, a verification set and a test set. The method specifically comprises the following steps:
step A1: and (6) data cleaning. Respectively carrying out hot card filling method and box type graph analysis method on the monitoring data yjLocal deletion value in (t)
Abnormal value
Processing in real time to obtain a corrected sequence
Step A2: and noise reduction and smoothing. The correction sequence obtained in the step A1 is processed by a wavelet transform method
Denoising to obtain a denoising smooth sequence
Step A3: and (5) data transformation. Smoothing sequence for noise reduction
Normalization processing is carried out, so that the speed of seeking an optimal solution can be increased, and the problem of numerical value brought to gradient updating can be avoided;
wherein the content of the first and second substances,
are respectively a smoothing sequence
Minimum and maximum values of;
step A4: and (6) homogenizing the data. Data homogenization is completed by using a segmented cubic Hermite interpolation function to obtain a final data sequence
All data are divided into training sets T according to a certain proportion after being preprocessedrVerification set VaAnd test set Te。
And step B, constructing a depth LSTM prediction model. The model is composed of an input layer, a plurality of hidden layers and an output layer, wherein the output of the previous hidden layer is used as the input of the next hidden layer, the input layer and the hidden layers jointly realize the extraction of input effect quantity data characteristics, the output of the last hidden layer is a one-dimensional column vector, and the predicted value of the processed effect quantity data is obtained through linear regression.
Memory module of deep LSTM network is composed of forgetting gate (f)t l) Input gate
And output gate
And (4) forming. Forgetting to control internal state at last moment
The input door controls the candidate state of the current time
The amount of information to be stored, and the output gate controls the internal state at the current time
Need to be output to external state
The amount of information of (2). Corresponding to the calculation process asThe following:
(1) using external state at last moment
And current time input
Calculate ft l、
And
as shown in formulas (2) - (4);
(2) bonding of
And
updating memory cell states
As shown in formula (5);
(3) by passing
Will be provided with
Information delivery to
As shown in formulas (6) - (7);
wherein, σ (·) and tanh (·) are respectively a Sigmoid function and a hyperbolic tangent function; w and b are respectively a weight matrix and an offset vector; an indicator indicates a scalar product of two vectors. The internal structure of the LSTM cell is shown in fig. 1. And updating internal parameters of the LSTM network by adopting an adaptive moment estimation algorithm.
And step C, performing hyper-parameter debugging and optimizing an LSTM algorithm. The method specifically comprises the following steps:
step C1: constructing a deep LSTM network, overlapping and combining a plurality of LSTM networks by adopting a stacking integration method to obtain an integrated LSTM network, carrying out unary time sequence prediction, and directly carrying out effect quantity y according to historical monitoring dataj(t) carrying out epitaxial analysis on the future variation trend;
step C2: and carrying out hyper-parameter debugging. For LSTM networks, the time window length w is optimized by Particle Swarm Optimization (PSO)tNumber n of hidden layershThe number of nodes n in each hidden layernLearning rate lrAnd optimizing the number of iterations niCarrying out iterative optimization on equal hyper-parameters, and selecting a hyper-parameter combination with the minimum corresponding fitness value
Step C2-1: and initializing parameters. Determining a population scale, iteration times, learning factors and limited intervals of position and speed values;
step C2-2: the position and velocity of the particles are initialized. Randomly generating a population of particles Xi，0(wt，n1，n2，lr，ni). Wherein n is1Number of neurons representing the first hidden layer, n2Representing the number of neurons in the second hidden layer;
step C2-3: an evaluation function of the particles is determined. The particles X obtained in the step C2-2i，0And assigning values to the parameters of the LSTM. The data is divided into training samples, validation samples and prediction samples. Inputting training samples to train the neural network, and obtaining the training sample output value of the neural network after the iteration times are limited
And verifying the sample output value
Then the individual XiFitness value fit ofiIs defined as:
wherein the content of the first and second substances,
and
respectively obtaining an expected output value of a training sample and an expected output value of a verification sample;
step C2-4: calculating the position X of each particleiDetermining individual extremum and group extremum according to the initial particle fitness value and taking the best position of each particle as the historical best position; the terms "best" and "best" mean "corresponding to the particle with the smallest fitness value within the number of iterations that have been performed".
Step C2-5: in each iteration process, updating the speed and position of the particle per se through the individual extremum and the global extremum according to the equations (9) and (10); calculating a new particle fitness value, and updating the individual extreme value and the population extreme value of the particles according to the new population particle fitness value;
Vi，t+1＝w×Vi，t+c1×rand×(pbesti-Xi，t)+c2×rand×(gbesti-Xi，t) (9)
Xi，t+1＝Xi，t+λ×Vi，t+1 (10)
wherein, Xi，tAnd Vi，tIs the position and velocity of the ith particle in the t iteration; pbestiIs an individual extremum; gbestiIs a global optimal solution; w is the inertial weight; c. C1And c2Is a learning factor; rand is [0, 1 ]]A random number in between; λ is a speed coefficient, λ ═ 1;
step C2-6: and obtaining the optimal hyper-parameter of the LSTM network after the maximum iteration times of the PSO algorithm is met.
And D, performing epitaxial prediction by using a direct multi-step prediction method.
Direct multi-step prediction returns predicted values at multiple moments at a time
The learning and estimation processes are as follows:
yj(t+1)＝f1(x1，j(t+1)，x2，j(t+1)，…，xI，j(t+1))+ (11)
wherein f is1(. to) a predictive model, referred to herein as LSTM; is a learning error; h is the number of epitaxial steps.
Since there is randomness in optimizing the LSTM prediction, the results of 10 runs of each model are averaged as the final output.
And E, performing visual representation on deep LSTM network correlation learning by adopting a t-SNE technology.
Extracting LSTM unit hidden layer vector under all time steps of second layer of network
And projecting the original high-dimensional space onto a two-dimensional plane by adopting a t-SNE visualization technology.
The feature extraction capability of the deep LSTM network is visually represented through visualization, namely the time correlation existing in the sequence can be actively extracted, and correlation measurement indexes are defined as follows:
wherein the content of the first and second substances,<·>representing the vector inner product;
a hidden layer vector corresponding to the ith time step;
the hidden layer vector corresponding to the jth time step.
The invention provides the multi-effect optimization prediction and visualization method suitable for different types of hydraulic buildings, which not only can more accurately predict the safety of the hydraulic buildings, but also lays a research foundation for deep learning, popularization and application.
The following is further described as an embodiment by a specific application scenario.
And selecting the effect measured data as a typical application scene, and verifying the effectiveness and the accuracy of the optimized LSTM model, aiming at explaining the advantages of the optimized LSTM model in the aspect of safety monitoring of various hydraulic buildings through the example. Therefore, taking the deformation monitoring data (data are shown in fig. 2, the vertical line is the right prediction period) of a certain concrete dam in the IP4_01_ X direction as an example, an actual application scene is set, and the optimized LSTM depth model is used for carrying out extrapolation prediction on the data.
As can be seen from fig. 2, the deformation of the measuring point shows obvious periodic variation, which is the most common and simplest data evolution form in engineering; the monitoring data has large sample amount and basically consistent numerical value change amplitude; there are several abnormal mutations distributed as discrete (box selection in fig. 2). The direct multi-step prediction results of the optimized LSTM model are shown in fig. 3. As can be seen from the figure: the prediction result of the optimized LSTM model is closer to the actually measured data, the approximate change trend is the same, and the potential of the depth model in the aspect of large-amount data mining analysis is shown. Fig. 4 and 5 visually represent the feature extraction capability of the deep LSTM network through visualization, i.e., it can actively extract the temporal correlation existing in the sequence. Each point in fig. 4 represents a time step, and as the time step progresses, the path in the two-dimensional plane takes a periodic loop shape. The darker the color in fig. 5 indicates greater correlation, the more regular the color shade changes, indicating that depth LSTM can effectively extract structural features in the time series, and the longer-spanning time correlations can be stored by the network due to the unique memory structure.
The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A hydraulic structure multi-effect optimization prediction and visualization method based on an LSTM network is characterized by comprising the following steps:
obtaining prototype observation data;
preprocessing the prototype observation data, and dividing a preprocessed data set into a training set, a verification set and a test set;
constructing an LSTM prediction model, namely an LSTM network, by utilizing the training set, the verification set and the test set;
carrying out hyper-parameter debugging on the LSTM network so as to optimize an LSTM algorithm;
carrying out extension prediction on the LSTM prediction model by utilizing a direct multi-step prediction method; and
and (3) performing visual representation on the LSTM network correlation learning by adopting a t-SNE technology, namely completing the multi-effect optimization prediction and visualization of the hydraulic structure based on the LSTM network.
2. The method of claim 1, wherein the preprocessing of the prototype observations comprises data cleaning, noise reduction smoothing, data transformation, and data homogenization.
3. The method for optimizing, predicting and visualizing multiple effects of a hydraulic structure according to claim 1, wherein the LSTM prediction model in the step of constructing the LSTM prediction model using the training set, the validation set, and the test set is composed of an input layer, a plurality of hidden layers, and an output layer, an output of a previous hidden layer is used as an input of a next hidden layer, the input layer and the hidden layers together achieve extraction of input effect data features, an output of a last hidden layer is a one-dimensional column vector, and a predicted value of processed effect data is obtained through linear regression.
4. The hydraulic structure multi-effect optimization prediction and visualization method according to claim 1, wherein the memory module of the LSTM network is a forgetting gate
Input gate
And output gate
Forming; forgetting to control internal state at last moment
Need to forgetAmount of information, input gate control of the current time candidate status
The amount of information to be stored, and the output gate controls the internal state at the current time
Need to be output to external state
The amount of information of (a); the corresponding calculation process is as follows:
(1) using external state at last moment
And current time input
Calculate out
And
as shown in formulas (1) - (3);
(2) bonding of
And
updating memory cell states
As shown in formula (4);
(3) by passing
Will be provided with
Information delivery to
As shown in formulas (5) - (6);
wherein, σ (·) and tanh (·) are respectively a Sigmoid function and a hyperbolic tangent function; w and b are respectively a weight matrix and an offset vector; an indicator indicates a scalar product of two vectors.
5. The method for optimizing, predicting and visualizing multiple effects of a hydraulic structure according to claim 1, wherein the step of performing hyper-parametric debugging on the LSTM network so as to optimize an LSTM algorithm specifically comprises:
constructing a deep LSTM network, overlapping and combining a plurality of LSTM networks by adopting a stacking integration method to obtain an integrated LSTM network, carrying out unitary time sequence prediction, and carrying out epitaxial analysis on future change trend of effect quantity directly according to historical monitoring data; and
and carrying out hyper-parameter debugging.
6. The method for predicting and visualizing multiple effects of hydraulic buildings according to claim 5, wherein the performing of the hyper-parameter debugging is realized by a particle swarm optimization algorithm, and the time window length w istNumber n of hidden layershThe number of nodes n in each hidden layernLearning rate lrAnd optimizing the number of iterations niCarrying out iterative optimization on equal hyper-parameters, and selecting a hyper-parameter combination with the minimum corresponding fitness value
7. The method for predicting and visualizing multiple effects in hydraulic structures according to claim 5, wherein said performing hyper-parametric debugging further comprises:
initializing parameters; determining a population scale, iteration times, learning factors and limited intervals of position and speed values;
initializing the position and speed of the particles; randomly generating a population of particles Xi，0(wt，n1，n2，lr，ni) (ii) a Wherein n is1Number of neurons representing the first hidden layer, n2Representing the number of neurons in the second hidden layer;
determining an evaluation function of the particle; the particles X obtained in the last step are treatedi，0Assigning values to parameters of the LSTM network; dividing data into training samples, verifying samples and predicting samples; inputting the training sample to train the neural network to reach the limit of iteration timesObtaining the output value of the training sample of the neural network
And verifying the sample output value
Then the individual XiFitness value fit ofiIs defined as:
wherein the content of the first and second substances,
and
respectively obtaining an expected output value of a training sample and an expected output value of a verification sample;
calculating the position X of each particleiDetermining individual extremum and group extremum according to the initial particle fitness value and taking the best position of each particle as the historical best position;
during each iteration, updating the speed and position of the particle itself through the individual extremum and the global extremum according to equations (7) and (8); calculating a new particle fitness value, and updating the individual extreme value and the population extreme value of the particles according to the new population particle fitness value;
Vi，t+1＝w×Vi，t+c1×rand×(pbesti-Xi，t)+c2×rand×(gbesti-Xi，t) (7)
Xi，t+1＝Xi，t+λ×Vi，t+1 (8)
wherein, Xi，tAnd Vi，tIs the position and velocity of the ith particle in the t iteration; pbestiIs an individual extremum; gbestiIs a global optimal solution; w is the inertial weight; c. C1And c2Is a learning factor; rand is [0, 1 ]]A random number in between; λ is a speed coefficient, λ ═ 1;
and obtaining the super parameter with the minimum corresponding fitness value of the LSTM network after the maximum iteration times of the particle swarm optimization algorithm are met.
8. The method for optimizing, predicting and visualizing multiple effects of hydraulic structures according to claim 1, wherein the direct multi-step prediction in the extensive prediction of the LSTM prediction model by using the direct multi-step prediction method returns predicted values at multiple moments at a time
The learning and estimation processes are as follows:
yj(t+1)＝f1(x1，j(t+1)，x2，j(t+1)，…，xI，j(t+1))+；
wherein f is1(. to) a predictive model, referred to herein as LSTM; is a learning error; h is the number of epitaxial steps.
9. The method for optimizing, predicting and visualizing the multiple effects of a hydraulic structure as claimed in claim 1, wherein the final output result is obtained by averaging the operation results of the LSTM prediction model for 10 times or more.
10. The method for predicting and visualizing the multiple effects of the hydraulic structure as claimed in claim 1, wherein said step of visually characterizing LSTM network dependency learning using t-SNE technique comprises:
extracting LSTM unit hidden layer vector under all time steps of second layer of network
And projecting the original high-dimensional space onto a two-dimensional plane by adopting a t-SNE visualization technology.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011020208.1A CN112131794A (en) | 2020-09-25 | 2020-09-25 | Hydraulic structure multi-effect optimization prediction and visualization method based on LSTM network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011020208.1A CN112131794A (en) | 2020-09-25 | 2020-09-25 | Hydraulic structure multi-effect optimization prediction and visualization method based on LSTM network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112131794A true CN112131794A (en) | 2020-12-25 |
Family
ID=73839835
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011020208.1A Pending CN112131794A (en) | 2020-09-25 | 2020-09-25 | Hydraulic structure multi-effect optimization prediction and visualization method based on LSTM network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112131794A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112633413A (en) * | 2021-01-06 | 2021-04-09 | 福建工程学院 | Underwater target identification method based on improved PSO-TSNE feature selection |
| CN113033879A (en) * | 2021-03-05 | 2021-06-25 | 西北大学 | Landslide displacement prediction method based on intuitive fuzzy memetic PSO-LSTM |
| CN114240913A (en) * | 2021-12-21 | 2022-03-25 | 歌尔股份有限公司 | Semiconductor abnormality analysis method, semiconductor abnormality analysis device, terminal device, and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170091615A1 (en) * | 2015-09-28 | 2017-03-30 | Siemens Aktiengesellschaft | System and method for predicting power plant operational parameters utilizing artificial neural network deep learning methodologies |
| US20180336674A1 (en) * | 2017-05-22 | 2018-11-22 | General Electric Company | Image analysis neural network systems |
| CN108986470A (en) * | 2018-08-20 | 2018-12-11 | 华南理工大学 | The Travel Time Estimation Method of particle swarm algorithm optimization LSTM neural network |
| CN109034264A (en) * | 2018-08-15 | 2018-12-18 | 云南大学 | Traffic accident seriousness predicts CSP-CNN model and its modeling method |
| CN109829587A (en) * | 2019-02-12 | 2019-05-31 | 国网山东省电力公司电力科学研究院 | Zonule grade ultra-short term and method for visualizing based on depth LSTM network |
| CN110381515A (en) * | 2019-08-12 | 2019-10-25 | 北京互联无界科技有限公司 | Based on the method for closing merotype realization subzone network floating resources index prediction |
| CN110390436A (en) * | 2019-07-25 | 2019-10-29 | 上海电力大学 | A kind of power plant's coal load quantity short term prediction method based on SSA Yu LSTM deep learning |
| CN111063194A (en) * | 2020-01-13 | 2020-04-24 | 兰州理工大学 | Traffic flow prediction method |
| CN111582900A (en) * | 2019-02-19 | 2020-08-25 | 腾讯科技（深圳）有限公司 | Media file delivery method and device, storage medium and electronic device |
-
2020
- 2020-09-25 CN CN202011020208.1A patent/CN112131794A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170091615A1 (en) * | 2015-09-28 | 2017-03-30 | Siemens Aktiengesellschaft | System and method for predicting power plant operational parameters utilizing artificial neural network deep learning methodologies |
| US20180336674A1 (en) * | 2017-05-22 | 2018-11-22 | General Electric Company | Image analysis neural network systems |
| CN109034264A (en) * | 2018-08-15 | 2018-12-18 | 云南大学 | Traffic accident seriousness predicts CSP-CNN model and its modeling method |
| CN108986470A (en) * | 2018-08-20 | 2018-12-11 | 华南理工大学 | The Travel Time Estimation Method of particle swarm algorithm optimization LSTM neural network |
| CN109829587A (en) * | 2019-02-12 | 2019-05-31 | 国网山东省电力公司电力科学研究院 | Zonule grade ultra-short term and method for visualizing based on depth LSTM network |
| CN111582900A (en) * | 2019-02-19 | 2020-08-25 | 腾讯科技（深圳）有限公司 | Media file delivery method and device, storage medium and electronic device |
| CN110390436A (en) * | 2019-07-25 | 2019-10-29 | 上海电力大学 | A kind of power plant's coal load quantity short term prediction method based on SSA Yu LSTM deep learning |
| CN110381515A (en) * | 2019-08-12 | 2019-10-25 | 北京互联无界科技有限公司 | Based on the method for closing merotype realization subzone network floating resources index prediction |
| CN111063194A (en) * | 2020-01-13 | 2020-04-24 | 兰州理工大学 | Traffic flow prediction method |
Non-Patent Citations (3)
| Title |
|---|
| 张宇帆等: ""基于深度长短时记忆网络的区域级超短期负荷预测方法"", 《电网技术》, vol. 43, no. 6, pages 1884 - 1891 * |
| 李明超;任秋兵;沈扬;: "贝叶斯框架下的大坝变形交互式时变预测模型及其验证", 水利学报, no. 11 * |
| 杜心: ""基于LSTM神经网络的可用停车位预测模型研究"", 《CNKI硕士电子期刊》 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112633413A (en) * | 2021-01-06 | 2021-04-09 | 福建工程学院 | Underwater target identification method based on improved PSO-TSNE feature selection |
| CN112633413B (en) * | 2021-01-06 | 2023-09-05 | 福建工程学院 | Underwater target identification method based on improved PSO-TSNE feature selection |
| CN113033879A (en) * | 2021-03-05 | 2021-06-25 | 西北大学 | Landslide displacement prediction method based on intuitive fuzzy memetic PSO-LSTM |
| CN113033879B (en) * | 2021-03-05 | 2023-07-07 | 西北大学 | Landslide displacement prediction method based on intuitive fuzzy denominator PSO-LSTM |
| CN114240913A (en) * | 2021-12-21 | 2022-03-25 | 歌尔股份有限公司 | Semiconductor abnormality analysis method, semiconductor abnormality analysis device, terminal device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ren et al. | An optimized combination prediction model for concrete dam deformation considering quantitative evaluation and hysteresis correction | |
| Qin et al. | Data-driven learning of nonautonomous systems | |
| CN112131794A (en) | Hydraulic structure multi-effect optimization prediction and visualization method based on LSTM network | |
| Wang et al. | Deep Boltzmann machine based condition prediction for smart manufacturing | |
| CN110427654B (en) | Landslide prediction model construction method and system based on sensitive state | |
| CN112949828B (en) | Graph convolution neural network traffic prediction method and system based on graph learning | |
| CN115688579B (en) | Drainage basin multipoint water level prediction and early warning method based on generation countermeasure network | |
| CN109635245A (en) | A kind of robust width learning system | |
| Zhang et al. | MeshingNet3D: Efficient generation of adapted tetrahedral meshes for computational mechanics | |
| Liu | Development of gradient-enhanced kriging approximations for multidisciplinary design optimization | |
| CN115935834A (en) | History fitting method based on deep autoregressive network and continuous learning strategy | |
| CN116187835A (en) | Data-driven-based method and system for estimating theoretical line loss interval of transformer area | |
| CN115982141A (en) | Characteristic optimization method for time series data prediction | |
| Khorram et al. | A hybrid CNN-LSTM approach for monthly reservoir inflow forecasting | |
| CN111507505A (en) | Method for constructing reservoir daily input prediction model | |
| Robati et al. | Inflation rate modeling: Adaptive neuro-fuzzy inference system approach and particle swarm optimization algorithm (ANFIS-PSO) | |
| Dong et al. | Point and interval prediction of the effective length of hot-rolled plates based on IBES-XGBoost | |
| Chen et al. | Efficient approximate dynamic programming based on design and analysis of computer experiments for infinite-horizon optimization | |
| Li et al. | A new multi-fidelity surrogate modelling method for engineering design based on neural network and transfer learning | |
| Manoj et al. | FWS-DL: forecasting wind speed based on deep learning algorithms | |
| Akoz et al. | Prediction of geometrical properties of perfect breaking waves on composite breakwaters | |
| Singaravel et al. | Explainable deep convolutional learning for intuitive model development by non–machine learning domain experts | |
| CN115204463A (en) | Residual service life uncertainty prediction method based on multi-attention machine mechanism | |
| Liu et al. | Prediction of dam horizontal displacement based on CNN-LSTM and attention mechanism | |
| Hinds et al. | Neural variance reduction for stochastic differential equations |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |