CN112561203B - Method and system for realizing water level early warning based on clustering and GRU - Google Patents
Method and system for realizing water level early warning based on clustering and GRU Download PDFInfo
- Publication number
- CN112561203B CN112561203B CN202011543873.9A CN202011543873A CN112561203B CN 112561203 B CN112561203 B CN 112561203B CN 202011543873 A CN202011543873 A CN 202011543873A CN 112561203 B CN112561203 B CN 112561203B
- Authority
- CN
- China
- Prior art keywords
- water level
- data
- points
- clustering
- neighborhood
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 177
- 238000000034 method Methods 0.000 title claims abstract description 29
- 230000002159 abnormal effect Effects 0.000 claims abstract description 55
- 238000012549 training Methods 0.000 claims abstract description 33
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 16
- 238000012360 testing method Methods 0.000 claims abstract description 10
- 230000008859 change Effects 0.000 claims description 27
- 238000012545 processing Methods 0.000 claims description 19
- 238000007689 inspection Methods 0.000 claims description 9
- 238000010606 normalization Methods 0.000 claims description 6
- 238000012546 transfer Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 claims description 5
- 238000005070 sampling Methods 0.000 claims description 5
- 238000000611 regression analysis Methods 0.000 claims description 2
- 241000288105 Grus Species 0.000 claims 1
- 230000004913 activation Effects 0.000 description 5
- 238000007621 cluster analysis Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 5
- 238000012795 verification Methods 0.000 description 5
- 230000002265 prevention Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000002547 anomalous effect Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002618 waking effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/04—Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
- Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping
Landscapes
- Business, Economics & Management (AREA)
- Engineering & Computer Science (AREA)
- Economics (AREA)
- Human Resources & Organizations (AREA)
- Strategic Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Marketing (AREA)
- General Physics & Mathematics (AREA)
- General Business, Economics & Management (AREA)
- Tourism & Hospitality (AREA)
- Quality & Reliability (AREA)
- Game Theory and Decision Science (AREA)
- Operations Research (AREA)
- Development Economics (AREA)
- Entrepreneurship & Innovation (AREA)
- Public Health (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Alarm Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a method for realizing water level early warning based on clustering and GRU (generalized regression), which comprises the following steps of: s11, collecting water level information, and calculating a water level variation data set according to the collected water level information; s12, clustering the water level variation data set based on a DBSCAN clustering algorithm to obtain normal data and abnormal data corresponding to the water level variation data set; s13, inputting the obtained normal data and abnormal data into a GRU network for training to obtain weight parameters for predicting abnormal points, and constructing a water level prediction model according to the obtained weight parameters; and S14, inputting the test data into the constructed water level prediction model, and outputting water level information by the water level prediction model. The invention can accurately predict the height of the water outlet level, achieve early warning, and take corresponding measures in advance to ensure that the water level of the water area is in a safe state, thereby ensuring the life safety and property safety of people.
Description
Technical Field
The invention relates to the technical field of water level prediction, in particular to a method and a system for realizing water level early warning based on clustering and GRU.
Background
With the rapid development of economy in China, the continuous progress of science and technology, particularly the rise of the Internet, a lot of work becomes simple, convenient and informationized. In the years, the flood season of China is more and more long, the flood prevention work of China is daily severe when the flood prevention work of China enters a rainy season, and therefore how to improve the information of the flood prevention work of China and quickly and accurately give an early warning becomes the most critical and urgent current flood prevention situation, and the prediction and early warning should be timely realized through an effective analysis method.
There are many methods for predicting water levels, but they all have some drawbacks. For example, the core task of traditional water level prediction is to construct a water level prediction model, depict and describe a water level process through different theoretical methods, hierarchical structures and modeling modes, express the water level process in a form of mathematical language or physical models, and finally apply the water level prediction model to prediction calculation and water level simulation analysis. For example, the models of the new anjiang river have a certain application range and can be mastered only by some professionals, so that the adaptability is not strong. Therefore, aiming at the technical problems, the invention provides a method and a system for realizing water level early warning based on clustering and GRU (generalized regression analysis) to solve the existing technical problems.
Disclosure of Invention
The invention aims to provide a method and a system for realizing water level early warning based on clustering and GRU (generalized regression unit) aiming at the defects of the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for realizing water level early warning based on clustering and GRU comprises the following steps:
s1, collecting water level information, and calculating a water level change data set according to the collected water level information;
s2, clustering the water level variation data set based on a DBSCAN clustering algorithm to obtain normal data and abnormal data corresponding to the water level variation data set;
s3, inputting the obtained normal data and abnormal data into a GRU network for training to obtain weight parameters for predicting abnormal points, and constructing a water level prediction model according to the obtained weight parameters;
and S4, inputting the test data into the constructed water level prediction model, and outputting water level information by the water level prediction model.
Further, after the step S1 of calculating the water level variation information set according to the collected water level information, the method further includes: for each piece of water level variation data X _0 ═ X 0 ，X 1 ，X 2 ...X t ...X n ]Carrying out maximum and minimum normalization processing to obtain a normalized data set X ═ X 0 ，x 1 ，x 2 ...x t ...x n ]。
Further, the step S2 specifically includes:
s21, selecting an unprocessed object x in the data set i Inspection object x i Is x, if i If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum contained point number minPts, a cluster K is established, and x is divided into i Marking as processed, marking x i Adding all the points in the corresponding neighborhood into the candidate set C, and executing the step S22; if x i If the number of the corresponding middle points in the neighborhood is less than the minimum number of points minPts, then mark x i Is a noise point;
s22, all objects x in the candidate set C which are not processed yet are subjected to processing j Inspection object x j If x j If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum number of contained points minPts, x is added j All points in the corresponding neighborhood are added to candidate set C and x is added j Marking as processed if x j If the cluster is not classified into any cluster, adding the object into the cluster K, and executing the step S23; if x j Corresponding adjacentIf the number of points in the field is less than the minimum number of points minPts, then mark x j Is a noise point;
s23, repeatedly executing the step S22, and continuously checking the objects which are not marked in the candidate set C until the objects in the candidate set C are processed;
s24, the steps S21-S23 are repeatedly executed until all the objects are classified into a certain cluster or marked as noise.
Further, the step S3 of inputting the obtained normal data and abnormal data into the GRU network for training is to input the normal data and abnormal data into the GRU network, and perform processing by forward and backward iterative transfer.
Further, the time interval of sampling in the water level information collected in step S1 is 5 minutes; the data set for calculating the water level change amount is used for calculating the water level change amount from the first 12 hours of the current time point to every 5 minutes of the current time.
Correspondingly, still provide a system based on clustering and GRU realize water level early warning, include:
the acquisition module is used for acquiring water level information and calculating a water level change data set according to the acquired water level information;
the clustering module is used for clustering the water level variation data set based on a DBSCAN clustering algorithm to obtain normal data and abnormal data corresponding to the water level variation data set;
the training module is used for inputting the obtained normal data and abnormal data into a GRU network for training to obtain weight parameters for predicting abnormal points, and a water level prediction model is constructed according to the obtained weight parameters;
and the output module is used for inputting the test data into the constructed water level prediction model, and the water level prediction model outputs water level information.
Further, after the acquiring module calculates the water level variation information set according to the acquired water level information, the method further includes: for each piece of water level variation data X _0 ═ X 0 ，X 1 ，X 2 ...X t ...X n ]Carrying out maximum and minimum normalization processing to obtain a normalized data set X ═ X 0 ，x 1 ，x 2 ...x t ...x n ]。
Further, the clustering module specifically includes:
a first processing module for selecting an unprocessed object x in the data set i Inspection object x i Is x, if i If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum contained point number minPts, a cluster K is established, and x is used i Marking as processed, marking x i Adding all the points in the corresponding neighborhood into a candidate set C; if x i If the number of the corresponding middle points in the neighborhood is less than the minimum number of points minPts, then mark x i Is a noise point;
a second processing module for processing all the objects x in the candidate set C which have not been processed j Inspection object x j In a field of (x) j If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum contained point number minPts, x is added j All points in the corresponding neighborhood are added to candidate set C and x is added j Marked as processed if x j If the object is not classified into any cluster, adding the object into the cluster K; if x j If the number of the corresponding middle points in the neighborhood is less than the minimum number of points minPts, then mark x j Is a noise point;
a first-loop module for continuing to check the objects in the candidate set C that are not marked until all objects in the candidate set C have been processed;
a second loop module for continuing to examine the objects until all objects fall into a certain cluster or are marked as noise.
Further, the training module inputs the obtained normal data and abnormal data into the GRU network for training, that is, the normal data and abnormal data are input into the GRU network and processed by adopting forward and reverse iterative transfer.
Furthermore, the sampling time interval in the water level information collected in the collection module is 5 minutes; the data set for calculating the water level change amount is used for calculating the water level change amount from the first 12 hours of the current time point to every 5 minutes of the current time.
Compared with the prior art, the invention utilizes the DBSCAN algorithm to perform cluster analysis on the water level change in each time period, accurately detects the abnormal change of the water level, uses the GRU network to simulate the water level change amount condition from the first 12 hours of the current time point to every 5 minutes of the current time when the abnormal water level change occurs, further accurately predicts the water level height, achieves early warning, and takes corresponding measures in advance to enable the water level of the water area to be in a safe state, thereby ensuring the life safety and property safety of people.
Drawings
Fig. 1 is a flowchart of a method for implementing water level early warning based on clustering and GRU according to an embodiment;
fig. 2 is a diagram of a GRU network structure according to an embodiment;
FIG. 3 is a schematic diagram of prediction of monitored data according to an embodiment;
FIG. 4 is a diagram illustrating the occurrence of an exception in providing time series data according to the second embodiment;
fig. 5 is a system structure diagram for implementing water level pre-warning based on clustering and GRU according to the third embodiment.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
The invention aims to provide a method and a system for realizing water level early warning based on clustering and GRU (generalized regression unit) aiming at the defects of the prior art.
Example one
The embodiment provides a method for realizing water level early warning based on clustering and GRU, as shown in fig. 1, including the steps of:
s11, collecting water level information, and calculating a water level variation data set according to the collected water level information;
s12, clustering the water level variation data set based on a DBSCAN clustering algorithm to obtain normal data and abnormal data corresponding to the water level variation data set;
s13, inputting the obtained normal data and abnormal data into a GRU network for training to obtain weight parameters for predicting abnormal points, and constructing a water level prediction model according to the obtained weight parameters;
and S14, inputting the test data into the constructed water level prediction model, and outputting water level information by the water level prediction model.
The embodiment provides a method for predicting and early warning water level by using a DBSCAN algorithm to perform cluster analysis to obtain normal data and abnormal data, inputting the normal data and the abnormal data obtained by the cluster analysis as training data into a GRU network for training, and realizing water level prediction and early warning through the trained network.
The embodiment comprises a training module and a verification module: the training module collects water level information at a time interval of 5 minutes and calculates water level variation, clustering is carried out by using DBSCAN, the water level variation information of every 5 minutes is divided into normal labels and abnormal labels, then the normal labels and the abnormal labels are input into a GRU network for training, and in the process of multiple forward and reverse iterations, data of the predicted labels and data of actual label types are compared, so that proper weight is obtained by prediction under the condition of error allowance, and model training is completed; the verification module verifies the test data set model, inputs the processed test data into the trained network model, and predicts the water level information of each 5-minute time interval by the model.
It should be noted that, in the present embodiment, the training module is steps S11-S13, and the verification module is step S14.
In step S11, water level information is collected, and a water level variation data set is calculated based on the collected water level information.
Firstly, acquiring water level information once every 5 minutes, and taking the acquired water level information as a modeling data sample; then, the water level variation from the first 12 hours of the current time point to every 5 minutes of the current time point is calculated according to the collected water level information, and each calculated water level variation is calculatedData X _0 ═ X 0 ，X 1 ，X 2 ...X t ...X n ]Maximum and minimum normalization processing is carried out, such as:
obtaining a normalized data set X ═ X 0 ，x 1 ，x 2 ...x t ...x n ]。
In step S12, the water level variation data set is clustered based on the DBSCAN clustering algorithm, and normal data and abnormal data corresponding to the water level variation data set are obtained.
The DBSCAN algorithm selects a core object without category as a seed, and finds a sample set with all the core objects with reachable density, i.e. a cluster. And then continuously selecting another core object without categories to search a sample set with reachable density, thereby obtaining another cluster. The method runs until all core objects have the types, and data outside the clusters are abnormal points, so that abnormal data can be effectively processed.
Step S12 specifically includes:
assuming a radius of eps and a density threshold of minPts.
S121, selecting an unprocessed object x in a data set i Inspection object x i Is x, if i If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum contained point number minPts, a cluster K is established, and x is used i Marking as processed, marking x i Adding all points in the corresponding neighborhood into the candidate set C, and executing the step S122; if x i If the number of the corresponding middle points in the neighborhood is less than the minimum number of points minPts, then mark x i Is a noise point;
counting the points contained in the circle with radius eps, if the number of points in a circle exceeds MinPts, then the center x of the circle is counted i And marking as a core point, otherwise, marking as a noise point. Core point x i All points in the eps neighborhood of (a) are x i Is direct to the direct density of (if x) j From x i Direct density, x k From x j Direct density, x n From x k Density through, then x n From x i The density can be reached.
Find all from x i The density of the accessible objects is established into a cluster K, x i Marked as processed, all points in the neighborhood are added to the candidate set C.
S122. for all the objects x in the candidate set C which are not processed yet j Inspection object x j In a field of (x) j If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum number of contained points minPts, x is added j All points in the corresponding neighborhood are added to candidate set C and x is added j Marking as processed if x j If the object is not classified into any cluster, adding the object into the cluster K, and executing the step S123; if x j If the number of the corresponding middle points in the neighborhood is less than the minimum number of points minPts, then mark x j Is a noise point;
s123, repeatedly executing the step S122, and continuously checking the unmarked objects in the candidate set C until the objects in the candidate set C are processed;
s124, repeatedly executing the steps S121-S123 until all the objects are classified into a certain cluster or marked as noise.
In the embodiment, the training samples of the GRU network are obtained by adopting the DBSCAN algorithm, so that the samples are divided into abnormal data and normal data, and the GRU water level prediction information is more accurate.
Before step S13, the method further includes: calculating the abnormal value score of the clustered time series: initializing the outlier score to 0, and making the time series X ═ X 0 ，x 1 ，x 2 ...x t ...x n ]Comparison of x i ，x i+1 Belonging cluster, if x i ，x i+1 If the clusters are different, shifting + | (x) i -x i+1 )/min(x i -x i+1 ) If x i ，x i+1 And if the adjacent numbers belong to the same cluster, shifting to shifting, and thus recursion is carried out until all adjacent numbers are compared. The purpose of this step is to detect anomalous data.
In step S13, the obtained normal data and abnormal data are input into the GRU network for training, so as to obtain weight parameters for predicting abnormal points, and a water level prediction model is constructed according to the obtained weight parameters.
As shown in fig. 2, a structure diagram of a GRU network, the GRU algorithm belongs to one of recurrent neural networks, and a chain structure thereof is suitable for processing various timing problems. There are two gates in the GRU model: respectively an update gate and a reset gate. The specific structure is shown in the following figures: z in the figure t And r t Respectively representing an update gate and a reset gate. The update gate is used to control the extent to which the state information at the previous time is brought into the current state, and a larger value of the update gate indicates that more state information at the previous time is brought in. How much information is written to the current candidate set before reset gate controls the previous state
In this way, the smaller the reset gate, the less information of the previous state is written.
In this embodiment, forward and backward iterative transfer is adopted according to input data to obtain a weight parameter of the predicted outlier.
In this embodiment, the forward and backward iterations specifically include:
forward propagation: assuming that at a certain time step t in the training, the activation value h of the last time step is obtained t-1 Input feature vector x at t t Weight corresponding to input feature vector at t
Updating the weight parameter W of the door r Resetting the weight parameter W of the gate z And when t is the corresponding weight parameter W is output 0 Activation value h at t t . Respectively passing the activation value at t-1 and the input at t through a weight W r ，W z Connected to obtain reset gates r respectively through sigmoid function t ＝σ(W r ·[h t-1 ，x t ]) Indicating how much information was written to the current candidate set for the previous state; updating the door z t ＝σ(W z ·[h t-1 ，x t ]) Is shown to be used forControlling the degree to which the state information at the previous moment is brought into the current state; passing the activation value at t-1 and the input at t by weight
Connecting, and calculating with tanh to obtain candidate set represented as
The activation value at the time of t is finally obtained and is expressed as
The output is expressed as yt ═ σ (W) 0 ·h t ). Wherein
[]Indicates that the two vectors are connected and indicates the product of the matrices.
Iteration and back propagation: learning the network by adopting a backward error propagation algorithm to obtain a loss function of the sample, and assuming that the obtained predicted value is
The true value is yd and the loss at a certain moment of a single sample is
The loss of a single sample at all times is
After the partial derivatives for each parameter are calculated, the parameters can be updated, and the loss convergence is known by iteration in turn.
In step S14, test data is input into the constructed water level prediction model, which outputs water level information.
And calculating water level variation data of the data in the verification data set, processing in step S12, waking up the input data through a water level prediction model, and outputting water level information. And subsequently, early warning can be achieved according to the predicted water level information, and the water level in the water area is in a safe state by taking corresponding measures in advance, so that the life safety and property safety of people are guaranteed.
As shown in fig. 3, the overall process of predicting the monitored data includes a communication security gateway, a receiving node, a distributing node, and a receiving program, where the communication security gateway is connected to a cloud cluster and a physical machine cluster, respectively.
1. Collecting reported and monitored data such as water level, water level variable quantity and the like through a communication security gateway according to a set time frequency; data can be monitored and collected in real time;
2. the data receiving node receives the data, then decrypts and analyzes the data, predicts the water level by using a GRU network through a prediction engine, and performs early warning analysis, warning pushing and data storage;
3. the data distribution node can distribute conventional data and early warning data to a receiving program;
4. and the receiving program receives the conventional data and the alarm to display the page.
Compared with the prior art, the embodiment utilizes the DBSCAN algorithm to perform cluster analysis on the water level change in each time period, the abnormal change of the water level is accurately detected, when the abnormal water level change occurs, the GRU network is used for simulating the water level change amount condition from the first 12 hours of the current time point to every 5 minutes of the current time point, the water level height is accurately predicted, early warning is achieved, and countermeasures are taken in advance to enable the water level of the water area to be in a safe state, so that the life safety and property safety of people are guaranteed.
Example two
The difference between the method for realizing water level prediction based on the GRU network provided by the embodiment and the first embodiment is that:
this embodiment is described by way of specific examples.
The abnormal degree of the sequence increases with the increase of shifting, and the present embodiment performs an experiment using about 4000 pieces of data, the radius eps is 0.02, the density threshold minpts is 5, and after clustering, abnormal value scores are calculated, the data distribution is as shown in fig. 4, the shifting value in fig. 4 is large, the number of sequence abnormal values is large, and the piece of sequence can be considered as an abnormal sequence.
Taking abnormal data detected by clustering as a class sample, taking normal data as another class label sample, selecting data accounting for 80% of the total amount as a training set of a model, taking 20% of the data as a model verification set, constructing GRU network training through Tensorflow and verifying the accuracy of a model water level prediction value by using a least square method, wherein the data pattern and the loss and mean-square error (MSE) obtained by model training are shown in tables 1 and 2, wherein X in the table 1 represents the water level variation from the first 12 hours of the current time to every 5 minutes of the current time, and y represents whether the water level is abnormal or not; as can be seen from table 2, the losses and errors obtained from training are within acceptable ranges.
TABLE 1 data styles
TABLE 2 loss and mean square error from model training
EXAMPLE III
The embodiment provides a system for realizing water level early warning based on clustering and GRU, as shown in fig. 5, including:
the acquisition module 11 is used for acquiring water level information and calculating a water level change data set according to the acquired water level information;
the clustering module 12 is configured to cluster the water level variation data set based on a DBSCAN clustering algorithm to obtain normal data and abnormal data corresponding to the water level variation data set;
the training module 13 is configured to input the obtained normal data and abnormal data into the GRU network for training, to obtain a weight parameter for predicting an abnormal point, and to construct a water level prediction model according to the obtained weight parameter;
and the output module 14 is used for inputting the test data into the constructed water level prediction model, and the water level prediction model outputs water level information.
Further, after the acquiring module calculates the water level variation information set according to the acquired water level information, the method further includes: for each piece of water level variation data X _0 ═ X 0 ，X 1 ，X 2 ...X t ...X n ]Carrying out maximum and minimum normalization processing to obtain a normalized data set X ═ X 0 ，x 1 ，x 2 ...x t ...x n ]。
Further, the clustering module specifically includes:
a first processing module for selecting an unprocessed object x in the data set i Inspection object x i Is x, if i If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum contained point number minPts, a cluster K is established, and x is used i Marking as processed, marking x i Adding all points in the corresponding neighborhood into a candidate set C; if x i If the number of the corresponding middle points in the neighborhood is less than the minimum number of points minPts, then mark x i Is a noise point;
a second processing module for processing all the objects x in the candidate set C which have not been processed j Examination object x j If x j If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum contained point number minPts, x is added j All points in the corresponding neighborhood are added to candidate set C and x is added j Marked as processed if x j If the object is not classified into any cluster, adding the object into the cluster K; if x j If the number of the corresponding middle points in the neighborhood is less than the minimum number of points minPts, then mark x j Is a noise point;
a first-loop module for continuing to check the objects in the candidate set C that are not marked until all objects in the candidate set C have been processed;
a second loop module for continuing to examine the objects until all objects fall into a certain cluster or are marked as noise.
Further, the training module inputs the obtained normal data and abnormal data into the GRU network for training, that is, the normal data and abnormal data are input into the GRU network and processed by adopting forward and reverse iterative transfer.
Furthermore, the sampling time interval in the water level information collected in the collection module is 5 minutes; the data set for calculating the water level change amount is used for calculating the water level change amount from the first 12 hours of the current time point to every 5 minutes of the current time.
It should be noted that, the system for implementing water level early warning based on clustering and GRU provided in this embodiment is similar to the embodiment, and will not be described herein again.
Compared with the prior art, the embodiment utilizes the DBSCAN algorithm to perform cluster analysis on the water level change in each time period, the abnormal change of the water level is accurately detected, when the abnormal water level change occurs, the GRU network is used for simulating the water level change amount condition from the first 12 hours of the current time point to every 5 minutes of the current time point, the water level height is accurately predicted, early warning is achieved, and countermeasures are taken in advance to enable the water level of the water area to be in a safe state, so that the life safety and property safety of people are guaranteed.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in some detail by the above embodiments, the invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the invention, and the scope of the invention is determined by the scope of the appended claims.
Claims (6)
1. A method for realizing water level early warning based on clustering and GRU is characterized by comprising the following steps:
s1, collecting water level information, and calculating a water level change data set according to the collected water level information;
s2, clustering the water level variation data set based on a DBSCAN clustering algorithm to obtain normal data and abnormal data corresponding to the water level variation data set;
s3, inputting the obtained normal data and abnormal data into a GRU network for training to obtain weight parameters for predicting abnormal points, and constructing a water level prediction model according to the obtained weight parameters;
s4, inputting the test data into the constructed water level prediction model, and outputting water level information by the water level prediction model;
after calculating the water level variation information set according to the collected water level information in step S1, the method further includes: for each piece of water level variation data X _0 ═ X 0 ,X 1 ,X 2 …X t …X n ]Carrying out maximum and minimum normalization processing to obtain a normalized data set X ═ X 0 ,x 1 ,x 2 …x t …x n ]；
Step S2 specifically includes:
s21, selecting an unprocessed object x in the data set i Inspection object x i Neighborhood of (c), if x i If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum contained point number minPts, a cluster K is established, and x is divided into i Marking as processed, marking x i Adding all points in the corresponding neighborhood into the candidate set C, and executing the step S22; if x i If the number of the corresponding middle points in the neighborhood is less than the minimum number of points minPts, then mark x i Is a noise point;
s22, all objects x in the candidate set C which are not processed yet are processed j Examination object x j Is x, if j If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum number of contained points minPts, x is added j All points in the corresponding neighborhood are added to candidate set C and x is added j Marking as processed if x j If the object is not classified into any cluster, adding the object into the cluster K, and executing the step S23; if x j If the number of the corresponding middle points in the neighborhood is less than the minimum number of points minPts, then mark x j Is a noise point;
s23, repeatedly executing the step S22, and continuously checking the objects which are not marked in the candidate set C until the objects in the candidate set C are processed;
s24, the steps S21-S23 are repeatedly executed until all the objects are classified into a certain cluster or marked as noise.
2. The method for realizing water level early warning based on clustering and GRU (generalized regression analysis) of claim 1, wherein the step S3 of inputting the obtained normal data and abnormal data into the GRU network for training is to input the normal data and abnormal data into the GRU network and to process the normal data and abnormal data by forward and backward iterative transfer.
3. The method for realizing the water level early warning based on the clustering and the GRU as claimed in claim 1, wherein the time interval of sampling in the step S1 for collecting the water level information is 5 minutes; the data set for calculating the water level change amount is used for calculating the water level change amount from the first 12 hours of the current time point to every 5 minutes of the current time.
4. The utility model provides a system for realize water level early warning based on cluster and GRU which characterized in that includes:
the acquisition module is used for acquiring water level information and calculating a water level change data set according to the acquired water level information;
the clustering module is used for clustering the water level variation data set based on a DBSCAN clustering algorithm to obtain normal data and abnormal data corresponding to the water level variation data set;
the training module is used for inputting the obtained normal data and abnormal data into a GRU network for training to obtain weight parameters for predicting abnormal points, and a water level prediction model is constructed according to the obtained weight parameters;
the output module is used for inputting the test data into the constructed water level prediction model, and the water level prediction model outputs water level information;
the acquisition module further comprises the following steps after calculating a water level variation information set according to the acquired water level information: for each piece of water level variation data X _0 ═ X 0 ,X 1 ,X 2 …X t …X n ]Carrying out maximum and minimum normalization processing to obtain a normalized data set X ═ X 0 ,x 1 ,x 2 …x t …x n ]；
The clustering module specifically comprises:
a first processing module for selecting an unprocessed object x in the data set i Inspection object x i Is x, if i If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum contained point number minPts, a cluster K is established, and x is used i Marking as processed, marking x i Adding all the points in the corresponding neighborhood into a candidate set C; if x i If the number of the corresponding middle points in the neighborhood is less than the minimum number of contained points minPts, then mark x i Is a noise point;
a second processing module for processing all the unprocessed objects x in the candidate set C j Examination object x j Is x, if j If the number of the corresponding middle points in the neighborhood is more than or equal to the minimum number of contained points minPts, x is added j All points in the corresponding neighborhood are added to candidate set C and x is added j Marked as processed if x j If the object is not classified into any cluster, adding the object into the cluster K; if x j If the number of the corresponding middle points in the neighborhood is less than the minimum number of contained points minPts, then mark x j Is a noise point;
a first loop module for continuing to check for objects in the candidate set C that are not marked until all objects in the candidate set C have been processed;
a second loop module for continuing to examine the objects until all objects fall into a certain cluster or are marked as noise.
5. The system of claim 4, wherein the training module is configured to input the obtained normal data and abnormal data into the GRU network for training, and the normal data and abnormal data are input into the GRU network and processed by forward and backward iterative transfer.
6. The system for realizing water level early warning based on clustering and GRUs as claimed in claim 4, wherein sampling time interval in water level information collection in the collection module is 5 minutes; the data set for calculating the water level change amount is used for calculating the water level change amount from the first 12 hours of the current time point to every 5 minutes of the current time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011543873.9A CN112561203B (en) | 2020-12-23 | 2020-12-23 | Method and system for realizing water level early warning based on clustering and GRU |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011543873.9A CN112561203B (en) | 2020-12-23 | 2020-12-23 | Method and system for realizing water level early warning based on clustering and GRU |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112561203A CN112561203A (en) | 2021-03-26 |
| CN112561203B true CN112561203B (en) | 2022-09-06 |
Family
ID=75031895
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011543873.9A Active CN112561203B (en) | 2020-12-23 | 2020-12-23 | Method and system for realizing water level early warning based on clustering and GRU |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112561203B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110826648A (en) * | 2020-01-09 | 2020-02-21 | 浙江鹏信信息科技股份有限公司 | Method for realizing fault detection by utilizing time sequence clustering algorithm |
| CN110991776A (en) * | 2020-03-04 | 2020-04-10 | 浙江鹏信信息科技股份有限公司 | Method and system for realizing water level prediction based on GRU network |
| CN111401599A (en) * | 2019-08-01 | 2020-07-10 | 河海大学 | Water level prediction method based on similarity search and L STM neural network |
| US10725438B1 (en) * | 2019-10-01 | 2020-07-28 | 11114140 Canada Inc. | System and method for automated water operations for aquatic facilities using image-based machine learning |
-
2020
- 2020-12-23 CN CN202011543873.9A patent/CN112561203B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111401599A (en) * | 2019-08-01 | 2020-07-10 | 河海大学 | Water level prediction method based on similarity search and L STM neural network |
| US10725438B1 (en) * | 2019-10-01 | 2020-07-28 | 11114140 Canada Inc. | System and method for automated water operations for aquatic facilities using image-based machine learning |
| CN110826648A (en) * | 2020-01-09 | 2020-02-21 | 浙江鹏信信息科技股份有限公司 | Method for realizing fault detection by utilizing time sequence clustering algorithm |
| CN110991776A (en) * | 2020-03-04 | 2020-04-10 | 浙江鹏信信息科技股份有限公司 | Method and system for realizing water level prediction based on GRU network |
Non-Patent Citations (2)
| Title |
|---|
| SOM-RBF神经网络模型在地下水位预测中的应用;刘博等;《吉林大学学报(地球科学版)》;20150126(第01期);全文 * |
| 改进RBF神经网络在地下水动态预报中的应用;邱林等;《华北水利水电学院学报》;20040630(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112561203A (en) | 2021-03-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109492822B (en) | Air pollutant concentration time-space domain correlation prediction method | |
| CN106453293B (en) | A kind of network security situation prediction method based on improved BPNN | |
| CN109141847B (en) | Aircraft system fault diagnosis method based on MSCNN deep learning | |
| CN109766583A (en) | Based on no label, unbalanced, initial value uncertain data aero-engine service life prediction technique | |
| CN113064093B (en) | Method and system for jointly estimating state of charge and state of health of energy storage battery | |
| CN109145516B (en) | Analog circuit fault identification method based on improved extreme learning machine | |
| CN110020712B (en) | Optimized particle swarm BP network prediction method and system based on clustering | |
| CN105784556A (en) | Soft measuring method of air particulate matter 2.5 (PM2.5) based on self-organizing fuzzy neural network | |
| CN112733997A (en) | Hydrological time series prediction optimization method based on WOA-LSTM-MC | |
| CN110991776A (en) | Method and system for realizing water level prediction based on GRU network | |
| CN104539601A (en) | Reliability analysis method and system for dynamic network attack process | |
| CN112116002A (en) | Determination method, verification method and device of detection model | |
| Wang et al. | A remaining useful life prediction model based on hybrid long-short sequences for engines | |
| CN114386686A (en) | Improved LSTM-based watershed water quality short-term prediction method | |
| CN116303786B (en) | Block chain financial big data management system based on multidimensional data fusion algorithm | |
| CN113836822A (en) | Aero-engine service life prediction method based on MCLSTM model | |
| CN112561203B (en) | Method and system for realizing water level early warning based on clustering and GRU | |
| CN117350146A (en) | GA-BP neural network-based drainage pipe network health evaluation method | |
| CN115618743B (en) | State evaluation method and state evaluation system of sighting telescope system | |
| CN111914488A (en) | Data regional hydrological parameter calibration method based on antagonistic neural network | |
| CN115330085A (en) | Wind speed prediction method based on deep neural network and without future information leakage | |
| CN113988210A (en) | Method and device for restoring distorted data of structure monitoring sensor network and storage medium | |
| CN113379063B (en) | Whole-flow task time sequence intelligent decision-making method based on online reinforcement learning model | |
| CN113762591A (en) | Short-term electric quantity prediction method and system based on GRU and multi-core SVM counterstudy | |
| Yang et al. | An improved probabilistic neural network with ga optimization |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |