CN109359698A - Leakage loss recognition methods based on long Memory Neural Networks model in short-term - Google Patents

Leakage loss recognition methods based on long Memory Neural Networks model in short-term Download PDF

Info

Publication number
CN109359698A
CN109359698A CN201811283725.0A CN201811283725A CN109359698A CN 109359698 A CN109359698 A CN 109359698A CN 201811283725 A CN201811283725 A CN 201811283725A CN 109359698 A CN109359698 A CN 109359698A
Authority
CN
China
Prior art keywords
time
data
flow
residual
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811283725.0A
Other languages
Chinese (zh)
Other versions
CN109359698B (en
Inventor
刘书明
王晓婷
吴雪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN201811283725.0A priority Critical patent/CN109359698B/en
Publication of CN109359698A publication Critical patent/CN109359698A/en
Application granted granted Critical
Publication of CN109359698B publication Critical patent/CN109359698B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

Present disclose provides a kind of pipe network model recognition methods based on long Memory Neural Networks model in short-term, comprising the following steps: S1 obtains DMA entry data;S2 cleans the DMA entry data of acquisition, and constructs multiple dimensioned time data set;S3 establishes long Memory Neural Networks model in short-term;Memory Neural Networks model carries out the identification of abnormal flow point to the length of S4, the multiple dimensioned time data set based on building and foundation in short-term;S5 carries out pipe network model identification according to the abnormal flow point of identification.Disclosure pipe network model recognition methods reduces accident rate of false alarm, increases the accuracy of leakage loss identification.

Description

Leakage identification method based on long-time memory neural network model
Technical Field
The disclosure relates to the field of neural networks, in particular to a leakage identification method based on a long-time memory neural network model.
Background
In recent years, water supply pipe network leakage accidents with sudden and high uncertainty are frequently generated and are widely concerned by society, and the leakage accidents not only can cause water energy waste and economic loss, but also can cause water quality pollution of the pipe network in serious cases and harm public health. However, the water supply pipe network is deeply buried at the bottom of the ground, the system is huge, and the environmental interference factors are many, so that the pipe network state is not easy to monitor, and the difficulty of detecting the leakage of the pipe network is greatly increased.
In recent years, in the global water supply industry, independent Metering areas (DMA) are vigorously popularized and practiced for effectively controlling leakage of a pipe network. The commonly used DMA leakage monitoring is mainly to obtain the flow change of the area by monitoring the change of the minimum flow of the DMA at night, but the method has the problems of low detection efficiency, long detection time and the like. The occurrence process of the leakage accident has a time difference, namely the time from the occurrence of the leakage of the pipe network to the discovery of the leakage accident by the water supply company is short, and the longer the time difference is, the larger the water leakage amount is. Therefore, how to find the leakage as soon as possible, reduce the time difference and reduce the water leakage amount is the key of the leakage control of the pipe network.
With the importance and application of the SCADA system in each water supply company, more and more water supply companies begin to pay attention to collecting huge monitoring data of a water supply network, and a data mining-based leakage identification method with higher efficiency and lower cost compared with a traditional leakage detection hardware method becomes a current research hotspot. The data mining method is mainly characterized in that a prediction model is built through a least square method, polynomial regression, an ARIMA model, support vector regression, a BP neural network and the like, and then the difference value of a predicted value and an actually measured value is compared through a fixed threshold value to identify the leakage accident of the pipe network.
The traditional model can successfully identify leakage accidents with large flow, but cannot rapidly and accurately process a large amount of flow data and has low fault-tolerant capability under the background of higher and higher online monitoring frequency of a pipe network, more intensive data acquisition and uneven data quality. Application number 201410507513.1 provides a water pipe water leakage detection method based on wavelet singularity analysis and an ARMA model, whether a slow leakage phenomenon of water pipe water leakage exists is judged through two-step prediction data analysis, and the problem of slow leakage judgment is successfully solved. Application number 201710998436.8 obtains a predicted water quantity value through neural network model prediction of a gate control circulation unit, and compares the predicted water quantity value with a measured water quantity value through a cosine included angle method, so as to realize pipe network leakage identification; however, these methods do not consider the correction and input of the feedback of the water amount, the continuous identification of pipe network pipe explosion cannot be carried out in real time, and a single threshold alarm causes large misjudgment, which causes unnecessary manpower and material resource loss of a water supply company, and the reliability in the actual operation of the method is low. Considering that the leakage accident of the water supply network is a continuous process, a continuous leakage identification method with strong learning capability, high fault tolerance capability and accurate identification precision is urgently needed.
Disclosure of Invention
Technical problem to be solved
In view of the above problems, it is a primary object of the present disclosure to provide a method for identifying a leakage based on a long-term and short-term memory neural network model, so as to solve at least one of the above problems.
(II) technical scheme
In order to achieve the above object, as an aspect of the present disclosure, a pipe network leakage identification method based on a long-time memory neural network model is provided, including the following steps:
s1, obtaining DMA entry data;
s2, cleaning the obtained DMA entry data, and constructing a multi-scale time data set;
s3, establishing a long-term and short-term memory neural network model;
s4, identifying abnormal flow points based on the built multi-scale time data set and the built long-time and short-time memory neural network model;
and S5, identifying the leakage of the pipe network according to the identified abnormal flow points.
In some embodiments, in step S1, when the DMA is a single entry, obtaining the single entry traffic data of the DMA; when the DMA is a multi-entry, acquiring the flow sum of the DMA multi-entry; the flow data sampling interval isThe sampling number is 24k times in minutes/time and all the day, namely 24k pieces of historical flow data are acquired each day.
In some embodiments, the step S2 includes the following sub-steps:
s21, cleaning the acquired DMA entry data, and constructing a continuous time sequence by using the cleaned data;
s22, segmenting the continuous time sequence to construct a date sequence;
and S23, constructing a multi-scale time data set based on the time sequence and the date sequence, wherein the multi-scale time data set is input data of the long-time and short-time memory neural network model.
In some embodiments, the step of flushing the obtained DMA entry data comprises checking DMA entry data consistency and padding missing values.
In some embodiments, the step S3 includes:
taking the constructed multi-scale time data set as model input; wherein the multi-scale temporal data set comprises a training set and a validation set;
taking the predicted flow of the DMA inlet at the next moment as a model to be output; and
selecting tanh, ReLU or Linear as an activation function of each network layer of the model, and constructing the long-term memory neural network model.
In some embodiments, in the step of using the constructed multi-scale temporal dataset as a model input,
selecting an adjacent date sequence of t-time flow to be predicted, namely date sequence data of t-1 time, t +1 time and t time as model input, and setting the date sequence data as a first dimension time period, a second dimension time period and a third dimension time period; and
and selecting the adjacent time sequence data of the flow at the t moment to be predicted as model input, namely setting the previous c moments of the t moment as a fourth dimension time period.
In some embodiments, the step S4 includes:
s41, calculating residual error, and calculating the predicted flow output by the long-time memory neural network model in the training set dataAnd measured value XtResidual error R oftThe calculation formula is as follows:
s42, cutting the residual error, and dividing the residual error RtAnd (4) segmenting according to time points, forming the same group of data by the same sampling time point, and obtaining 24k groups of residual sequences.
S43, calculating residual threshold after segmentation, calculating 3 sigma intervals for the 24k groups of residual errors respectively, wherein the upper and lower boundaries of the 3 sigma intervals form 24k groups of upper and lower residual threshold corresponding to 24k time points, and the nth time residual r of the mth daymnThe 3 σ threshold interval calculation formula is as follows:
in the formula,is the residual mean value corresponding to the moment n; var [ R ]n]Is the residual standard deviation corresponding to the moment n;
s44, identifying abnormal flow points, calculating flow residual errors of the verification set at the moment t, and if the residual error value at the moment t is greater than an upper residual error threshold corresponding to the moment t, determining the time point as an abnormal flow point and marking as a high flow point; and if the residual value at the moment t is smaller than the lower residual threshold corresponding to the moment, determining that the time point is an abnormal flow point and recording as a low flow point.
In some embodiments, the step S5 includes:
s51, correcting the abnormal flow point and inputting the abnormal flow point into the long-time and short-time memory neural network model for calculation;
and S52, counting the number of continuous high-flow points according to the calculation result obtained by inputting the corrected long-time and short-time memory neural network model, and identifying the leakage of the pipe network according to the number of the continuous high-flow points.
In some embodiments, in the step S51, when the nth time is determined as the abnormal flow rate point, the corrected value of the flow rate data at the nth time is determined as the abnormal flow rate pointWherein,the flow prediction value output by the long-time memory neural network corresponding to the nth time of the mth day is obtained;is the residual error correction value at the moment, which is R in the residual error sequence of the training setnThe average value of the values is calculated,
in some embodiments, in the step S52, the number Q of continuous high flow points is counted, when Q is greater than or equal to a threshold Q, a warning is started, and the leakage accident duration T is calculatedburst(ii) a The leakage accident detection time calculation formula is as follows:
the leakage accident duration calculation formula is as follows:
(III) advantageous effects
According to the technical scheme, the leakage identification method based on the long-time and short-time memory neural network model has at least one of the following beneficial effects:
(1) the method is different from a traditional machine learning method for leakage identification, a deep learning method is adopted in the method, a date sequence and time sequence data of predicted flow data are fused through an LSTM model, the predicted flow is closer to non-fault real flow compared with a traditional algorithm, a long-time memory neural network has strong fault tolerance, abnormal data can be filtered in a certain range through a forgetting gate in the LSTM under the condition that input of a model training set is abnormal, output is guaranteed to be closer to non-fault real flow again, the training significance is not to predict real-time flow data, but to learn the water volume change trend under the condition that the predicted moment is real and not fault, and a prediction basis is provided for subsequent leakage identification.
(2) Different from the traditional single-threshold leakage identification method, the multi-threshold identification method based on the instantaneized residual error is provided in the disclosure. And carrying out segmentation processing on the residual sequence to obtain 24k residual subsequences corresponding to 24k sampling moments, calculating each group of residual subsequences, and taking the upper boundary of each group of 3 sigma intervals as leakage identification thresholds corresponding to 24k sampling moments respectively to form refined thresholds aiming at different sampling moments. The multi-threshold leakage identification method based on the time-varying residual carries out refined treatment on the residual judgment, reduces the influence of water consumption change of a user on detection, greatly reduces the false alarm rate of a pipe network, avoids manpower loss caused by false alarm, and improves the model trust level.
(3) Different from the traditional single abnormal point non-correction alarm identification method, the multi-point alarm method based on feedback correction is provided in the disclosure. By comparing the data characteristics of the true value and the abnormal value, an accurate replacement method for replacing the abnormal value with the sum of the predicted value and the time-based residual error deviation is provided, the abnormality is replaced in time and fed back to the model input, the next-time prediction deviation caused when the abnormal value is used as the input is avoided, and the continuous abnormal point identification capability of the model is ensured. Compared with single-point abnormity judgment, the multipoint identification method greatly reduces noise misjudgment caused by instrument faults, transmission errors and the like, reduces the false alarm rate of accidents, increases the accuracy of leakage identification, effectively identifies leakage accidents and the duration of the leakage accidents, and assists a tap water company to make intelligent decisions.
Drawings
Fig. 1 is a topological structure diagram of a long-term and short-term memory-based neural network according to the present disclosure.
Fig. 2 is a structural diagram of a multi-threshold leakage detection method based on a temporal residual error according to the present disclosure.
Fig. 3 is a topological structure diagram of a long-term and short-term memory-based neural network according to the present disclosure.
Fig. 4 is a flowchart of a leak identification method of the present disclosure.
Detailed Description
For the purpose of promoting a better understanding of the objects, aspects and advantages of the present disclosure, reference is made to the following detailed description taken in conjunction with the accompanying drawings.
In order to protect water resources, reduce the leakage rate of a pipe network, reduce economic loss and water quality safety risks caused by leakage accidents and guarantee safe and reliable operation of water supply, the prior art is required to deeply excavate hydraulic characteristic data which can be monitored by the water supply pipe network and establish a model which is real-time, strong in fault-tolerant capability, high in precision and low in false alarm to identify the leakage of the pipe network. Aiming at the defects of the prior art, the present disclosure aims to provide a time recursive cyclic neural network model based on long-time and short-time memory, a training method and an application thereof, and a water supply network continuous leakage accident identification method and an application thereof based on a multi-threshold feedback correction model. By constructing a long-time and short-time memory neural network prediction model and a multi-threshold feedback correction recognition model, a large amount of short-time data are processed in real time, the characteristic relation of pipe network data is mined, the abnormal flow points of the pipe network are judged at a high accurate report rate and a low false report rate, the duration of the leakage accident is monitored, and the rapid alarm and continuous recognition of the leakage accident are realized. The method can diagnose and early warn the leakage of the water supply pipe network with high precision, greatly shorten the response time of leakage accidents, improve the accident judgment reliability, and effectively assist water supply companies in diagnosing the leakage of the water supply pipe network and making early warning decisions.
The utility model provides a water supply network leakage recognition method based on model (LSTM) of long-time memory neural network, input and training, which is a water supply network leakage accident recognition method with multiple thresholds, can comprehensively recognize water supply network leakage accidents based on multi-point alarm of feedback correction, and comprises the following steps:
s1, acquiring DMA (independent Metering Area) entry data;
s2, cleaning the obtained DMA entry data, and constructing a multi-scale time data set;
s3, establishing a long-time memory warp lining network model;
s4, identifying abnormal flow points;
and S5, identifying the leakage of the pipe network.
The steps of the leakage identification method of the long-term memory neural network model are described in detail below with reference to fig. 1 to 4.
S1, acquiring DMA entry data:
specifically, when the DMA cell is a single entry, the DMA entry flow data is acquired; when the DMA cell is a multi-entry, acquiring the flow sum of the DMA entry;
wherein the traffic data sampling interval isThe sampling number is 24k times in minutes/time and all the day, namely 24k pieces of historical flow data are acquired each day. Of course, the sampling interval and the number of samples in the present disclosure are not limited thereto.
In addition, the DMA entry data may also be a user water volume or other data.
S2, cleaning the obtained DMA entry data, and constructing a multi-scale time data set, which comprises the following sub-steps:
and S21, cleaning the acquired DMA entry data, and constructing a continuous time sequence by using the cleaned data.
Specifically, the data cleaning includes filling in missing data values, and the method selects an interpolation method. Of course, the disclosed data cleansing process and method are not so limited.
S22, segmenting the continuous time sequence to construct a date sequence;
and acquiring historical flow data of the pipe network of the data to be detected, and reconstructing the historical flow time sequence in a segmentation mode. The daily sampling frequency of the historical flow data is k times/hour, the daily sampling number is 24k times, and a days are sampled. The segmentation reconstruction expression is as follows:
f11flow data representing the 1 st acquisition on day 1, f1,24kFlow data representing 24k acquisitions on day 1, fa1Flow data representing the 1 st acquisition on day a, fa,24kFlow data representing the 24k acquisition on day a. Similarly, f in FIG. 1d-7,t-1Represents the data corresponding to t-1 time 7 days ago, fd-6,tData corresponding to time t 6 days ago are shown.
The historical flow matrix is reconstructed by reconstructing the historical flow sequence by taking the date as a matrix row and taking the sampling time point as a matrix column.
The matrix is a date sequence in which dates are sequentially arranged to represent periodic variation in the longitudinal direction.
And S23, forming a multi-scale time data set based on the time sequence and the date sequence, wherein the data set is input data for a long-time memory neural network model.
S3, establishing a long-time memory neural network model, which comprises the following substeps:
and S31, normalizing the model input data. The formula is as follows:
wherein y represents normalized data, x represents input raw data, and x representsmaxAnd xminRespectively representing the maximum value and the minimum value of input flow data;
and S32, extracting model input features, extracting date sequences containing three dimensions and time sequences containing one dimension, and extracting input features containing four time dimensions. And (4) sequentially inputting feature data of features closer to the time to be predicted in consideration of the LSTM long-time memory characteristics.
The steps S2, S31, and S32 correspond to the feature extraction step in fig. 4.
The three-dimensional characteristic data in front of the model is a date sequence near the t moment to be predicted, and specifically comprises,
selecting an adjacent date sequence of t-time flow to be predicted, namely date sequence data of t-1 time, t +1 time and t time as model input, setting the date sequence data as a first dimension time period, a second dimension time period and a third dimension time period, and expressing the date sequence data as:
the data set at times t-1-24 k.
A data set at times t +1-24k b, t +1-24k (b-1) of a second dimension,
a data set at time t-24 k.
Wherein the data sets at times t-1-24k b, t-1-24k (b-1.).. t-1-24k represent data from previous day b, previous day b-1, and … … to previous day 1, respectively; the data sets at times t +1-24k b, t +1-24k (b-1.).. t +1-24k represent data from days t +1, from day b-1, and from … … to day 1, respectively; the data set at time t-24 k.b, t-24k (b-1.).. t-24k represents data from previous b days, previous b-1 days, and.... until time t of previous 1 day, respectively.
Preferably, b is an integer of 3 to 28, preferably 7.
The model fourth-dimensional feature data is a time sequence near a t moment to be predicted, and specifically comprises the steps of selecting near time sequence data of flow near the t moment to be predicted as model input, namely c moments before the t moment, setting the near time sequence data as fourth-dimensional time periods which are represented as t-c and t- (c-1).. the. b is c.
S33, referring to fig. 1, the long-short term memory neural network model includes a long-short term memory unit layer and a plurality of fully connected neural unit layers, each unit layer is connected in series to form a multi-level input transmission layer;
each cycle of the long-short time memory unit layer comprises a plurality of long-short time memory cells. The data transmission relationship among the cells is that the data input by each cell passes through an input gate, a forgetting gate and an output gate in the cell, the output is long-term memory and short-term memory at the next moment, the data enter the next cell, sequentially circulate and finally are output to the next neural unit layer.
The fully-connected neural unit layer is a basic unit layer of the artificial neural network;
the model input is a data set constructed based on time series and date series flow data;
the model output is the predicted flow at the next moment of the DMA inlet;
the number of the fully-connected neural network layers is an integer greater than or equal to 2;
the activation function of each network layer selects tanh, ReLU or Linear.
Dividing the obtained DMA flow data (cleaned data) into a training set and a verification set according to a proportion, sequentially inputting the training set and the verification set into a long-time memory neural network model, and outputting the flow data at the next time (the next time of the time corresponding to the input data).
S4, identifying abnormal flow points:
the abnormal flow points comprise a high flow point and a low flow point;
s41, calculating residual error, and calculating the predicted flow output by the long-time memory neural network model in the training set dataAnd measured value XtResidual error R oft(t ═ m-1) × 24k + n), the formula calculated is:
s42, cutting the residual error, and combining the residual error sequence RtThe data are divided according to time points, the same group of data is formed by the same sampling time points, 24k groups of residual error sequences are obtained in total, and the residual error sequences R are carved immediately1~R24k
And S43, calculating a residual threshold value after segmentation. Respectively calculating 3 sigma threshold values of the 24k groups of residual errors, and the n-th time residual error r of the m daymnThe 3 σ threshold interval calculation formula is as follows:
in the formula (4), RnIs the nth column data of the residual matrix R,is the residual average value corresponding to the nth moment; var [ R ]n]Is the residual standard deviation corresponding to the nth time.
The upper and lower boundaries of the 3 sigma interval form 24k groups of upper and lower residual threshold values corresponding to 24k time points
And S44, identifying abnormal flow points, and calculating the flow residual error at the moment t of the verification set. the segmentation time corresponding to the time t is the nth time of the mth day, and when the residual value at the time t is greater than the upper residual threshold corresponding to the timeAnd then, determining that the time point is an abnormal flow point and marking as a high flow point. When the residual value at the moment t is smaller than the corresponding lower residual threshold value at the momentThen, it is determined that the time point is an abnormal traffic point, and the time point is marked as a low traffic point, as shown in fig. 2.
S5, identifying pipe network leakage, as shown in fig. 3:
s51, correcting abnormal flow point
When the residual error is marked as an abnormal traffic point, the abnormal traffic of the point needs to be replaced. The replaced value is used as input to enter the model again for carrying out flow calculation at the next moment. When the nth time is judged as the abnormal flow rate point on the mth day, the corrected value of the flow rate data at the mth dayWherein,the flow prediction value output by the long-time memory neural network corresponding to the nth time of the mth day is obtained;is residual error correction value of nth time of mth day, which is R in residual error sequence of training setnIs determined by the average value of (a) of (b),
s52, an accident identification and alarm step, counting the number Q of continuous high flow points, starting early warning when Q is larger than or equal to an alarm threshold value Q (pipe network leakage occurs), and calculating the duration time T of the leakage accidentburst. The quantity threshold Q is preferably 2 to 3.
Leakage accident detection time TdetectCalculation formula (7)
The leakage accident duration TburstCalculation formula (8)
In the following embodiment, Python 3.6 software is used as a development platform of the model, Numpy and Pandas libraries are used for reading, storing and analyzing data, a Matplotlib library is used for visualizing the data, and a Keras library is used for building a neural network model, so that the development efficiency is greatly improved.
The specific steps of a leakage identification method based on a long-time and short-time memory neural network model (LSTM) and multi-threshold feedback correction are described in detail below by taking a certain DMA water supply network in SX city in China as an embodiment:
1. acquiring DMA entry data:
and acquiring the inlet flow data of a DMA water supply pipe network in SX city, wherein the data date ranges from 10 days at 7 months in 2015 to 21 days at 11 months in 2015, and the total period is 135 days. The sensor sampling interval records instantaneous flow data once every 5min (12 times/h), i.e. 288 samples per day. When the DMA cell is a single inlet, acquiring the DMA inlet flow data; and when the DMA cell is a multi-entry, acquiring the total flow of the DMA entry. This example is a 3-entry multiple entry DMA, and therefore the sum of the DMA entry flows is calculated as historical flow data.
2. Data cleaning and construction of multi-scale time data sets:
1) the DMA entry data is preprocessed. The sampling frequency of the DMA entry instantaneous flow data obtained in this example is 12 times/hour, the sampling frequency is 288 times per day, the total amount of data is 38880, the data is cleaned, the data consistency is checked, and the missing value is filled. Wherein, the method for filling missing values selects an interpolation method. The washed historical flow data are sequentially arranged in time sequence to form a time sequence { f1,f2,…,f38880And characterizing the trend change characteristics of the flow.
2) And (4) segmenting continuous time sequences to construct date sequences. And reconstructing the data after data cleaning in a segmentation mode, reconstructing the time sequence by taking the date as a matrix row and taking the sampling time point as a matrix column, and reconstructing the time sequence into a historical flow matrix.
The matrix is longitudinally a date sequence with the same time point arranged in sequence by date, and the periodic variation characteristic of the flow is represented.
3) The time sequence and the date sequence form a model data set, and the data set provides basic data for inputting the long-time memory neural network.
3. Establishing long-time and short-time memory neural network model
1) Extracting the characteristics of the model data set, extracting a date sequence within 7 days close to the flow data at the time t and a time sequence within 35 minutes (7 sampling time points), normalizing according to the formula (2) and inputting the normalized data into a model as characteristics,
the date sequence flow data of the time t-1 in the first time period comprise flow data of the time t-1-24h 7, t-1-24h 6, t-1-24h 5, t-1-24h 4, t-1-24h 3, t-1-24h 2 and t-1-24h 1;
the date sequence flow data of the second time period at the moment t +1 comprise flow data at the moments t +1-24h 7, t +1-24h 6, t +1-24h 5, t +1-24h 4, t +1-24h 3, t +1-24h 2 and t +1-24h 1;
the third time period is date sequence flow data at the time t, and the date sequence flow data comprises flow data at the time t-24h 7, t-24h 6, t-24h 5, t-24h 4, t-24h 3, t-24h 2 and t-24h 1;
the fourth time period is time series flow data close to the t moment, and comprises the flow data of t-35min, t-30min, t-25min, t-20min, t-15min, t-10min, t-5min and the moment;
the four time periods are respectively input into the long-time and short-time memory neural network model at 7 moments in four dimensions.
2) The Long Short-Term Memory neural network (LSTM) provided by the disclosure is an evolution model of a recurrent neural network, and is more suitable for learning from experience so as to classify, process and predict a time sequence. The method is characterized in that the method can compress input vector representation in a recurrent neural network and memorize and update prediction output of a model in long and short time.
The model is composed of 3 LSTM layers and 3 fully-connected neural network layers as input layers, and a multi-level input transfer layer is formed and configured to receive the four-dimensional flow characteristic data. The number of 3 LSTM layer nodes is set to 128, 64, 48, respectively, and the LSTM layer activation function is set to tanh. The activation function of the first and second fully-connected neural network layers is ReLU, and the activation function of the third fully-connected neural network layer is Linear. The third fully-connected neural network layer is connected with the output layer, the activation function of the output layer is Linear, the learning rate is 0.002, and the batch _ size is 60.
28000 samples are selected from a training set, historical flow characteristic input samples are randomly disturbed, 25200 samples are randomly selected as training samples, and 5184 samples are selected from a verification set.
The input of the model is the normalized four-dimensional flow characteristic data.
The output of the model is the next period of flow for the DMA entry.
4. Abnormal flow point identification
1) Calculating the actual measured value sequence X of the training set of the calculation model as the formula (3)tAnd a sequence of predictorsResidual sequence R betweent
2) For training set residual error sequence RtAnd performing time-interval segmentation processing. The model training set contains 87 days of flow data, and a residual sequence R is obtained at intervals of 5mintThe division into 288 groups each contains 87 data, i.e. each group corresponds to the change of data with days at the same time.
3) For each set of 3 sigma intervals of the residual calculation formula (4), the upper and lower bounds of the 3 sigma interval are taken as the upper and lower residual thresholds at the moment.
4) And calculating the residual error of the verification set, and marking an abnormal flow point when the residual error value is outside the 3 sigma interval. And when the residual value is larger than the upper residual threshold value, marking as a high-flow point. Table 1 is a partial example of a temporal residual threshold (upper residual threshold). The threshold value can be regarded as the identifiable minimum flow size of the disclosure, namely the identifiable minimum leakage range of the disclosure is between 2.85% and 13.14% of the daily average flow percentage, and the threshold value has the characteristics of strong practical applicability of an actual pipe network, high accident sensitivity and the like.
TABLE 1 residual threshold
5. Pipe network leakage identification:
1) and correcting abnormal flow points.
The 288 groups of residual sequences in the training set in 4(2) are averaged to form a residual correction sequence, which corresponds to 288 time instants.
And carrying out model effect verification by adopting verification set data. Namely, during the period from 3/11/2015 to 21/11/2017, when a residual error is detected as an abnormal traffic point, the traffic of the point needs to be marked and replaced in time. And the replaced value is used as an input to enter the model again, and the flow calculation at the next moment is carried out. And replacing the abnormal value at the time t with the sum of the flow predicted value at the time t and the residual error correction sequence as input to perform flow prediction at the time t + 1. Taking the accident 1 in table 2 as an example, after the 2:20 residual error is greater than the upper residual error threshold value in 17 days 11 months, the point is marked as a high flow rate point, and the abnormal flow rate value at the moment is replaced.
2) And accident identification and alarm. In this embodiment, the alarm quantity threshold Q is 2, that is, when the number of the continuous high flow points is counted to be greater than 2, the alarm is started to identify that the leakage accident occurs at the moment, and the quick detection characteristic is provided when the leakage detection time is calculated to be within 10 minutes according to formula (5).
The embodiment includes the real simulation of the leakage accident in the pipe network, that is, the flow change of the leakage accident is simulated by opening the fire hydrant, and the included accident and the recognition result are shown in table 2. The method successfully identifies the flow change of a real leakage experiment, carries out model identification effect evaluation by a common evaluation method confusion matrix method, has the advantages that the accurate report rate is up to 100 percent, the false report rate is as low as 0.19 percent, and the model has the characteristics of high accurate report rate, low false report rate and the like.
TABLE 2 identification results
The above results show that the leakage identification method based on the long-time and short-time memory neural network model and the multi-threshold feedback correction model can quickly and accurately identify the leakage accident of the pipe network with high report rate and low false alarm rate, and the method has strong practicability and strong fault tolerance capability on data quality. The method expands and strengthens the research content of the existing water supply network identification model, and provides a new idea for a tap water company to make scientific and reasonable decisions.
Further, the above definitions of the various elements and methods are not limited to the various specific structures, shapes or arrangements of parts mentioned in the examples, which may be easily modified or substituted by those of ordinary skill in the art.
It should be noted that directional terms, such as "upper", "lower", "front", "rear", "left", "right", and the like, mentioned in the embodiments are only directions referring to the drawings, and are not intended to limit the scope of the present disclosure. Throughout the drawings, like elements are represented by like or similar reference numerals. Conventional structures or constructions will be omitted when they may obscure the understanding of the present disclosure. And the shapes and sizes of the respective components in the drawings do not reflect actual sizes and proportions, but merely illustrate the contents of the embodiments of the present disclosure. Furthermore, in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim.
Furthermore, the word "comprising" or "comprises" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements.
The use of ordinal numbers such as "first," "second," "third," etc., in the specification and claims to modify a corresponding element does not by itself connote any ordinal number of the element or any ordering of one element from another or the order of manufacture, and the use of the ordinal numbers is only used to distinguish one element having a certain name from another element having a same name.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the disclosure, various features of the disclosure are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various disclosed aspects. However, the disclosed method should not be interpreted as reflecting an intention that: that is, the claimed disclosure requires more features than are expressly recited in each claim. Rather, as the following claims reflect, disclosed aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of this disclosure.
The above-mentioned embodiments are intended to illustrate the objects, aspects and advantages of the present disclosure in further detail, and it should be understood that the above-mentioned embodiments are only illustrative of the present disclosure and are not intended to limit the present disclosure, and any modifications, equivalents, improvements and the like made within the spirit and principle of the present disclosure should be included in the scope of the present disclosure.

Claims (10)

1. A pipe network leakage identification method based on a long-time memory neural network model comprises the following steps:
s1, obtaining DMA entry data;
s2, cleaning the obtained DMA entry data, and constructing a multi-scale time data set;
s3, establishing a long-term and short-term memory neural network model;
s4, identifying abnormal flow points based on the built multi-scale time data set and the built long-time and short-time memory neural network model;
and S5, identifying the leakage of the pipe network according to the identified abnormal flow points.
2. The method according to claim 1, wherein, in step S1, when the DMA is a single entry, the DMA single entry traffic data is obtained; when the DMA is a multi-entry, acquiring the flow sum of the DMA multi-entry; the flow data sampling interval isThe sampling number is 24k times in minutes/time and all the day, namely 24k pieces of historical flow data are acquired each day.
3. The method according to claim 2, wherein the step S2 includes the sub-steps of:
s21, cleaning the acquired DMA entry data, and constructing a continuous time sequence by using the cleaned data;
s22, segmenting the continuous time sequence to construct a date sequence;
and S23, constructing a multi-scale time data set based on the time sequence and the date sequence, wherein the multi-scale time data set is input data of the long-time and short-time memory neural network model.
4. The method of claim 3, wherein the step of flushing the obtained DMA entry data comprises checking DMA entry data for consistency and padding missing values.
5. The method according to claim 3, wherein the step S3 includes:
taking the constructed multi-scale time data set as model input; wherein the multi-scale temporal data set comprises a training set and a validation set;
taking the predicted flow of the DMA inlet at the next moment as a model to be output; and
selecting tanh, ReLU or Linear as an activation function of each network layer of the model, and constructing the long-term memory neural network model.
6. The method of claim 5, wherein, in the step of inputting the constructed multi-scale temporal dataset as a model,
selecting an adjacent date sequence of t-time flow to be predicted, namely date sequence data of t-1 time, t +1 time and t time as model input, and setting the date sequence data as a first dimension time period, a second dimension time period and a third dimension time period; and
and selecting the adjacent time sequence data of the flow at the t moment to be predicted as model input, namely setting the previous c moments of the t moment as a fourth dimension time period.
7. The method according to claim 6, wherein the step S4 includes:
s41, calculating residual error, and calculating the predicted flow output by the long-time memory neural network model in the training set dataAnd measured value XtResidual error R oftThe calculation formula is as follows:
s42, cutting the residual error, and dividing the residual error RtAnd (4) segmenting according to time points, forming the same group of data by the same sampling time point, and obtaining 24k groups of residual sequences.
S43, calculating residual threshold after segmentation, calculating 3 sigma intervals for the 24k groups of residual errors respectively, wherein the upper and lower boundaries of the 3 sigma intervals form 24k groups of upper and lower residual threshold corresponding to 24k time points, and the nth time residual r of the mth daymnThe 3 σ threshold interval calculation formula is as follows:
in the formula,is the residual mean value corresponding to the moment n; var [ R ]n]Is the residual standard deviation corresponding to the moment n;
s44, identifying abnormal flow points, calculating flow residual errors of the verification set at the moment t, and if the residual error value at the moment t is greater than an upper residual error threshold corresponding to the moment t, determining the time point as an abnormal flow point and marking as a high flow point; and if the residual value at the moment t is smaller than the lower residual threshold corresponding to the moment, determining that the time point is an abnormal flow point and recording as a low flow point.
8. The method according to claim 7, wherein the step S5 includes:
s51, correcting the abnormal flow point and inputting the abnormal flow point into the long-time and short-time memory neural network model for calculation;
and S52, counting the number of continuous high-flow points according to the calculation result obtained by inputting the corrected long-time and short-time memory neural network model, and identifying the leakage of the pipe network according to the number of the continuous high-flow points.
9. The method according to claim 8, wherein in the step S51, when the nth time on the mth day is determined as the abnormal flow rate point, the correction value of the flow rate data at that time isWherein,the flow prediction value output by the long-time memory neural network corresponding to the nth time of the mth day is obtained;is the residual error correction value at the moment, which is R in the residual error sequence of the training setnThe average value of the values is calculated,
10. the method as claimed in claim 9, wherein in the step S52, the number Q of continuous high flow points is counted, when Q is greater than or equal to a threshold Q, a pre-warning is started, and the leakage accident duration T is calculatedburst(ii) a The leakage accident detection time calculation formula is as follows:
the leakage accident duration calculation formula is as follows:
CN201811283725.0A 2018-10-30 2018-10-30 Leakage identification method based on long-time memory neural network model Active CN109359698B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811283725.0A CN109359698B (en) 2018-10-30 2018-10-30 Leakage identification method based on long-time memory neural network model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811283725.0A CN109359698B (en) 2018-10-30 2018-10-30 Leakage identification method based on long-time memory neural network model

Publications (2)

Publication Number Publication Date
CN109359698A true CN109359698A (en) 2019-02-19
CN109359698B CN109359698B (en) 2020-07-21

Family

ID=65347554

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811283725.0A Active CN109359698B (en) 2018-10-30 2018-10-30 Leakage identification method based on long-time memory neural network model

Country Status (1)

Country Link
CN (1) CN109359698B (en)

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110837933A (en) * 2019-11-11 2020-02-25 重庆远通电子技术开发有限公司 Leakage identification method, device, equipment and storage medium based on neural network
CN111237646A (en) * 2020-02-17 2020-06-05 清华大学 Automatic identification and positioning method for leakage of water supply pipe network
CN112468312A (en) * 2019-09-09 2021-03-09 中兴通讯股份有限公司 Network flow prediction method, communication equipment and storage medium
CN112780953A (en) * 2021-02-08 2021-05-11 浙江工业大学 Independent metering area pipe network leakage detection method based on mode detection
CN112818495A (en) * 2021-02-22 2021-05-18 成都四为电子信息股份有限公司 Novel dynamic correction method for pipeline pressure drop measurement and calculation algorithm parameters
CN112836758A (en) * 2021-02-08 2021-05-25 浙江工业大学 Independent metering area pipe network leakage detection method based on double-layer circulating neural network
CN112923989A (en) * 2021-02-24 2021-06-08 中国水利水电科学研究院 Household terminal water consumption monitoring method and intelligent water meter
CN112925822A (en) * 2021-02-08 2021-06-08 山东大学 Time series classification method, system, medium and device based on multi-representation learning
CN113588179A (en) * 2021-06-24 2021-11-02 武汉众智鸿图科技有限公司 Water supply pipe network leakage detection method and system
CN113944888A (en) * 2021-11-03 2022-01-18 北京软通智慧科技有限公司 Gas pipeline leakage detection method, device, equipment and storage medium
CN114352947A (en) * 2021-12-08 2022-04-15 天翼物联科技有限公司 Gas pipeline leakage detection method, system and device and storage medium
CN115580486A (en) * 2022-11-18 2023-01-06 宁波市镇海区大数据投资发展有限公司 Network security sensing method and device based on big data
CN116642138A (en) * 2023-05-25 2023-08-25 大连智水慧成科技有限责任公司 New leakage detection method for water supply network
CN117093854A (en) * 2023-10-19 2023-11-21 安徽建筑大学 Transformer mechanical fault diagnosis method, equipment and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102072407A (en) * 2009-11-23 2011-05-25 中国科学院生态环境研究中心 Leakage loss detection method combining leakage loss recording instrument with district metering areas (DMA)
CN107103758A (en) * 2017-06-08 2017-08-29 厦门大学 A kind of city area-traffic method for predicting based on deep learning
US20170372475A1 (en) * 2016-06-23 2017-12-28 Siemens Healthcare Gmbh Method and System for Vascular Disease Detection Using Recurrent Neural Networks
CN108051035A (en) * 2017-10-24 2018-05-18 清华大学 The pipe network model recognition methods of neural network model based on gating cycle unit
CN108416478A (en) * 2018-03-19 2018-08-17 国通广达(北京)技术有限公司 Gas concentration LSTM prediction techniques and system inside a kind of pipe gallery
CN108647839A (en) * 2018-06-28 2018-10-12 中国人民解放军陆军军事交通学院镇江校区 Voltage-stablizer water level prediction method based on cost-sensitive LSTM Recognition with Recurrent Neural Network

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102072407A (en) * 2009-11-23 2011-05-25 中国科学院生态环境研究中心 Leakage loss detection method combining leakage loss recording instrument with district metering areas (DMA)
US20170372475A1 (en) * 2016-06-23 2017-12-28 Siemens Healthcare Gmbh Method and System for Vascular Disease Detection Using Recurrent Neural Networks
CN107103758A (en) * 2017-06-08 2017-08-29 厦门大学 A kind of city area-traffic method for predicting based on deep learning
CN108051035A (en) * 2017-10-24 2018-05-18 清华大学 The pipe network model recognition methods of neural network model based on gating cycle unit
CN108416478A (en) * 2018-03-19 2018-08-17 国通广达(北京)技术有限公司 Gas concentration LSTM prediction techniques and system inside a kind of pipe gallery
CN108647839A (en) * 2018-06-28 2018-10-12 中国人民解放军陆军军事交通学院镇江校区 Voltage-stablizer water level prediction method based on cost-sensitive LSTM Recognition with Recurrent Neural Network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
GUANCHENG GUO ET.AL: "Short-Term Water Demand Forecast Based on Deep Learning Method", 《JOURNAL OF WATER RESOURCES PLANNING AND MANAGEMENT》 *
李玉全: "城市供水管网实时建模及漏损事件侦测定位研究", 《万方数据知识服务平台》 *

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112468312A (en) * 2019-09-09 2021-03-09 中兴通讯股份有限公司 Network flow prediction method, communication equipment and storage medium
WO2021047270A1 (en) * 2019-09-09 2021-03-18 中兴通讯股份有限公司 Network traffic prediction method, communication device and storage medium
CN110837933A (en) * 2019-11-11 2020-02-25 重庆远通电子技术开发有限公司 Leakage identification method, device, equipment and storage medium based on neural network
CN111237646A (en) * 2020-02-17 2020-06-05 清华大学 Automatic identification and positioning method for leakage of water supply pipe network
CN111237646B (en) * 2020-02-17 2021-03-02 清华大学 Automatic identification and positioning method for leakage of water supply pipe network
CN112925822A (en) * 2021-02-08 2021-06-08 山东大学 Time series classification method, system, medium and device based on multi-representation learning
CN112836758A (en) * 2021-02-08 2021-05-25 浙江工业大学 Independent metering area pipe network leakage detection method based on double-layer circulating neural network
CN112780953A (en) * 2021-02-08 2021-05-11 浙江工业大学 Independent metering area pipe network leakage detection method based on mode detection
CN112836758B (en) * 2021-02-08 2024-03-01 浙江工业大学 Independent metering area pipe network leakage detection method based on double-layer circulating neural network
CN112780953B (en) * 2021-02-08 2022-07-12 浙江工业大学 Independent metering area pipe network leakage detection method based on mode detection
CN112818495A (en) * 2021-02-22 2021-05-18 成都四为电子信息股份有限公司 Novel dynamic correction method for pipeline pressure drop measurement and calculation algorithm parameters
CN112923989A (en) * 2021-02-24 2021-06-08 中国水利水电科学研究院 Household terminal water consumption monitoring method and intelligent water meter
CN112923989B (en) * 2021-02-24 2022-05-10 中国水利水电科学研究院 Household terminal water consumption monitoring method and intelligent water meter
CN113588179A (en) * 2021-06-24 2021-11-02 武汉众智鸿图科技有限公司 Water supply pipe network leakage detection method and system
CN113588179B (en) * 2021-06-24 2023-11-21 武汉众智鸿图科技有限公司 Water supply network leakage detection method and system
CN113944888B (en) * 2021-11-03 2023-12-08 北京软通智慧科技有限公司 Gas pipeline leakage detection method, device, equipment and storage medium
CN113944888A (en) * 2021-11-03 2022-01-18 北京软通智慧科技有限公司 Gas pipeline leakage detection method, device, equipment and storage medium
CN114352947A (en) * 2021-12-08 2022-04-15 天翼物联科技有限公司 Gas pipeline leakage detection method, system and device and storage medium
CN114352947B (en) * 2021-12-08 2024-03-12 天翼物联科技有限公司 Gas pipeline leakage detection method, system, device and storage medium
CN115580486A (en) * 2022-11-18 2023-01-06 宁波市镇海区大数据投资发展有限公司 Network security sensing method and device based on big data
CN116642138A (en) * 2023-05-25 2023-08-25 大连智水慧成科技有限责任公司 New leakage detection method for water supply network
CN117093854A (en) * 2023-10-19 2023-11-21 安徽建筑大学 Transformer mechanical fault diagnosis method, equipment and storage medium
CN117093854B (en) * 2023-10-19 2024-02-09 安徽建筑大学 Transformer mechanical fault diagnosis method, equipment and storage medium

Also Published As

Publication number Publication date
CN109359698B (en) 2020-07-21

Similar Documents

Publication Publication Date Title
CN109359698B (en) Leakage identification method based on long-time memory neural network model
CN112098714B (en) Electricity stealing detection method and system based on ResNet-LSTM
CN106650797B (en) Power distribution network electricity stealing suspicion user intelligent identification method based on integrated ELM
CN113298297B (en) Wind power output power prediction method based on isolated forest and WGAN network
CN114118640B (en) Long-term precipitation prediction model construction method, long-term precipitation prediction method and device
CN112836758B (en) Independent metering area pipe network leakage detection method based on double-layer circulating neural network
CN116341901B (en) Integrated evaluation method for landslide surface domain-monomer hazard early warning
CN111104736B (en) Abnormal data detection method, device, medium and equipment based on time sequence
CN111898831A (en) Real-time flood probability forecasting practical method
CN116383773A (en) Data center energy efficiency anomaly detection method, system and medium based on self-adaptive prediction interval
CN108510072A (en) A kind of discharge of river monitoring data method of quality control based on chaotic neural network
CN110414715B (en) Community detection-based passenger flow volume early warning method
CN113128098B (en) Concrete dam singular value detection method and equipment based on DE-LOF
CN115470850A (en) Water quality abnormal event recognition early warning method based on pipe network water quality time-space data
CN113988210A (en) Method and device for restoring distorted data of structure monitoring sensor network and storage medium
CN115935139A (en) Space field interpolation method for ocean observation data
CN117312769A (en) BiLSTM-based method for detecting abnormality of time sequence data of Internet of things
CN117791856B (en) Power grid fault early warning method and device based on inspection robot
CN116484219A (en) Water supply network water quality abnormal pollution source identification method based on gate control graph neural network
CN114779001A (en) Cable grounding circulation abnormality detection method and device, electronic device and storage medium
CN113011657A (en) Method and device for predicting typhoon water level, electronic equipment and storage medium
CN118444084B (en) Low-voltage distribution network early warning method, device, equipment and storage medium
CN118194235B (en) Multi-mode fused precipitation dry-wet time sequence change reconstruction method and system
CN116862116B (en) Intelligent early warning method and system for preventing electricity larceny, electronic equipment and storage medium
CN115964939A (en) Flood peak flow calculation method and system based on numerical simulation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant