CN114444290A - Method and system for automatically generating pressure and flow monitoring threshold of water supply system - Google Patents
Method and system for automatically generating pressure and flow monitoring threshold of water supply system Download PDFInfo
- Publication number
- CN114444290A CN114444290A CN202210064920.4A CN202210064920A CN114444290A CN 114444290 A CN114444290 A CN 114444290A CN 202210064920 A CN202210064920 A CN 202210064920A CN 114444290 A CN114444290 A CN 114444290A
- Authority
- CN
- China
- Prior art keywords
- data
- threshold
- value
- moment
- water supply
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 37
- 238000012544 monitoring process Methods 0.000 title claims abstract description 31
- 230000002159 abnormal effect Effects 0.000 claims abstract description 40
- 238000007619 statistical method Methods 0.000 claims abstract description 19
- 230000000737 periodic effect Effects 0.000 claims abstract description 12
- 238000007689 inspection Methods 0.000 claims abstract description 11
- 238000005457 optimization Methods 0.000 claims description 20
- 238000007781 pre-processing Methods 0.000 claims description 10
- 238000012545 processing Methods 0.000 claims description 10
- 238000013499 data model Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 8
- 230000008569 process Effects 0.000 claims description 7
- 238000009499 grossing Methods 0.000 claims description 6
- 239000000126 substance Substances 0.000 claims description 6
- 238000004880 explosion Methods 0.000 claims description 5
- 238000005096 rolling process Methods 0.000 claims description 5
- 230000003442 weekly effect Effects 0.000 claims description 5
- 230000002776 aggregation Effects 0.000 claims description 3
- 238000004220 aggregation Methods 0.000 claims description 3
- 238000009826 distribution Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 2
- 239000013589 supplement Substances 0.000 abstract description 2
- 230000002354 daily effect Effects 0.000 description 4
- 230000008030 elimination Effects 0.000 description 3
- 238000003379 elimination reaction Methods 0.000 description 3
- 230000004931 aggregating effect Effects 0.000 description 2
- 230000003203 everyday effect Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000009172 bursting Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013524 data verification Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 239000000945 filler Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q50/00—Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
- G06Q50/06—Energy or water supply
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Business, Economics & Management (AREA)
- Health & Medical Sciences (AREA)
- Economics (AREA)
- General Physics & Mathematics (AREA)
- Water Supply & Treatment (AREA)
- General Health & Medical Sciences (AREA)
- Primary Health Care (AREA)
- Strategic Management (AREA)
- Tourism & Hospitality (AREA)
- Human Resources & Organizations (AREA)
- General Business, Economics & Management (AREA)
- Marketing (AREA)
- Public Health (AREA)
- Computer Hardware Design (AREA)
- Evolutionary Computation (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The invention provides a method and a system for automatically generating a pressure and flow monitoring threshold of a water supply system, which select the historical data of a recent period of time for statistical analysis aiming at each sensor; removing abnormal values from the data; carrying out periodicity inspection on the data processed by the S2, and distinguishing the periodicity; selecting a strong periodic sensor, and checking data at each moment until a threshold generation condition is met; carrying out statistical analysis on the data by using a statistical 3 sigma rule, and optimizing a threshold value; for other periodic sensors, generating a constant threshold; repeating the above contents on the latest historical data in a day period to generate the latest threshold. The method for generating the threshold fully considers the disturbance caused by abnormal data, filters the input data by using three eliminating methods, smoothes the data at each moment, and simultaneously checks and supplements the data quantity at each moment in order to ensure the reliability of the input data.
Description
Technical Field
The invention belongs to the technical field of setting of pressure and flow thresholds of water supply networks, and particularly relates to a method and a system for automatically generating a pressure and flow monitoring threshold of a water supply system.
Background
At the present stage, the water department usually adopts the following mode for judging the working condition of a water supply network, firstly, an administrator sets a corresponding alarm threshold value for each measuring point through monitoring center software according to the peripheral water supply condition, when the pressure or the flow rate is out of limit, monitoring equipment uploads alarm information to a monitoring center, a water supply department timely processes the alarm information according to the pressure or the flow rate, the water supply pressure is ensured to be normal, and pipe explosion is prevented. And secondly, the customer service center feeds back to the emergency repair center according to the leakage and the pipe explosion reflected by the masses, and the emergency repair center checks the working condition nearby and handles the abnormity. The first mode is that a manager sets a threshold value according to experience, the number of monitoring points is more and more along with the expansion of the scale of a pipe network, the threshold values of flow and pressure of the monitoring points are set one by one, great workload is brought to an operator, and the accuracy and timeliness of data are difficult to guarantee. The second mode has strong uncertainty and time delay through the loss and tube bursting of crowd feedback.
For the unusual incident in more accurate efficient detection and the location water supply network, guarantee water supply network system quality, reduce manual operation or empirical judgement, need urgently to establish suitable data model to the sensor, the automatic threshold value that generates pipe network flow pressure in real time according to SCADA historical data, monitor the dynamic change of pipe network data at any time, in time discover the unusual operating mode of water supply network and pinpoint relevant pipeline address and salvage and maintain, effectual improvement water resource utilization ratio, energy saving, ensure the city normal water consumption. Accordingly, a method and system for automatically generating a supply network pressure flow threshold is presented herein.
Disclosure of Invention
In view of the above, the present invention is directed to a method and a system for automatically generating a pressure and flow monitoring threshold of a water supply system, so as to solve the problem of more accurately and efficiently detecting and locating an abnormal event in a water supply network and ensuring the quality of the water supply network system.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
a method of automatically generating a water supply system pressure flow monitoring threshold, comprising the steps of:
s1, selecting historical data of each sensor in a certain period before the latest date, and performing statistical analysis, wherein the sensors comprise pressure sensors or flow sensors;
s2, carrying out periodic inspection on the selected historical data, and distinguishing strong periodicity, weak periodicity and no periodicity;
s3, selecting strong periodic sensor history data, smoothing the data at each moment to form data at one moment based on the similarity and continuity of the history data, performing aggregation statistics on the average value and the quantity of the history data at each moment, verifying the quantity, and enabling the data to meet threshold generation conditions after verification is completed;
s4, performing statistical analysis on the historical data at different moments in the step S3 to generate upper limit threshold values and lower limit threshold values at different moments;
s5, adjusting the weekly threshold by using the weekly weight coefficient, and optimizing the adjusted threshold according to the fluctuation ranges of different sensors;
s6, for the sensor with no periodicity or weak periodicity, counting the interval in which data with a certain numerical value proportion falls according to historical data of a set time, acquiring the upper limit and the lower limit of the interval, and generating a constant threshold;
s7, updating the historical data, repeating the steps S1 to S6, and optimizing the generated threshold.
Further, the process of performing statistical analysis on the historical data in step S1 includes the following steps:
s11, filtering the selected historical data according to an empirical extreme value, and selecting data in the range of the empirical extreme value;
the empirical extremum ranges are as follows: vmin＜V＜VmaxIn which V isminIs an empirical minimum, VmaxIs an empirical maximum;
s12, filtering the data filtered by the empirical value by a quartile algorithm, generating threshold ranges at different moments, determining the data with the relative error between the threshold range and a normal value exceeding 80% as abnormal data, and removing the abnormal data;
the formula of the quartile algorithm is as follows:
QUPPER＝Q3+k(Q3-Q1)；
QLOWER＝Q1-k(Q3-Q1)；
wherein Q3 represents the upper quartile, Q1 represents the lower quartile, k represents the coefficient, and QUPPER is the upper threshold; QLOWER is the lower threshold;
s13, removing abnormal values according to historical working conditions: and if pipe explosion or other abnormal working condition records occur in the historical data time period, rejecting the data of the abnormal working condition.
Further, in step S2, the data from which the abnormal values are removed is periodically inspected by using an ADF inspection method; and judging whether a unit root exists according to the ADF result, wherein if the unit root does not exist, the unit root is strong periodicity, and if the unit root exists, the unit root is weak periodicity or no periodicity.
Further, each time represents 15 minutes in step S3;
the process of checking the amount of the historical data is as follows: obtaining a threshold generation condition which needs to be met by the data at each moment according to the data acquired in the step S1 within a certain period;
because the original data may have data missing or undergo processing of eliminating abnormal values, the number of data at a certain time is smaller than the set minimum value, and the data at the time needs to be interpolated: the interpolation is replaced by the data of the previous moment and the next moment or the mean value of the moment; or expanding the range of the historical data to enable the number of the data at the moment to meet the threshold generation condition.
Further, in step S4, statistical analysis is performed on the data at different times by using a statistical 3 σ rule to generate upper and lower threshold values at different times, which includes the following specific steps:
through a large amount of statistics and analysis, the flow and pressure data at each moment basically present normal distribution, statistical analysis is carried out on the data by using a statistics 3 sigma rule, and a normal variation range in a time period corresponding to the measuring point is generated;
the 3 σ rule formula is as follows:
wherein the content of the first and second substances,
is the mean value of the flow or pressure of the sensor at time j within n days, sigma is the standard deviation of the sensor at time j within n days, UPPER is the UPPER threshold limit at time j calculated by the 3 sigma rule, and LOWER is the LOWER threshold limit at time j calculated by the 3 sigma rule.
Further, the step of optimizing the threshold value according to the weight and the fluctuation range in step S5 includes the following steps:
s51, setting importance weight of data every week, and setting an upper limit threshold and a lower limit threshold according to the weight:
wherein, ω isiIs the weight of week i, uiAt the upper limit of week i,/iThe lower limit of the ith week is,
to optimize the upper threshold according to the weight,
the lower threshold value is optimized according to the weight;
s52, optimizing and adjusting the upper and lower limits of the threshold value according to different fluctuation ranges:
wherein the content of the first and second substances,
for the upper threshold limit at a certain time after optimization according to the week weight,
for the lower threshold at a certain time after optimization according to the week weight,
for the upper threshold adjusted according to the fluctuation range,
in order to adjust the lower threshold according to the fluctuation range, Δ u is the upper offset, the offset value is a fixed value or a percentage value of the upper limit, and Δ l is the lower offsetThe offset value is a fixed value, or a percentage value of the lower limit.
A system for automatically generating a water supply system pressure flow monitoring threshold, comprising: the device comprises a data input module, a data preprocessing module, a threshold generating module, a threshold optimizing module and a data output module;
the data input module is used for acquiring the historical data of the latest specified week of all the sensors as the input of the data model;
the data preprocessing module is used for removing abnormal values of input data, reducing dimensions of the data and verifying the data;
the threshold generating module is used for generating thresholds at different moments for each monitoring point in the data preprocessing module;
the threshold optimization module is used for adjusting and optimizing the threshold in the threshold generation module;
and the data output module outputs the threshold values of all the sensors in the threshold optimization module at multiple moments.
Further, the adjusting and optimizing modes of the threshold optimizing module comprise weight optimization, fluctuation adjustment and daily rolling update.
Compared with the prior art, the method and the system for automatically generating the pressure and flow monitoring threshold of the water supply system have the following beneficial effects:
(1) the invention relates to a method and a system for automatically generating a pressure and flow monitoring threshold of a water supply system, which fully consider the disturbance caused by abnormal data, filter the input data by methods such as an empirical extreme value method, a quartile method, abnormal working condition elimination and the like, smooth the data at each moment, and simultaneously check and supplement the data quantity at each moment in order to ensure the reliability of the input data.
(2) According to the method and the system for automatically generating the water supply system pressure flow monitoring threshold, threshold generation modes of different waveforms are fully considered, and strong periodicity, weak periodicity or aperiodic thresholds are respectively processed differently, so that the threshold generation mode is suitable for various sensor types.
(3) According to the method and the system for automatically generating the water supply system pressure flow monitoring threshold, the influence of the distance of data generation time on the accuracy of the threshold is considered, the week weight setting and the mode of updating the threshold by day rolling are used, and the accuracy of the threshold is further improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate an embodiment of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:
fig. 1 is a block diagram of a system data model structure for automatically generating a pressure and flow monitoring threshold of a water supply system according to an embodiment of the present invention;
fig. 2 is a data processing flow chart of a method for automatically generating a monitoring threshold of pressure and flow of a water supply system according to an embodiment of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are used only for convenience in describing the present invention and for simplicity in description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the present invention. Furthermore, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first," "second," etc. may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless otherwise specified.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art through specific situations.
The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
As shown in fig. 2, a method for automatically generating a monitoring threshold for pressure and flow in a water supply system includes the following steps:
s1, selecting historical data of a selected period (N weeks) set before the latest date of each sensor for statistical analysis, wherein each sensor is a pressure sensor or a flow sensor;
s2, removing abnormal values from the historical data: filtering abnormal values by an experience extreme method, removing discrete values at different moments by a quartile method, and deleting data in an abnormal working condition time period;
s3, carrying out periodic inspection on the data processed by the S2 by adopting an ADF (automatic dicky-filler test) inspection method, and judging whether a unit root exists according to the ADF result, wherein the unit root is strong periodicity if the unit root does not exist, and the unit root is weak periodicity or non-periodicity if the unit root exists;
s4, taking strong periodic sensor history data, smoothing the data at each moment to form data at one moment based on the similarity and continuity of the history data, aggregating and counting the average value and the quantity of the history data at each moment, checking the quantity, and enabling the data to meet threshold generation conditions after checking is finished;
each time is 15 minutes, smoothing processing is performed on data of every 15 minutes to obtain data of one time, that is, data of every day of each sensor is 96 (24 hours × 4 times), the number of data of each time is aggregated and counted, because raw data may have data missing or is processed by S2, the number of data of a certain time is smaller than a set minimum value, interpolation processing (interpolation may be replaced by data of previous and subsequent times or by an average value of the time) or expanding a historical data range (for example, obtaining historical data of N +1, N +2 weeks) needs to be performed on the data of the time, so that the number of data of the time meets a threshold generation condition;
s5, performing statistical analysis on the data of each week by using a statistical 3 sigma rule according to the data of different moments generated in S4 to generate an upper limit and a lower limit of a threshold value of each week at different moments;
s6, adjusting the threshold value according to the week weight coefficient for the weekly threshold value data generated in the step S5; optimizing the threshold value according to the fluctuation range of different sensors aiming at the adjusted threshold value;
s7, for a sensor with no periodicity or weak periodicity, counting an interval in which data with a certain numerical value proportion falls according to long-time historical data, acquiring an upper limit and a lower limit of the interval, and generating a constant threshold;
s8, relearning the latest history data in the above-described procedure at a daily cycle, and generating the latest threshold value.
The step S2 of removing abnormal values from the selected historical data includes the following steps:
s21, filtering all data according to an empirical extreme value, and selecting data in the range of the empirical extreme value;
the empirical extremum ranges are as follows: vmin＜V＜VmaxIn which V isminIs an empirical minimum, VmaxIs an empirical maximum;
s22, filtering the data filtered by the empirical value by a quartile algorithm, generating threshold ranges at different moments, determining the data with the relative error between the threshold range and a normal value exceeding 80% as abnormal data, and removing the abnormal data;
the formula of the quartile algorithm is as follows:
QUPPER＝Q3+k(Q3-Q1)；
QLOWER＝Q1-k(Q3-Q1)；
wherein Q3 represents the upper quartile, Q1 represents the lower quartile, k represents the coefficient, and QUPPER is the upper threshold; QLOWER is the lower threshold;
s23, removing abnormal values according to historical working conditions: and if pipe explosion or other abnormal working condition records occur in the historical data time period, rejecting the data of the abnormal working condition.
In step S3, performing periodic inspection on the data from which the abnormal values are removed by using an ADF inspection method; and judging whether a unit root exists according to the ADF result, wherein the unit root is strong periodicity if the unit root does not exist, and the unit root is weak periodicity or non-periodicity if the unit root exists.
The process of checking the amount of the historical data is as follows: obtaining a threshold generation condition to be met by data at each moment according to the set period in the step S1;
because the original data may have data missing or undergo processing of removing abnormal values, the number of data at a certain time is smaller than the set minimum value, and the data at the time needs to be interpolated: the interpolation is replaced by the data of the previous moment and the next moment or the mean value of the moment; or expanding the range of the historical data to enable the number of the data at the moment to meet the threshold generation condition.
In step S5, statistical analysis is performed using the data at different times according to the statistical 3 σ rule, and the specific process of generating the upper limit and the lower limit of the threshold at different times is as follows:
through a large amount of statistics and analysis, the flow and pressure data at each moment basically present normal distribution, statistical analysis is carried out on the data by using a statistics 3 sigma rule, and a normal variation range in a time period corresponding to the measuring point is generated;
the 3 σ rule formula is as follows:
wherein the content of the first and second substances,
is the mean value of the flow or pressure of the sensor at time j within n days, sigma is the standard deviation of the sensor at time j within n days, UPPER is the UPPER threshold limit calculated by the 3 sigma rule, and LOWER is the LOWER threshold limit calculated by the 3 sigma rule.
The optimization of the threshold value according to the weight and the fluctuation range in step S6 includes the following steps:
s61, setting importance weight of data per week, and setting an upper limit threshold and a lower limit threshold according to the weight:
wherein, ω isiIs the weight of week i, uiAt the upper limit of week i,/iThe lower limit of the ith week is,
to optimize the upper threshold according to the weight,
the lower threshold value is optimized according to the weight;
s62, in order to reduce invalid alarms and improve the reliability of the threshold, the upper and lower limits of the threshold are adjusted according to different fluctuation ranges:
wherein the content of the first and second substances,
for the upper threshold limit at a certain time after optimization according to the week weight,
for the lower threshold at a certain time after optimization according to the week weight,
to adjust the upper threshold according to the fluctuation range,
in order to adjust the lower threshold value according to the fluctuation range, Δ u is an upper limit offset, the offset value may be a fixed value or a percentage value of the upper limit, Δ l is a lower limit offset, and the offset value may be a fixed value or a percentage value of the lower limit.
As shown in fig. 1, a system for automatically generating a water supply system pressure flow monitoring threshold includes: the device comprises a data input module, a data preprocessing module, a threshold generating module, a threshold optimizing module and a data output module;
the data input module is used for acquiring the historical data of the latest specified week of all the sensors as the input of the data model;
the data preprocessing module is used for removing abnormal values of input data, reducing dimensions of the data and verifying the data;
the threshold generating module is used for generating thresholds at different moments for each monitoring point in the data preprocessing module;
the threshold optimization module is used for adjusting and optimizing the threshold in the threshold generation module;
and the data output module outputs the threshold values of all the sensors in the threshold optimization module at multiple moments.
Specifically, the data input module acquires the latest N-week historical data of all sensors as the input of the data model, and the N value can be adjusted, preferably the minimum is not less than 1, and preferably the maximum is not more than 5.
The data preprocessing module comprises three modules of abnormal value elimination, data dimension reduction and data verification. And eliminating abnormal values by respectively using an empirical extreme value method, a quartile method and an abnormal working condition elimination method to eliminate abnormal data. And (3) reducing the dimension of the data, wherein the influence of disturbance at partial moments is eliminated by considering the continuity and the similarity of historical data, the data is averaged every 15 minutes, the data is smoothed, and the dimension of the data of 60 minutes is reduced into 4 moments. And data checking, namely checking whether the number of the input data meets the generation condition of a threshold value, and performing expansion and filling processing.
The threshold generation module is used for generating thresholds at different moments for each monitoring point, and more specifically, the threshold generation module is used for periodically detecting a waveform, distinguishing strong periodicity, weak periodicity or no periodicity, a strong periodic data model carries out statistical analysis on data by adopting a 3 sigma rule to generate normal variation ranges of the different moments of the measuring points, the weak periodicity or no periodicity counts an interval in which data with a certain numerical value proportion falls according to long-time historical data, obtains an upper limit and a lower limit of the interval, and generates a constant threshold.
And the threshold optimization module is used for optimizing the output result of the data model, and the optimization mode comprises weight optimization, fluctuation adjustment and daily rolling update. Specifically, weight optimization is performed, wherein the weight of the importance of data every week is given, and the output result of the model is optimized according to the weight. The fluctuation adjustment is specifically performed by floating the threshold value by a certain ratio according to the fluctuation range of different sensors, and in principle, the larger the fluctuation range is, the smaller the fluctuation range is, and the larger the fluctuation range is. And (4) updating in a rolling manner every day, and relearning the latest historical data according to the steps by taking a day as a period to generate the latest threshold.
And the data output module generates 96 time thresholds of all the sensors through the steps.
As shown in fig. 2, the specific embodiment includes the following:
s1, selecting historical data of each sensor in a certain period before the latest date, and performing statistical analysis, wherein the sensors comprise pressure sensors or flow sensors;
s2, removing abnormal values from the data, filtering the abnormal values from all the data according to an empirical extreme method, removing discrete values at a certain moment by a quartile method, and deleting the data in the abnormal working condition time period;
s3, carrying out periodicity inspection on the data processed by the S2 to distinguish strong periodicity, weak periodicity or no periodicity;
s4, selecting a strong periodic sensor, smoothing the data every 15 minutes based on historical data similarity and continuity, wherein the data before processing is shown in Table 1, and the result after processing is shown in Table 2. Aggregating and counting the average value and the number of the historical data at each moment, and for the sensors with insufficient number, filling the data or expanding the range of the historical data until the threshold generation condition is met;
as shown in table 3, 3 weeks of history data are selected, and after aggregation statistics, HHMM is 14:45, the COUNT value of the data at this time is 3, which is far lower than the normal COUNT of 21, where the COUNT number is set to be not lower than 80% of the normal COUNT. Therefore, it is necessary to perform processing for filling up the data at that time or expanding the range of the history data.
Table 1: data before smoothing
| SN | COLLECTTIME | VALUE |
| HD72_1016_SS | 2021/11/29 13:16 | -12 |
| HD72_1016_SS | 2021/11/29 13:17 | -12 |
| HD72_1016_SS | 2021/11/29 13:18 | -12 |
| HD72_1016_SS | 2021/11/29 13:19 | -12 |
| HD72_1016_SS | 2021/11/29 13:20 | -10 |
| HD72_1016_SS | 2021/11/29 13:21 | -12 |
| HD72_1016_SS | 2021/11/29 13:22 | -12 |
| HD72_1016_SS | 2021/11/29 13:23 | -12 |
| HD72_1016_SS | 2021/11/29 13:24 | -11 |
| HD72_1016_SS | 2021/11/29 13:25 | -10 |
| HD72_1016_SS | 2021/11/29 13:26 | -12 |
| HD72_1016_SS | 2021/11/29 13:27 | -11 |
| HD72_1016_SS | 2021/11/29 13:28 | -10 |
| HD72_1016_SS | 2021/11/29 13:29 | -10 |
| HD72_1016_SS | 2021/11/29 13:30 | -10 |
Table 2: smoothed data
| SN | COLLECTTIME | VALUE |
| HD72_1016_SS | 2021/11/29 13:30 | -11.2 |
Table 3: aggregate statistics on data
| SN | HHMM | HM_NUM | COUNT | MEAN |
| HD72_1016_SS | 13:30 | 1330 | 21 | -12 |
| HD72_1016_SS | 13:45 | 1345 | 21 | -11.7143 |
| HD72_1016_SS | 14:00 | 1400 | 21 | -11.3333 |
| HD72_1016_SS | 14:15 | 1415 | 21 | -11.2381 |
| HD72_1016_SS | 14:30 | 1430 | 21 | -10.9524 |
| HD72_1016_SS | 14:45 | 1445 | 3 | -10.0619 |
| HD72_1016_SS | 15:00 | 1500 | 21 | -10.8571 |
| HD72_1016_SS | 15:15 | 1515 | 21 | -10.619 |
| HD72_1016_SS | 15:30 | 1530 | 21 | -10.8571 |
| HD72_1016_SS | 15:45 | 1545 | 21 | -11.0952 |
| HD72_1016_SS | 16:00 | 1600 | 21 | -11.1429 |
| HD72_1016_SS | 16:15 | 1615 | 21 | -11.8095 |
| HD72_1016_SS | 16:30 | 1630 | 21 | -12.381 |
S5, carrying out statistical analysis on the data by using a statistical 3 sigma rule to generate a normal variation range in a time period corresponding to the measuring point;
table 4: 3 sigma rule Generation threshold
| SN | HHMM | MEAN | STD | LOWER | UPPER |
| HD72_1016_SS | 13:30 | -12 | 1.0488 | -15.1464 | -8.8536 |
| HD72_1016_SS | 13:45 | -11.7143 | 1.0071 | -14.7356 | -8.693 |
| HD72_1016_SS | 14:00 | -11.3333 | 0.9661 | -14.2316 | -8.435 |
| HD72_1016_SS | 14:15 | -11.2381 | 1.0443 | -14.371 | -8.1052 |
| HD72_1016_SS | 14:30 | -10.9524 | 1.0713 | -14.1663 | -7.7385 |
| HD72_1016_SS | 14:45 | -10.7619 | 1.0443 | -13.8948 | -7.629 |
| HD72_1016_SS | 15:00 | -10.8571 | 0.9636 | -13.7479 | -7.9663 |
| HD72_1016_SS | 15:15 | -10.619 | 0.9735 | -13.5395 | -7.6985 |
| HD72_1016_SS | 15:30 | -10.8571 | 1.1952 | -14.4427 | -7.2715 |
| HD72_1016_SS | 15:45 | -11.0952 | 1.2209 | -14.7579 | -7.4325 |
| HD72_1016_SS | 16:00 | -11.1429 | 1.4243 | -15.4158 | -6.87 |
| HD72_1016_SS | 16:15 | -11.8095 | 1.569 | -16.5165 | -7.1025 |
| HD72_1016_SS | 16:30 | -12.381 | 1.322 | -16.347 | -8.415 |
S6, optimizing the threshold value according to the weight of each week and the fluctuation range of different sensors, and the result is shown in Table 5;
table 5: optimizing the threshold value according to the week weight and fluctuation range
| SN | HHMM | MEAN | STD | LOWER | UPPER |
| HD72_1016_SS | 13:30 | -12 | 1.0488 | -19.0845 | 0 |
| HD72_1016_SS | 13:45 | -11.7143 | 1.0071 | -18.5668 | 0 |
| HD72_1016_SS | 14:00 | -11.3333 | 0.9661 | -17.9318 | 0 |
| HD72_1016_SS | 14:15 | -11.2381 | 1.0443 | -18.1075 | 0 |
| HD72_1016_SS | 14:30 | -10.9524 | 1.0713 | -17.8496 | 0 |
| HD72_1016_SS | 14:45 | -10.7619 | 1.0443 | -17.5075 | 0 |
| HD72_1016_SS | 15:00 | -10.8571 | 0.9636 | -17.3224 | 0 |
| HD72_1016_SS | 15:15 | -10.619 | 0.9735 | -17.0598 | 0 |
| HD72_1016_SS | 15:30 | -10.8571 | 1.1952 | -18.1978 | 0 |
| HD72_1016_SS | 15:45 | -11.0952 | 1.2209 | -18.595 | 0 |
| HD72_1016_SS | 16:00 | -11.1429 | 1.4243 | -19.424 | 0 |
| HD72_1016_SS | 16:15 | -11.8095 | 1.569 | -20.8108 | 0 |
| HD72_1016_SS | 16:30 | -12.381 | 1.322 | -20.5972 | 0 |
S7, for the sensor with no periodicity or weak periodicity, counting the interval in which the data with a certain numerical value proportion fall according to the historical data for a long time, acquiring the upper limit and the lower limit of the interval, and generating a constant threshold value, as shown in Table 6;
table 6: generating constant thresholds for sensors with no or weak periodicity
| SN | HHMM | MAX | MIN | LOWER | UPPER |
| HD72_3243_SS | 3:45 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 4:00 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 4:15 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 4:30 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 4:45 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 5:00 | 0 | 0 | -5 | 10.5 |
| HD72-3243_SS | 5:15 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 5:30 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 5:45 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 6:00 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 6:15 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 6:30 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 6:45 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 7:00 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 7:15 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 7:30 | 0 | 0 | -5 | 10.5 |
| HD72_3243_SS | 7:45 | 0 | 0 | -5 | 10.5 |
S8, relearning the latest history data in the above-described procedure at a daily cycle, and generating the latest threshold value.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.
Claims (8)
1. A method of automatically generating a water supply system pressure flow monitoring threshold, comprising the steps of:
s1, selecting historical data of each sensor in a certain period before the latest date, and performing statistical analysis, wherein the sensors comprise pressure sensors or flow sensors;
s2, carrying out periodic inspection on the selected historical data, and distinguishing strong periodicity, weak periodicity and no periodicity;
s3, selecting strong periodic sensor history data, smoothing the data at each moment to form data at one moment based on the similarity and continuity of the history data, performing aggregation statistics on the average value and the quantity of the history data at each moment, verifying the quantity, and enabling the data to meet threshold generation conditions after verification is completed;
s4, performing statistical analysis on the historical data at different moments in the step S3 to generate upper limit threshold values and lower limit threshold values at different moments;
s5, adjusting the weekly threshold by using the weekly weight coefficient, and optimizing the adjusted threshold according to the fluctuation ranges of different sensors;
s6, for the sensor with no periodicity or weak periodicity, counting the interval in which data with a certain numerical value proportion falls according to historical data of a set time, acquiring the upper limit and the lower limit of the interval, and generating a constant threshold;
s7, updating the historical data, repeating the steps S1 to S6, and optimizing the generated threshold.
2. A method of automatically generating a water supply system pressure flow monitoring threshold as claimed in claim 1, wherein: the statistical analysis process of the history data in step S1 includes the steps of:
s11, filtering the selected historical data according to an empirical extreme value, and selecting data in the range of the empirical extreme value;
the empirical extremum ranges are as follows: vmin＜V＜VmaxIn which V isminIs an empirical minimum, VmaxIs an empirical maximum;
s12, filtering the data filtered by the empirical value by a quartile algorithm, generating threshold ranges at different moments, determining the data with the relative error between the threshold range and a normal value exceeding 80% as abnormal data, and removing the abnormal data;
the formula of the quartile algorithm is as follows:
QUPPER＝Q3+k(Q3-Q1)；
QLOWER＝Q1-k(Q3-Q1)；
wherein Q3 represents the upper quartile, Q1 represents the lower quartile, k represents the coefficient, and QUPPER is the upper threshold; QLOWER is the lower threshold;
s13, removing abnormal values according to historical working conditions: and if pipe explosion or other abnormal working condition records occur in the historical data time period, rejecting the data of the abnormal working condition.
3. A method of automatically generating a water supply system pressure flow monitoring threshold as claimed in claim 1, wherein: in step S2, performing periodic inspection on the data from which the abnormal values are removed by using an ADF inspection method; and judging whether a unit root exists according to the ADF result, wherein if the unit root does not exist, the unit root is strong periodicity, and if the unit root exists, the unit root is weak periodicity or no periodicity.
4. A method of automatically generating a water supply system pressure flow monitoring threshold as claimed in claim 1, wherein: each time represents 15 minutes in step S3;
the process of checking the amount of the historical data is as follows: obtaining a threshold generation condition which needs to be met by data at each moment according to the data acquired in the step S1 within a certain period;
because the original data may have data missing or undergo processing of removing abnormal values, the number of data at a certain time is smaller than the set minimum value, and the data at the time needs to be interpolated: the interpolation is replaced by the data of the previous moment and the next moment or the mean value of the moment; or expanding the historical data range to enable the number of data at the moment to meet the threshold generation condition.
5. A method of automatically generating a water supply system pressure flow monitoring threshold as claimed in claim 1, wherein: in step S4, statistical analysis is performed on the data at different times by using the statistical 3 σ rule to generate upper and lower threshold values at different times, which includes the following specific steps:
through a large amount of statistics and analysis, the flow and pressure data at each moment basically present normal distribution, statistical analysis is carried out on the data by using a statistics 3 sigma rule, and a normal variation range in a time period corresponding to the measuring point is generated;
the 3 σ rule formula is as follows:
wherein the content of the first and second substances,
is the mean value of the flow or pressure of the sensor at the time j within n days, sigma is the standard deviation of the sensor at the time j within n days, UPPER is the UPPER threshold limit of the time j calculated by the 3 sigma rule, and LOWER is the LOWER threshold limit of the time j calculated by the 3 sigma rule.
6. A method of automatically generating a water supply system pressure flow monitoring threshold as claimed in claim 1, wherein: the optimization of the threshold value according to the weight and the fluctuation range in step S5 includes the following steps:
s51, setting importance weight of data every week, and setting an upper limit threshold and a lower limit threshold according to the weight:
wherein, ω isiIs the weight of week i, uiAt the upper limit of week i,/iThe lower limit of the ith week is,
to optimize the upper threshold according to the weight,
the lower threshold value is optimized according to the weight;
s52, optimizing and adjusting the upper and lower limits of the threshold value according to different fluctuation ranges:
wherein the content of the first and second substances,
for the upper threshold limit at a certain time after optimization according to the week weight,
for the lower threshold limit at a certain moment after optimization according to the week weight,
to adjust the upper threshold according to the fluctuation range,
in order to adjust the lower threshold value according to the fluctuation range, Δ u is an upper limit offset, the offset value is a fixed value or a percentage value of the upper limit, Δ l is a lower limit offset, and the offset value is a fixed value or a percentage value of the lower limit.
7. A system for automatically generating a monitoring threshold value of pressure and flow of a water supply system, which is applied to a method for automatically generating a monitoring threshold value of pressure and flow of a water supply system according to any one of claims 1 to 6, and is characterized by comprising the following steps: the device comprises a data input module, a data preprocessing module, a threshold generating module, a threshold optimizing module and a data output module;
the data input module is used for acquiring the historical data of the latest specified week of all the sensors as the input of the data model;
the data preprocessing module is used for removing abnormal values of input data, reducing dimensions of the data and verifying the data;
the threshold generating module is used for generating thresholds at different moments for each monitoring point in the data preprocessing module;
the threshold optimization module is used for adjusting and optimizing the threshold in the threshold generation module;
and the data output module outputs the threshold values of all the sensors in the threshold optimization module at multiple moments.
8. The system for automatically generating a water supply system pressure flow monitoring threshold of claim 7, wherein: the adjusting and optimizing modes of the threshold optimizing module comprise weight optimization, fluctuation adjustment and daily rolling update.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210064920.4A CN114444290A (en) | 2022-01-20 | 2022-01-20 | Method and system for automatically generating pressure and flow monitoring threshold of water supply system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210064920.4A CN114444290A (en) | 2022-01-20 | 2022-01-20 | Method and system for automatically generating pressure and flow monitoring threshold of water supply system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114444290A true CN114444290A (en) | 2022-05-06 |
Family
ID=81368053
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210064920.4A Pending CN114444290A (en) | 2022-01-20 | 2022-01-20 | Method and system for automatically generating pressure and flow monitoring threshold of water supply system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114444290A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115931055A (en) * | 2023-01-06 | 2023-04-07 | 长江信达软件技术（武汉）有限责任公司 | Rural water supply operation diagnosis method and system based on big data analysis |
| CN116206427A (en) * | 2023-05-06 | 2023-06-02 | 安徽智寰科技有限公司 | Hierarchical alarm method based on universal index self-adaptive threshold |
| CN117520996A (en) * | 2024-01-04 | 2024-02-06 | 长沙小枰建材有限公司 | Real-time pressure testing method for culvert pipe |
| CN117826112A (en) * | 2024-03-05 | 2024-04-05 | 天津智云水务科技有限公司 | Soil water content inversion method based on sar |
-
2022
- 2022-01-20 CN CN202210064920.4A patent/CN114444290A/en active Pending
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115931055A (en) * | 2023-01-06 | 2023-04-07 | 长江信达软件技术（武汉）有限责任公司 | Rural water supply operation diagnosis method and system based on big data analysis |
| CN116206427A (en) * | 2023-05-06 | 2023-06-02 | 安徽智寰科技有限公司 | Hierarchical alarm method based on universal index self-adaptive threshold |
| CN116206427B (en) * | 2023-05-06 | 2023-06-30 | 安徽智寰科技有限公司 | Hierarchical alarm method based on universal index self-adaptive threshold |
| CN117520996A (en) * | 2024-01-04 | 2024-02-06 | 长沙小枰建材有限公司 | Real-time pressure testing method for culvert pipe |
| CN117520996B (en) * | 2024-01-04 | 2024-04-02 | 长沙小枰建材有限公司 | Real-time pressure testing method for culvert pipe |
| CN117826112A (en) * | 2024-03-05 | 2024-04-05 | 天津智云水务科技有限公司 | Soil water content inversion method based on sar |
| CN117826112B (en) * | 2024-03-05 | 2024-05-31 | 天津智云水务科技有限公司 | Soil water content inversion method based on sar |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114444290A (en) | Method and system for automatically generating pressure and flow monitoring threshold of water supply system | |
| CN111555716B (en) | Method, device, equipment and storage medium for determining working state of photovoltaic array | |
| CN103616878B (en) | Tobacco processing course feeds in raw material loop presence detection system and detection method | |
| CN111259334B (en) | Abnormal water consumption monitoring and early warning method for large users of industrial enterprises | |
| CN112188531B (en) | Abnormality detection method, abnormality detection device, electronic apparatus, and computer storage medium | |
| CN113435725B (en) | Power grid host dynamic threshold setting method based on FARIMA-LSTM prediction | |
| CN107888441A (en) | A kind of network traffics baseline self study adaptive approach | |
| CN113448298B (en) | Data acquisition system for automatic production equipment | |
| CN103198147A (en) | Method for distinguishing and processing abnormal automatized monitoring data | |
| CN116345700B (en) | Energy consumption monitoring method and monitoring system for energy storage power station | |
| CN116976557A (en) | Energy-saving and carbon-reducing park energy control method and system | |
| CN115376296A (en) | Production fault early warning system based on industrial internet | |
| CN116974310A (en) | Concentrated dosing automatic control system based on cloud computing | |
| CN110400001B (en) | High-voltage cable inspection cycle optimization method based on risk assessment | |
| CN106325258B (en) | Relay protection device state evaluation method based on online monitoring information | |
| CN116050716A (en) | Intelligent park management control platform based on Internet | |
| CN116484302A (en) | Enterprise abnormal electricity utilization discriminating algorithm based on electric quantity data analysis | |
| CN114662058A (en) | Wireless station monitoring method and device | |
| CN114049199A (en) | Risk assessment method and system implementation based on consumption credit scene | |
| CN114563709A (en) | Storage battery monitoring system based on cloud computing platform | |
| CN115480997A (en) | Index abnormity warning method and device, electronic equipment and storage medium | |
| CN112432307A (en) | Water pump evaluation method and device, air conditioning system, storage medium and electronic equipment | |
| CN111555917A (en) | Alarm information processing method and device based on cloud platform | |
| CN117196541B (en) | Property management monitoring management system and method | |
| CN117474710B (en) | Hollow glass whole-process informationized management system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |