CN113420816A - Data abnormal value determination method for full-spectrum water quality monitoring equipment - Google Patents
Data abnormal value determination method for full-spectrum water quality monitoring equipment Download PDFInfo
- Publication number
- CN113420816A CN113420816A CN202110705858.8A CN202110705858A CN113420816A CN 113420816 A CN113420816 A CN 113420816A CN 202110705858 A CN202110705858 A CN 202110705858A CN 113420816 A CN113420816 A CN 113420816A
- Authority
- CN
- China
- Prior art keywords
- abnormal
- data
- parameter
- value
- water quality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000002159 abnormal effect Effects 0.000 title claims abstract description 128
- 238000012544 monitoring process Methods 0.000 title claims abstract description 74
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 45
- 238000000034 method Methods 0.000 title claims abstract description 35
- 238000001228 spectrum Methods 0.000 title claims abstract description 34
- 238000011144 upstream manufacturing Methods 0.000 claims abstract description 28
- 238000011835 investigation Methods 0.000 claims abstract description 4
- 230000008859 change Effects 0.000 claims description 33
- 230000005856 abnormality Effects 0.000 claims description 25
- 238000011156 evaluation Methods 0.000 claims description 6
- 238000012806 monitoring device Methods 0.000 claims description 6
- 230000000630 rising effect Effects 0.000 claims description 6
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000001174 ascending effect Effects 0.000 claims description 3
- 238000003908 quality control method Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 description 7
- 238000003911 water pollution Methods 0.000 description 4
- 238000011161 development Methods 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 239000002352 surface water Substances 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000003153 chemical reaction reagent Substances 0.000 description 1
- 238000004883 computer application Methods 0.000 description 1
- 239000003651 drinking water Substances 0.000 description 1
- 235000020188 drinking water Nutrition 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000003344 environmental pollutant Substances 0.000 description 1
- 239000003041 laboratory chemical Substances 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 231100000719 pollutant Toxicity 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 239000010865 sewage Substances 0.000 description 1
- 230000006641 stabilisation Effects 0.000 description 1
- 238000011105 stabilization Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000004083 survival effect Effects 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 238000013024 troubleshooting Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/2433—Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/243—Classification techniques relating to the number of classes
- G06F18/24323—Tree-organised classifiers
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Health & Medical Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The embodiment of the invention provides a method for determining a data abnormal value of full-spectrum water quality monitoring equipment, which comprises the following steps: s1: monitoring abnormal values of point location parameters; s2: obtaining an abnormal parameter s of the point location by combining the abnormal value judgment of the dynamic data; s3: carrying out regional point location abnormal value investigation, and further investigating the abnormal value corresponding to the abnormal parameter S obtained in S2 by using the upstream and downstream relation of the point location; s4: from the data further investigated in S3, an abnormal value is determined. The data abnormal value determination method for the full-spectrum water quality monitoring equipment, provided by the invention, is used for performing quality control on the acquired monitoring data to determine the abnormal value, so that the occurrence of subsequent quality control errors caused by the introduction of abnormal data is effectively reduced, and high-quality and high-reliability monitoring data are provided for the operation of the full-spectrum water quality monitoring equipment.
Description
Technical Field
The invention relates to the field of water quality monitoring, in particular to a method for determining a data abnormal value of full-spectrum water quality monitoring equipment.
Background
Surface water is an important resource for human survival and national sustainable development, and is an important drinking water supply source for people of all countries. Due to the rapid development of economy, some watersheds are suffering from different degrees of water pollution, threatening the living health of human beings and the health of the national ecological environment. Therefore, the water environment problem not only becomes the focus of people's attention, but also becomes one of the hot spots of the current water pollution monitoring and researching work, and the surface water monitoring network in China is built from 1988 and gradually adds an automatic monitoring technology with 1996. However, most of water quality monitoring adopts a laboratory chemical method, so that the measurement period is long, a large amount of chemical reagents are needed, secondary pollution exists, and the requirements on online and real-time water quality monitoring are difficult to meet.
At present, with the increasing severity of water pollution problems and the constant concern of people on environmental problems, water pollution monitoring technology is rapidly developed. Progress in monitoring means has been benefited from the widespread use of new materials and the rapid development of integrated circuits, and the improvement of analytical accuracy has mainly relied on the improvement of complex computational calculation capability and the widespread use of chemometrics. The full spectrum technology-based water quality monitoring method has the advantages of high speed, no secondary pollution, low cost, capability of realizing multi-parameter synchronous online monitoring and the like.
The full spectrum automatic water quality monitoring station is a comprehensive automatic water quality on-line monitoring system which takes an automatic on-line analysis instrument-full spectrum water quality monitoring equipment as a core and utilizes the modern sensor technology, the automatic measurement technology, the automatic control technology, the computer application technology, an outdoor cabinet, relevant special analysis software and a communication network.
In the operation process of the equipment, the monitoring data is the basis for monitoring the mass concentration, and the monitoring data is used as the input of a pollutant mass concentration quality control model and is required to have the requirements of high quality and high reliability. However, due to the existence of faults such as high-density equipment faults, communication faults, system transmission faults and the like and external environment influences such as catkin influences, rainfall influences and the like, data directly acquired through the high-density equipment does not meet the requirement, quality control of the acquired data is required to be performed to improve the quality and reliability of the data, the data quality and data application of the full-spectrum automatic water quality monitoring station can be guaranteed through an abnormal data self-learning method, information behind abnormal data is mined, and the occurrence of subsequent quality control errors caused by the introduction of missing data or abnormal data is effectively reduced.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method for determining abnormal data values for a full-spectrum water quality monitoring device, which at least partially solves the problems in the prior art.
The invention aims to provide a method for determining a data abnormal value of full-spectrum water quality monitoring equipment, which comprises the following steps:
s1: monitoring abnormal values of point location parameters;
s2: obtaining an abnormal parameter s of the point location by combining the abnormal value judgment of the dynamic data;
s3: carrying out regional point location abnormal value investigation, and further investigating the abnormal value corresponding to the abnormal parameter S obtained in S2 by using the upstream and downstream relation of the point location;
s4: from the data further investigated in S3, an abnormal value is determined.
The method for determining the abnormal data value of the full-spectrum water quality monitoring equipment, provided by the invention, has the characteristics that S1 is used for establishing an abnormal data value monitoring model of the water quality parameter by utilizing an isolated forest, and the steps are as follows:
s1.1, generating an isolated forest model;
and S1.2, evaluating new data by using the isolated forest model, and monitoring to obtain a point location parameter abnormal value.
The method for determining the abnormal data value of the full-spectrum water quality monitoring equipment, provided by the invention, is further characterized in that the S1.1 step is as follows:
s1.1.1, randomly selecting w sample points as a sample subset, and placing the sample subset at a root node;
s1.1.2 randomly appointing a dimension, randomly generating a cutting point p in the current node data, wherein the cutting point p is positioned between the maximum value and the minimum value of the appointed dimension in the current node data;
s1.1.3, generating a hyperplane according to the cutting point p, and dividing the data space of the current node into 2 subspaces; placing data smaller than p in the specified dimension at the left child node of the current node, and placing data larger than or equal to p at the right child node of the current node;
s1.1.4 recurse through steps S1.1.2 and S1.1.3 in the child nodes, and new child nodes are constructed until only one of the child nodes that can no longer continue to cut or the child node has reached a defined height;
s1.1.5 loop S1.1.1 through S1.1.4 until T orphan trees iTrees are generated.
The method for determining the abnormal data value of the full-spectrum water quality monitoring equipment, provided by the invention, is also characterized in that S1.2 comprises the following steps: traversing each data point x through each isolated tree iTree, calculating the evaluation height h (x) of the point x in the forest, and normalizing all the evaluation heights, wherein the abnormal value fraction calculation formula is as follows:
wherein:
h (i) is a harmonic number, and when the abnormal value score is greater than 0.7, the abnormal value is determined.
The method for determining the abnormal data value of the full-spectrum water quality monitoring equipment, provided by the invention, is further characterized in that the S2 comprises the following steps:
s2.1: judging the type of the point location parameter abnormal value monitored in the S1;
s2.2: establishing a relation model of the change trend of the abnormal values of the dynamic parameters and the historical monitoring parameters;
s2.3: judging whether the abnormality exists according to the abnormality type and the change trend of the monitoring data obtained in the step S2.1 to obtain an abnormality coefficient S, wherein the judgment result is as follows: if the abnormality type matches the trend of change, the abnormality coefficient is s1, if the abnormality type is opposite to the trend of change, the abnormality coefficient is s 0, and if the trend of change is stable, the abnormality coefficient is s 0.5.
The method for determining the data abnormal value of the full-spectrum water quality monitoring equipment, provided by the invention, has the characteristics that the type judgment in S2.1 is as follows: z-vi-m, where vi is the value of the anomaly parameter and m is the mean of the parameter, a rising anomaly when z >0, and a falling anomaly when z < 0.
The method for determining the abnormal data value of the full-spectrum water quality monitoring equipment, provided by the invention, has the characteristics that the relation model of the abnormal data value of the dynamic parameter and the historical monitoring parameter established in the S2.2 is y ═ f (x), wherein x is dynamic data influencing the monitoring data, y is the parameter change trend, and y comprises rising, falling and stabilization.
The method for determining the abnormal data value of the full-spectrum water quality monitoring equipment, provided by the invention, has the characteristics that S3 rechecks the screened abnormal value by utilizing the upstream and downstream relationship of the monitoring point, and comprises the following steps:
s3.1: judging the upstream and downstream variation trend q, q ═ vi-m, wherein vi is a point parameter, m is the parameter mean value of the point, and if q > p, the variation trend is ascending; if q < -p, the change trend is descending; if-p < q < p, the variation trend is stable, and p is a variation trend threshold;
s3.2: and judging the abnormal coefficient according to the upstream and downstream variation trends.
The method for determining the abnormal data value of the full-spectrum water quality monitoring equipment, provided by the invention, is further characterized in that the judgment standard of S3.2 is as follows:
if the variation trends of the upstream and the downstream are consistent with the abnormal parameter types, the abnormal coefficient is not changed,
if one point in the upstream and downstream variation trend is opposite to the abnormal parameter type and the other point is the same as the abnormal parameter type, the abnormal coefficient s is s +1,
if one point in the change trend of the upstream and the downstream is opposite to the abnormal parameter type, and the change trend of the other point is stable, the abnormal coefficient s is s +1.5,
if one point in the change trend of the upstream and the downstream is the same as the abnormal parameter type, and the change trend of the other point is stable, the abnormal coefficient s is s +0.5,
if the variation trend of the upstream and the downstream is stable, the abnormal coefficient s is s +1.
The method for determining the data abnormal value of the full-spectrum water quality monitoring equipment, provided by the invention, has the characteristic that S4 is abnormal if the abnormal coefficient S is more than 2.
Advantageous effects
The data abnormal value determination method for the full-spectrum water quality monitoring equipment, provided by the invention, is used for performing quality control on the acquired monitoring data to determine the abnormal value, so that the occurrence of subsequent quality control errors caused by the introduction of abnormal data is effectively reduced, and high-quality and high-reliability monitoring data are provided for the operation of the full-spectrum water quality monitoring equipment.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the disclosure, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number of the aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.
In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details.
When abnormal values occur in a high-density monitoring network, data are processed reasonably by adopting an abnormal value eliminating method, and in order to obtain high-quality monitoring data, the embodiment of the invention provides a method for determining the abnormal values of the data of full-spectrum water quality monitoring equipment, which comprises the following steps:
s1: monitoring abnormal values of point location parameters;
s2: obtaining an abnormal parameter s of the point location by combining the abnormal value judgment of the dynamic data;
s3: carrying out regional point location abnormal value investigation, and further investigating the abnormal value corresponding to the abnormal parameter S obtained in S2 by using the upstream and downstream relation of the point location;
s4: from the data further investigated in S3, an abnormal value is determined.
In some embodiments, the step S1 is to establish an abnormal value monitoring model of the water quality parameter by using isolated forests, and includes the following steps:
s1.1, generating an abnormal value monitoring isolated forest model of each water quality parameter;
s1.2, evaluating new data by using the isolated forest model, monitoring to obtain a point location parameter abnormal value,
wherein, the water quality parameters refer to a plurality of parameters such as CODCr, CODMn and the like.
In some embodiments, the S1.1 step is as follows:
s1.1.1, randomly selecting w sample points as a sample subset to be placed in a root node, wherein w is determined according to data volume;
s1.1.2 randomly appointing a dimension, randomly generating a cutting point p in the current node data, wherein the cutting point p is positioned between the maximum value and the minimum value of the appointed dimension in the current node data;
s1.1.3, generating a hyperplane according to the cutting point p, and dividing the data space of the current node into 2 subspaces; placing data smaller than p in the specified dimension at the left child node of the current node, and placing data larger than or equal to p at the right child node of the current node;
s1.1.4 recurse through steps S1.1.2 and S1.1.3 in the child node, building new child nodes until only one data in the child node (no further cutting can be done) or the child node has reached a defined height;
s1.1.5 loop S1.1.1 through S1.1.4 until T orphan trees iTrees are generated.
In some embodiments, S1.2 includes: traversing each data point x through each isolated tree iTree, calculating the evaluation height h (x) of the point x in the forest, and normalizing all the evaluation heights, wherein the abnormal value fraction calculation formula is as follows:
wherein:
h (i) is a harmonic number, and when the abnormal value score is greater than 0.7, the abnormal value is determined.
In some embodiments, the S2 includes:
s2.1: judging the type of the point location parameter abnormal value monitored in the S1;
s2.2: establishing a relation model of the change trend of the abnormal values of the dynamic parameters and the historical monitoring parameters;
s2.3: judging whether the abnormality exists according to the abnormality type and the change trend of the monitoring data obtained in the step S2.1 to obtain an abnormality coefficient S, wherein the judgment result is as follows: if the abnormality type is consistent with the trend of change, the abnormality coefficient is s-1, if the abnormality type is opposite to the trend of change, the abnormality coefficient is s-0, if the trend of change is stable, the abnormality coefficient is s-0.5,
wherein, the dynamic data refers to the data value of the monitoring data which can be influenced by rainfall, sewage discharge, flow, catkin influence and the like.
In some embodiments, the type determination in S2.1 is as follows: z-vi-m, where vi is the value of the anomaly parameter and m is the mean of the parameter, a rising anomaly when z >0, and a falling anomaly when z < 0.
In some embodiments, the relationship model between the dynamic parameter and the abnormal value trend of the historical monitoring parameter established in S2.2 is y ═ f (x), where x is dynamic data affecting the monitoring data, y is a parameter trend, and y includes rising, falling, and steady.
In some embodiments, the step S3 is to perform a second troubleshooting on the screened outliers by using the upstream and downstream relationships of the monitoring points, including:
s3.1: judging the upstream and downstream variation trend q, q ═ vi-m, wherein vi is a point parameter, m is the parameter mean value of the point, and if q > p, the variation trend is ascending; if q < -p, the change trend is descending; if-p < q < p, the variation trend is stable, p is a variation trend threshold, the threshold p is set differently according to the empirical value in consideration of the water quality condition of the point location, and different parameters correspond to different thresholds;
s3.2: and judging the abnormal coefficient according to the upstream and downstream variation trends.
In some embodiments, the criteria for S3.2 are as follows:
if the variation trends of the upstream and the downstream are consistent with the abnormal parameter types, the abnormal coefficient is not changed,
if one point in the upstream and downstream variation trend is opposite to the abnormal parameter type and the other point is the same as the abnormal parameter type, the abnormal coefficient s is s +1,
if one point in the change trend of the upstream and the downstream is opposite to the abnormal parameter type, and the change trend of the other point is stable, the abnormal coefficient s is s +1.5,
if one point in the change trend of the upstream and the downstream is the same as the abnormal parameter type, and the change trend of the other point is stable, the abnormal coefficient s is s +0.5,
if the variation trend of the upstream and the downstream is stable, the abnormal coefficient s is s +1.
In some embodiments, if the abnormality coefficient S > 2 at S4 is determined to be abnormal.
The above description is only for the specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A method for determining abnormal data values of full-spectrum water quality monitoring equipment is characterized by comprising the following steps:
s1: monitoring abnormal values of point location parameters;
s2: obtaining an abnormal parameter s of the point location by combining the abnormal value judgment of the dynamic data;
s3: carrying out regional point location abnormal value investigation, and further investigating the abnormal value corresponding to the abnormal parameter S obtained in S2 by using the upstream and downstream relation of the point location;
s4: from the data further investigated in S3, an abnormal value is determined.
2. The method for determining the abnormal data value for the full-spectrum water quality monitoring equipment as claimed in claim 1, wherein the step S1 is to establish an abnormal value monitoring model of the water quality parameter by using isolated forest, and the steps are as follows:
s1.1, generating an isolated forest model;
and S1.2, evaluating new data by using the isolated forest model, and monitoring to obtain a point location parameter abnormal value.
3. The method for determining the abnormal data value for the full-spectrum water quality monitoring equipment as claimed in claim 2, wherein the step S1.1 is as follows:
s1.1.1, randomly selecting w sample points as a sample subset, and placing the sample subset at a root node;
s1.1.2 randomly appointing a dimension, randomly generating a cutting point p in the current node data, wherein the cutting point p is positioned between the maximum value and the minimum value of the appointed dimension in the current node data;
s1.1.3, generating a hyperplane according to the cutting point p, and dividing the data space of the current node into 2 subspaces; placing data smaller than p in the specified dimension at the left child node of the current node, and placing data larger than or equal to p at the right child node of the current node;
s1.1.4 recurse through steps S1.1.2 and S1.1.3 in the child nodes, and new child nodes are constructed until only one of the child nodes that can no longer continue to cut or the child node has reached a defined height;
s1.1.5 loop S1.1.1 through S1.1.4 until T orphan trees iTrees are generated.
4. The data outlier determination method for the full-spectrum water quality monitoring device as recited in claim 2, wherein the S1.2 comprises: traversing each data point x through each isolated tree iTree, calculating the evaluation height h (x) of the point x in the forest, and normalizing all the evaluation heights, wherein the abnormal value fraction calculation formula is as follows:
wherein:
h (i) is a harmonic number, and when the abnormal value score is greater than 0.7, the abnormal value is determined.
5. The method for determining data outliers for a full-spectrum water quality monitoring device as claimed in claim 1, wherein said S2 comprises:
s2.1: judging the type of the point location parameter abnormal value monitored in the S1;
s2.2: establishing a relation model of the change trend of the abnormal values of the dynamic parameters and the historical monitoring parameters;
s2.3: judging whether the abnormality exists according to the abnormality type and the change trend of the monitoring data obtained in the step S2.1 to obtain an abnormality coefficient S, wherein the judgment result is as follows: if the abnormality type matches the trend of change, the abnormality coefficient is s1, if the abnormality type is opposite to the trend of change, the abnormality coefficient is s 0, and if the trend of change is stable, the abnormality coefficient is s 0.5.
6. The method for determining the abnormal data value for the full-spectrum water quality monitoring equipment as claimed in claim 5, wherein the type judgment in S2.1 is as follows: z-vi-m, where vi is the value of the anomaly parameter and m is the mean of the parameter, a rising anomaly when z >0, and a falling anomaly when z < 0.
7. The method for determining the abnormal data values for the full-spectrum water quality monitoring equipment as claimed in claim 5, wherein the model of the relationship between the dynamic parameter established in S2.2 and the abnormal change trend of the historical monitoring parameter is y ═ f (x), where x is the dynamic data affecting the monitoring data, y is the change trend of the parameter, and y includes rising, falling and stable.
8. The method for determining the abnormal data values for the full-spectrum water quality monitoring device according to claim 1, wherein the step S3 is to utilize the upstream and downstream relationship of the monitoring points to perform a recheck on the abnormal values that have been screened out, and includes:
s3.1: judging the upstream and downstream variation trend q, q ═ vi-m, wherein vi is a point parameter, m is the parameter mean value of the point, and if q > p, the variation trend is ascending; if q < -p, the change trend is descending; if-p < q < p, the variation trend is stable, and p is a variation trend threshold;
s3.2: and judging the abnormal coefficient according to the upstream and downstream variation trends.
9. The method for determining the abnormal data value for the full-spectrum water quality monitoring device according to claim 8, wherein the determination criteria of S3.2 are as follows:
if the variation trends of the upstream and the downstream are consistent with the abnormal parameter types, the abnormal coefficient is not changed,
if one point in the upstream and downstream variation trend is opposite to the abnormal parameter type and the other point is the same as the abnormal parameter type, the abnormal coefficient s is s +1,
if one point in the change trend of the upstream and the downstream is opposite to the abnormal parameter type, and the change trend of the other point is stable, the abnormal coefficient s is s +1.5,
if one point in the change trend of the upstream and the downstream is the same as the abnormal parameter type, and the change trend of the other point is stable, the abnormal coefficient s is s +0.5,
if the variation trend of the upstream and the downstream is stable, the abnormal coefficient s is s +1.
10. The method for determining the abnormal data value for the full-spectrum water quality monitoring device according to claim 1, wherein S4 is determined to be abnormal if the abnormal coefficient S > 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110705858.8A CN113420816A (en) | 2021-06-24 | 2021-06-24 | Data abnormal value determination method for full-spectrum water quality monitoring equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110705858.8A CN113420816A (en) | 2021-06-24 | 2021-06-24 | Data abnormal value determination method for full-spectrum water quality monitoring equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113420816A true CN113420816A (en) | 2021-09-21 |
Family
ID=77717667
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110705858.8A Pending CN113420816A (en) | 2021-06-24 | 2021-06-24 | Data abnormal value determination method for full-spectrum water quality monitoring equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113420816A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105279386A (en) * | 2015-11-16 | 2016-01-27 | 拉扎斯网络科技（上海）有限公司 | Method and device for determining abnormal index data |
| CN106599271A (en) * | 2016-12-22 | 2017-04-26 | 江苏方天电力技术有限公司 | Emission monitoring time series data abnormal value detection method for coal-fired unit |
| CN106874280A (en) * | 2015-12-10 | 2017-06-20 | 博雅网络游戏开发（深圳）有限公司 | The alarm method and device of abnormal data |
| CN108921440A (en) * | 2018-07-11 | 2018-11-30 | 平安科技（深圳）有限公司 | Pollutant method for monitoring abnormality, system, computer equipment and storage medium |
| CN109308306A (en) * | 2018-09-29 | 2019-02-05 | 重庆大学 | A kind of user power utilization anomaly detection method based on isolated forest |
| CN109345137A (en) * | 2018-10-22 | 2019-02-15 | 广东精点数据科技股份有限公司 | A kind of rejecting outliers method based on agriculture big data |
| CN110149258A (en) * | 2019-04-12 | 2019-08-20 | 北京航空航天大学 | A kind of automobile CAN-bus network data method for detecting abnormality based on isolated forest |
| CN111740991A (en) * | 2020-06-19 | 2020-10-02 | 上海仪电（集团）有限公司中央研究院 | Anomaly detection method and system |
| CN112527788A (en) * | 2020-12-17 | 2021-03-19 | 北京中恒博瑞数字电力科技有限公司 | Method and device for detecting and cleaning abnormal value of transformer monitoring data |
| CN113011325A (en) * | 2021-03-18 | 2021-06-22 | 重庆交通大学 | Stacker track damage positioning method based on isolated forest algorithm |
-
2021
- 2021-06-24 CN CN202110705858.8A patent/CN113420816A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105279386A (en) * | 2015-11-16 | 2016-01-27 | 拉扎斯网络科技（上海）有限公司 | Method and device for determining abnormal index data |
| CN106874280A (en) * | 2015-12-10 | 2017-06-20 | 博雅网络游戏开发（深圳）有限公司 | The alarm method and device of abnormal data |
| CN106599271A (en) * | 2016-12-22 | 2017-04-26 | 江苏方天电力技术有限公司 | Emission monitoring time series data abnormal value detection method for coal-fired unit |
| CN108921440A (en) * | 2018-07-11 | 2018-11-30 | 平安科技（深圳）有限公司 | Pollutant method for monitoring abnormality, system, computer equipment and storage medium |
| CN109308306A (en) * | 2018-09-29 | 2019-02-05 | 重庆大学 | A kind of user power utilization anomaly detection method based on isolated forest |
| CN109345137A (en) * | 2018-10-22 | 2019-02-15 | 广东精点数据科技股份有限公司 | A kind of rejecting outliers method based on agriculture big data |
| CN110149258A (en) * | 2019-04-12 | 2019-08-20 | 北京航空航天大学 | A kind of automobile CAN-bus network data method for detecting abnormality based on isolated forest |
| CN111740991A (en) * | 2020-06-19 | 2020-10-02 | 上海仪电（集团）有限公司中央研究院 | Anomaly detection method and system |
| CN112527788A (en) * | 2020-12-17 | 2021-03-19 | 北京中恒博瑞数字电力科技有限公司 | Method and device for detecting and cleaning abnormal value of transformer monitoring data |
| CN113011325A (en) * | 2021-03-18 | 2021-06-22 | 重庆交通大学 | Stacker track damage positioning method based on isolated forest algorithm |
Non-Patent Citations (1)
| Title |
|---|
| 黄永钦: "基于无监督学习的复杂数据异常检测算法研究", 《硕士学位论文》, 1 December 2020 (2020-12-01) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107742053B (en) | Wind turbine generator set abnormity identification method and device | |
| CN105512799B (en) | Power system transient stability evaluation method based on mass online historical data | |
| CN107480698B (en) | Quality control method based on multiple monitoring indexes | |
| CN111898691A (en) | River sudden water pollution early warning tracing method, system, terminal and medium | |
| CN113011481B (en) | Electric energy meter function abnormality assessment method and system based on decision tree algorithm | |
| CN201392343Y (en) | Quality inspection automatic collecting analysis system | |
| CN108319131B (en) | Unit peak regulation capacity evaluation method based on data mining | |
| CN105488592A (en) | Method for predicting generated energy of photovoltaic power station | |
| CN105719094A (en) | State evaluation method of power transmission equipment | |
| CN116614525A (en) | Big data analysis-based land parcel soil environment rapid monitoring system | |
| CN116432123A (en) | Electric energy meter fault early warning method based on CART decision tree algorithm | |
| CN109613372B (en) | Power grid fault diagnosis method based on multi-element power grid database | |
| CN115980531A (en) | GIS switch cabinet quality detection method and system under specific environment | |
| CN113934536A (en) | Data acquisition method facing edge calculation | |
| CN106770861B (en) | The evaluation method of oil-filled transformer on-line monitoring availability of data | |
| CN113420816A (en) | Data abnormal value determination method for full-spectrum water quality monitoring equipment | |
| CN110632191B (en) | Transformer chromatographic peak qualitative method and system based on decision tree algorithm | |
| CN110750760A (en) | Abnormal theoretical line loss detection method based on situation awareness and control chart | |
| CN114971308A (en) | Transformer health condition assessment method based on multi-dimensional data analysis | |
| CN110259435B (en) | Well condition change identification method based on oil pumping unit electrical parameters | |
| CN113986892A (en) | Coal consumption prediction and data cleaning method and system for thermal power generating unit | |
| CN114295162A (en) | Environmental monitoring system based on data acquisition | |
| CN110648484B (en) | Intelligent laser drilling judgment early warning system and judgment method thereof | |
| CN113611514A (en) | Dry-type transformer monitoring and protecting system | |
| CN113052380A (en) | Method, device, equipment and medium for detecting operation and maintenance cleaning effect of photovoltaic power station |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |