CN109165247B - Intelligent pretreatment method for sewage measurement data - Google Patents
Intelligent pretreatment method for sewage measurement data Download PDFInfo
- Publication number
- CN109165247B CN109165247B CN201811162519.4A CN201811162519A CN109165247B CN 109165247 B CN109165247 B CN 109165247B CN 201811162519 A CN201811162519 A CN 201811162519A CN 109165247 B CN109165247 B CN 109165247B
- Authority
- CN
- China
- Prior art keywords
- data
- input characteristic
- sewage
- intelligent
- time interval
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention discloses an intelligent pretreatment method for sewage measurement data. The method comprises the following steps: 1) preprocessing data for predicting or controlling the quality of the sewage based on the time series; 2) determining an intelligent model, input characteristic quantity in the preprocessed data and target characteristic quantity in the preprocessed data; 3) grouping input characteristic quantities in the preprocessed data according to the positions of actual measuring points; 4) traversing each group of input characteristic quantities by taking the previous group of input characteristic quantities as a reference object, wherein the traversal unit is a sampling period, and outputting combined data; 5) performing intelligent model training by using the combined data in a preset proportion; 6) calculating a mean square error value (MSE) of the intelligent model by using the residual data after combination; 7) updating the time interval of each group of input characteristic quantities; 8) if the time interval of each group is less than or equal to the time of the whole sewage treatment process, returning to the step 4); 9) outputting the time interval with the minimum mean square error value in the traversal process to obtain the time difference between each input characteristic quantity; 10) and recombining the data according to the time interval between each input characteristic quantity for intelligent model training.
Description
Technical Field
The invention belongs to the field of sewage data processing, and particularly relates to a method for preprocessing sewage measurement data during modeling by using an artificial intelligence technology.
Background
In China, most municipal sewage treatment plants adopt an activated sludge process to treat sewage. The sewage treatment process of the activated sludge process is a strongly coupled multi-input multi-output dynamic system, has the characteristics of large hysteresis, nonlinearity, time variation, uncertainty and the like, and brings difficulty to modeling.
Traditionally, people can model actual systems by using acquired rich data knowledge, and the modeling method is widely applied to various control fields. However, for complex continuous, time-varying, large-hysteresis, and nonlinear wastewater control systems, the approach of mathematical modeling is severely challenging. In recent years, with the development of artificial intelligence, a black box mechanism model can be established for sewage measurement data by utilizing the technology, the model does not need to clearly understand the mechanism of the microbial reaction process, and only needs to research input and output data in the process, select an effective model and adjust model parameters, so that a time series model of the system can be obtained. However, since the measurement of the sewage data is based on the same time, and the sewage treatment system has time lag, the problem of analyzing the time sequence of the sewage measurement characteristics in the intelligent modeling process becomes a difficult point.
Disclosure of Invention
In order to overcome the defects, the invention aims to provide an intelligent pretreatment method for sewage measurement data. The method can intelligently output the time interval between each input characteristic quantity and the target characteristic quantity, and solves the problem of large time lag of sewage characteristics.
In order to achieve the purpose, the invention provides an intelligent treatment method of sewage measurement data, which comprises the following steps: the following steps:
1) preprocessing data for predicting or controlling the quality of the sewage based on the time series;
2) determining an intelligent model, input characteristic quantity in the preprocessed data and target characteristic quantity in the preprocessed data;
3) grouping input characteristic quantities in the preprocessed data according to the positions of actual measuring points;
4) traversing each group of input characteristic quantities by taking the previous group of input characteristic quantities as a reference object, wherein the traversal unit is a sampling period, and outputting combined data;
5) performing intelligent model training by using the combined data in a preset proportion;
6) calculating a mean square error value (MSE) of the intelligent model by using the residual data after combination;
definition of mean squared error value:
wherein the content of the first and second substances,
-a mean square error value; n issamples-total number of samples;
yi-the ith measurement;
-the ith predictor;
7) updating the time interval of each group of input characteristic quantities;
8) if the time interval of each group is less than or equal to the time of the whole sewage treatment process, returning to the step 4);
9) outputting the time interval with the minimum mean square error value in the traversal process to obtain the time difference between each input characteristic quantity;
10) and recombining the data according to the time interval between each input characteristic quantity for intelligent model training.
Wherein, the step 1) is specifically as follows:
acquiring collected data for predicting or controlling the quality of the sewage based on the time sequence;
missing value filling, 3delta processing and difference processing are carried out on data
The invention can intelligently calculate the time interval between the sewage measurement data and provides high-quality preprocessing data for the training of an intelligent model. A new idea is provided for data processing based on time series, and the complexity of an intelligent model is simplified.
Drawings
FIG. 1 is a flow chart of the intelligent data processing method of the present invention.
Detailed description of the invention
The technical solution of the present invention is further explained below with reference to actual operational data of a certain sewage treatment plant.
S101, acquiring collected data for predicting or controlling the quality of the sewage based on a time sequence;
specifically, the collected data is detection data in a sewage treatment process, and the detection data is data automatically collected and reported every two hours and mainly comprises inflow water flow, inflow water COD, inflow water ammonia nitrogen, dissolved oxygen, sludge activity, water temperature, outflow water flow, outflow water COD and outflow water ammonia nitrogen. Note that all the following data processing is performed on a time basis, and therefore, each acquisition record should include a data acquisition time.
S102, missing value filling, 3delta processing and difference processing are carried out on the data;
specifically, missing value supplement processing is carried out on missing detection data of a sewage treatment plant by using a Lagrange method at present, a 3delta boundary value is used for replacing data beyond a 3delta range, and finally difference processing is carried out on the processed data, namely, the last moment is subtracted from the data at the next moment.
S103, determining an intelligent model, an input characteristic quantity and a target characteristic quantity;
specifically, a BP neural network model is adopted to predict the ammonia nitrogen of the outlet water, namely the target characteristics are the ammonia nitrogen of the outlet water, and the input characteristics comprise inlet water flow, inlet water COD, inlet water ammonia nitrogen, dissolved oxygen, sludge activity and water temperature.
S104, grouping the input characteristic quantity according to the position of an actual measuring point;
specifically, according to the actual measurement point of the data acquisition of the sewage treatment plant, the data are divided into 3 groups, wherein the first group comprises inflow flow, inflow COD (chemical oxygen demand) and inflow ammonia nitrogen, the second group comprises dissolved oxygen, sludge activity and water temperature, and the third group comprises outflow ammonia nitrogen.
S105, traversing each group of input characteristic quantities by taking the previous group of input characteristic quantities as a reference object, wherein the traversing unit is a sampling period, and outputting combined data;
s106, performing intelligent model training by using the combined data with a certain proportion (such as 80% of data);
s107, calculating a mean square error value (MSE) of the intelligent model by using the residual data (such as 20% of data);
definition of mean squared error value:
wherein the content of the first and second substances,
-a mean square error value; n issamples-total number of samples
yi-the ith measurement
- - - -ith prediction value
S108, updating the time interval of each group of input characteristic quantities;
s109, if the time interval of each group is less than or equal to the time of the whole sewage process treatment, returning to the step S105;
specifically, assuming that the whole sewage treatment period of the current sewage treatment plant is 24 hours, the time interval of each group is less than or equal to 24 hours. Traversing the second group of data (dissolved oxygen, sludge activity and water temperature) by taking the first group of data (inflow flow, inflow COD and inflow ammonia nitrogen) as a reference, wherein the traversal range is 0-12, and traversing the third group of data (outflow ammonia nitrogen) by taking the second group of data as a reference in the same way, wherein the traversal range is 0-12. And (4) dividing the Data after each traversal assembly by taking the last 20% of the Data as a verification set { Data _ Test } and taking other Data as a training set { Data _ Train }. And then, carrying out parameter adjustment on the BP model by using a training set { Data _ Train }, predicting by using a verification set { Data _ Test } after the model is finished, and outputting a predicted mean square error value until the traversal is finished.
S110, outputting the time interval with the minimum mean square error value in the traversal process, namely obtaining the time difference between each input characteristic quantity;
and S111, recombining the data according to the time interval between each input characteristic quantity for intelligent model training.
Specifically, the time interval with the minimum mean square error value in the traversal process is found to be [4,2], namely, offset assembly operation needs to be carried out on data when effluent ammonia nitrogen prediction is carried out on a certain sewage treatment plant, and the recombined data is provided for the intelligent model. For example, if the outlet water ammonia nitrogen of 2018-06-0212: 00:00 needs to be predicted, the inlet water flow, inlet water COD and inlet water ammonia nitrogen at 2018-06-0200: 00:00 time, and the dissolved oxygen, sludge activity and water temperature at 2018-06-0208: 00:00 time need to be input.
The above description is only a preferred embodiment of the present invention and should not be considered as limiting the scope of the present invention, and the claimed invention is not limited thereto, and those skilled in the art can easily conceive equivalent variations based on the disclosure of the present invention and the protection scope of the present invention. For example, when dividing the combined data, the division ratio is not limited to 8: 2; the evaluation index is not limited to a mean square error value.
Claims (3)
1. An intelligent pretreatment method for sewage measurement data is characterized by comprising the following steps:
1) preprocessing data for predicting or controlling the quality of the sewage based on the time series;
2) determining an intelligent model, input characteristic quantity in the preprocessed data and target characteristic quantity in the preprocessed data;
3) grouping input characteristic quantities in the preprocessed data according to the positions of actual measuring points;
4) traversing each group of input characteristic quantities by taking the previous group of input characteristic quantities as a reference object, wherein the traversal unit is a sampling period, and outputting combined data;
5) performing intelligent model training by using the combined data in a preset proportion;
6) calculating a mean square error value (MSE) of the intelligent model by using the residual data after combination;
definition of mean squared error value:
wherein the content of the first and second substances,
-a mean square error value; n issamples-total number of samples; y isi-the ith measurement;
7) updating the time interval of each group of input characteristic quantities;
8) if the time interval of each group is less than or equal to the time of the whole sewage treatment process, returning to the step 4);
9) outputting the time interval with the minimum mean square error value in the traversal process to obtain the time difference between each input characteristic quantity;
10) and recombining the data according to the time interval between each input characteristic quantity for intelligent model training.
2. The intelligent pretreatment method of sewage measurement data according to claim 1, wherein the step 1) is specifically:
acquiring collected data for predicting or controlling the quality of the sewage based on the time sequence;
missing value filling and 3delta processing are carried out on the collected data;
and performing differential processing on the processed data.
3. The intelligent pretreatment method of measured sewage data according to claim 2, wherein the collected data is detected data in the sewage treatment process, and the detected data is reported data automatically collected every two hours, including inflow water flow, inflow water COD, inflow water ammonia nitrogen, dissolved oxygen, sludge activity, water temperature, outflow water flow, outflow water COD and outflow water ammonia nitrogen.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811162519.4A CN109165247B (en) | 2018-09-30 | 2018-09-30 | Intelligent pretreatment method for sewage measurement data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811162519.4A CN109165247B (en) | 2018-09-30 | 2018-09-30 | Intelligent pretreatment method for sewage measurement data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109165247A CN109165247A (en) | 2019-01-08 |
| CN109165247B true CN109165247B (en) | 2021-07-23 |
Family
ID=64877250
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811162519.4A Active CN109165247B (en) | 2018-09-30 | 2018-09-30 | Intelligent pretreatment method for sewage measurement data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109165247B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101923083A (en) * | 2009-06-17 | 2010-12-22 | 复旦大学 | Sewage chemical oxygen demand soft measuring method based on support vector machine and neural network |
| CN102616927A (en) * | 2012-03-28 | 2012-08-01 | 中国科学技术大学 | Adjusting method of technological parameters of sewage treatment and device |
| CN103399134A (en) * | 2013-08-20 | 2013-11-20 | 渤海大学 | Sewage COD soft measurement method based on output observer |
| CN103942457A (en) * | 2014-05-09 | 2014-07-23 | 浙江师范大学 | Water quality parameter time series prediction method based on relevance vector machine regression |
| CN107169610A (en) * | 2017-06-02 | 2017-09-15 | 中国农业大学 | Aquaculture dissolved oxygen prediction method and device |
| JP2018018413A (en) * | 2016-07-29 | 2018-02-01 | ペンタフ株式会社 | Sewage unknown water simple evaluation method |
| CN107688871A (en) * | 2017-08-18 | 2018-02-13 | 中国农业大学 | A kind of water quality prediction method and device |
| CN108414016A (en) * | 2018-03-01 | 2018-08-17 | 深圳市晟达机械设计有限公司 | A kind of sewage network monitoring system based on big data technology |
-
2018
- 2018-09-30 CN CN201811162519.4A patent/CN109165247B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101923083A (en) * | 2009-06-17 | 2010-12-22 | 复旦大学 | Sewage chemical oxygen demand soft measuring method based on support vector machine and neural network |
| CN102616927A (en) * | 2012-03-28 | 2012-08-01 | 中国科学技术大学 | Adjusting method of technological parameters of sewage treatment and device |
| CN103399134A (en) * | 2013-08-20 | 2013-11-20 | 渤海大学 | Sewage COD soft measurement method based on output observer |
| CN103942457A (en) * | 2014-05-09 | 2014-07-23 | 浙江师范大学 | Water quality parameter time series prediction method based on relevance vector machine regression |
| JP2018018413A (en) * | 2016-07-29 | 2018-02-01 | ペンタフ株式会社 | Sewage unknown water simple evaluation method |
| CN107169610A (en) * | 2017-06-02 | 2017-09-15 | 中国农业大学 | Aquaculture dissolved oxygen prediction method and device |
| CN107688871A (en) * | 2017-08-18 | 2018-02-13 | 中国农业大学 | A kind of water quality prediction method and device |
| CN108414016A (en) * | 2018-03-01 | 2018-08-17 | 深圳市晟达机械设计有限公司 | A kind of sewage network monitoring system based on big data technology |
Non-Patent Citations (1)
| Title |
|---|
| 污水预警中的化学需氧量(COD)预测技术;叶延亮 等;《化工自动化及仪表》;20131210;正文第1528页至第1531页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109165247A (en) | 2019-01-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108898215B (en) | Intelligent sludge bulking identification method based on two-type fuzzy neural network | |
| US20170185892A1 (en) | Intelligent detection method for Biochemical Oxygen Demand based on a Self-organizing Recurrent RBF Neural Network | |
| Côte et al. | Dynamic modelling of the activated sludge process: improving prediction using neural networks | |
| CN110054274B (en) | Water purification flocculation precipitation dosing control method | |
| CN111553468B (en) | Method for accurately predicting water quality of effluent of sewage treatment plant | |
| KR101703981B1 (en) | Control methods of sewage processing facilities using rainfall forecasting | |
| CN108876021B (en) | Medium-and-long-term runoff forecasting method and system | |
| Lee et al. | Real-time remote monitoring of small-scaled biological wastewater treatment plants by a multivariate statistical process control and neural network-based software sensors | |
| CN102262147A (en) | Soft measurement method and system for effluent chemical oxygen demand (COD) of waste water treatment system | |
| CN110889085A (en) | Intelligent wastewater monitoring method and system based on complex network multiple online regression | |
| CN108595892A (en) | Soft-measuring modeling method based on time difference model | |
| CN108872508A (en) | A kind of eutrophy quality evaluation method of GA-BP optimization TSFNN | |
| CN113325702B (en) | Aeration control method and device | |
| CN106054951B (en) | A kind of control method and system of dissolved oxygen concentration | |
| US20220316994A1 (en) | A method for predicting operation effectiveness of decentralized sewage treatment facility by using support vector machine | |
| CN110221540B (en) | Continuous stirred reactor system control method based on Hammerstein model | |
| CN103399134B (en) | Sewage COD soft measurement method based on output observer | |
| CN109165247B (en) | Intelligent pretreatment method for sewage measurement data | |
| KR101208595B1 (en) | Gross error detecting method and system using lagrangian multiplier, and data reconciliation method using gross error detecting system | |
| CN116859839A (en) | Industrial control method and device based on model training | |
| CN112836859A (en) | Intelligent fusion and analysis method for river mouth area pollutant monitoring data | |
| CN114693493B (en) | IoT-based polluted river water ecological restoration system | |
| CN113780356B (en) | Water quality prediction method and system based on integrated learning model | |
| CN116341705A (en) | Long-period memory network water quality parameter prediction method based on sparse label | |
| CN112700050A (en) | Method and system for predicting ultra-short-term 1 st point power of photovoltaic power station |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |