CN105243256A - Biochemical oxygen demand parameter online soft measurement method - Google Patents

Abstract

The invention discloses a biochemical oxygen demand parameter online soft measurement method. The method comprises: performing normalization processing on sample data, and performing decoupling and dimension reduction by utilizing principal component analysis; deleting off-group points unmatched with main body data by utilizing a Jolliffe parameter, and removing noise interferences in data generation and acquisition processes by utilizing a median filter; determining a local linear regression model; performing weight initialization; performing model output prediction and model training, and adjusting a positive semidefinite distance matrix D with a gradient descent method by utilizing a cost function to obtain a weight; and performing online BOD parameter soft measurement on effective sample data through the trained local linear regression prediction model. The method has the beneficial effects that online prediction can be realized and only certain easily measured auxiliary variables need to be detected, so that the cost is relatively low.

Description

A kind of biochemical oxygen demand parameter online soft sensor method

Technical field

The invention belongs to DATA REASONING technical field, relate to a kind of biochemical oxygen demand parameter online soft sensor method.

Background technology

BOD refers to the biochemical oxygen demand of sewage effluent index, and in sewage disposal process, biochemical oxygen demand (BOD) is one of key parameter of evaluating water quality quality and treatment effect.Because it relates to a lot of complicated biochemical reaction process, therefore the measurement of BOD all also exists a lot of problem for a long time, is in particular in: international standard detecting method measurement delay is comparatively large, and method for quick is measured high cost and measured inaccurate.

Sewage disposal process high complexity, measurement parameter is numerous, the data of on-the-spot test are due to the impact by factors such as measurement accuracy of instrument, reliability and in-site measurement environment, can with various measuring error, if data are without further process, the low precision collected or the data of inefficacy can cause measuring accuracy significantly to decline, and even directly can cause the inefficacy of wastewater treatment soft-sensing model.Meanwhile, usually need to gather more Parameter data information better to reflect sewage disposal process state, but too much supplemental characteristic makes follow-up model very complicated.Therefore, how to ensure the validity of image data, and how eliminate redundancy information is the key technical problem that first BOD parameter hard measurement will solve.

Mainly survey pressure differential method and hexavalent chrome bio-removal that BOD measuring instrument in the market adopts are that principle makes.Such as U.S.'s Hash BODTrakTMlI analyser, German LovibondBOD analyser etc., employing be survey pressure differential method; Japan CKC company α 1000 type BOD measuring instrument adopt be method for biosensor make.Adopt the surveying instrument Measuring Time of pressure differential method long, need 5 days, the needs that sewage disposal process controls in real time cannot be met far away; Adopt the BOD surveying instrument of hexavalent chrome bio-removal to there is instrument biology sensor and make difficulty, measurement range is narrower, and relevant membrane material easily damages thus causes the serviceable life of instrument not enough, and use cost is too high, thus also cannot widespread use.

Summary of the invention

The object of the present invention is to provide a kind of biochemical oxygen demand parameter online soft sensor method, solve the existing measuring method time long, the problem that cost is high.

The technical solution adopted in the present invention is carried out according to following steps:

Step 1: sample data pre-service, obtains effective sample data;

(1) sample data normalized, makes sample data be between [-1,1];

(2) pivot analysis decoupling zero and dimensionality reduction is utilized;

(3) Jolliffe parameter is utilized to reject the outlier do not conformed to body data, to improve the reliability of data;

(4) noise in median filter removal data genaration and gatherer process is utilized;

Step 2: the foundation of local linear smoothing forecast model and training thereof;

(1) Locallinearregressionmodel is determined;

(2) weight initialization;

(3) the model prediction of output, a given new data entry point, a calculating K linear input model also obtains predicted value;

(4) model training, utilizes cost function to adjust positive semidefinite Distance matrix D by gradient descent method and obtains weights;

(5) if cost function does not meet the demands, then adjust matrix D and then amendment weights, get back to (3) step, until cost function meets the demands, then model training terminates, and weights are determined;

Step 3:BOD parameter online soft sensor exports; Effective sample data are carried out online BOD parameter hard measurement by the local linear smoothing forecast model trained.

Further, in described online BOD parameter hard measurement, pre-service is carried out to the auxiliary variable detected in real time, comprise as normalization, pivot analysis, determine whether outlier, noise filtering.

Further, in described step 2, weight initialization, weight w _igaussian kernel is utilized to obtain:

w _i＝exp(-0.5(x _i-x _c) ^TD(x _i-x _c))，W＝diag{w ₁，...，w _M}

Wherein D is positive semidefinite distance matrix, which determines data point x _cthe size and shape of neighbouring data set;

The model prediction of output, a given new data entry point, a calculating K linear input model also obtains predicted value total output is then carry out linear averaging according to weights:

$\hat{y}=\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{w}_k{\hat{y}}_k/\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{w}_k---\left(8\right);$

Model training, utilizes the cost function shown in formula (8) to adjust matrix D by gradient descent method thus study obtains weights;

$J=\frac{1}{{\mathrm{\Σ}}_{i=1}^M}\underset{i=1}{\overset{M}{\mathrm{\Σ}}}{w}_i(y_i-{\hat{y}}_{i,-i}^2)+\frac{\mathrm{\γ}}{N}\underset{i,j=1}{\overset{N}{\mathrm{\Σ}}}{D}_{\mathrm{ij}}^2---\left(9\right)$

Wherein M is the number of data point in data acquisition, and parameter γ is obtained by the mode of gathering through test.

The invention has the beneficial effects as follows and can be implemented in line prediction, and only need detect some auxiliary variables easily measured, cost is lower.

Accompanying drawing explanation

Fig. 1 is data prediction process flow diagram;

Fig. 2 is model training process flow diagram;

Fig. 3 is active sludge waste water processes;

Fig. 4 is the outlier detection of the test data based on Jolliffe parameter;

Fig. 5 is LWPR algorithm predicts design sketch.

Embodiment

Below in conjunction with embodiment, the present invention is described in detail.

Step 1: sample data pre-service;

As shown in Figure 1, the pretreated concrete steps of sample data are:

(1) sample data normalized, makes sample data be between [-1,1].

(2) pivot analysis (PCA) decoupling zero and dimensionality reduction is utilized.The such as observed reading of given one group of M centralization, its standard variance is σ, simultaneously each variable x _kall m dimension data, x (k)=(x _k(1); :::; x _k(m)) wherein m < M.Pivot analysis is by variable x _kbe mapped to another one variable t _k:

t _k＝p ^Tx _k(1)

Wherein P is the orthogonal matrix of m × m, its i-th row υ _ii-th proper vector of covariance matrix C.In other words PCA first must solve eigenvalue problem:

λ _iυ _i＝Cυ _i，i＝1，...，m(2)

Wherein λ _ii-th eigenwert of C, v _ithen the proper vector relevant to this eigenwert.Based on v _iand x _korthogonal transformation can obtain pivot t _k, as shown by the following formula:

$t_k\left(i\right)={\mathrm{\&upsi;}}_i^T{x}_k,i=1,...,m---\left(3\right)$

Through PCA conversion, front several proper vector arranges out according to the mode of eigenwert descending.Obtain important eigenwert, proper vector and pivot with this, and ignore inessential variable.With this simultaneously, the coupling between raw data have also been obtained elimination.

(3) Jolliffe parameter is utilized to reject the outlier do not conformed to body data, to improve the reliability of data.

$\begin{array}{c}{d}_{1i}^2=\underset{k=M-q+1}{\overset{R}{\mathrm{\&Sigma;}}}{t}_{\mathrm{ik}}^2,\\ d_{2i}^2=\underset{k=M-q+1}{\overset{R}{\mathrm{\&Sigma;}}}\frac{t_{\mathrm{ik}}^2}{{\mathrm{\&sigma;}}_k},\\ d_{3i}^2=\underset{k=M-q+1}{\overset{R}{\mathrm{\&Sigma;}}}{\mathrm{\&sigma;}}_k{t}_{\mathrm{ik}}^2,\end{array}---\left(4\right)$

Wherein t _ikbe i-th observed reading of a kth pivot, namely the number of M variable is attribute number, and q represents the number (such as variance < 1) of little variance pivot, and R represents large variance pivot number, and σ is the standard deviation of a kth pivot, statistic d _1iand d _2imainly for detection of the data of offset from body structure, and d _3ifor detecting the data of dramatic impact agent structure variance, d is set _1i, d _2iand d _3ithreshold value can reject outlier.

(4) noise in median filter removal data genaration and gatherer process is utilized.

The present invention is to obtain clean effective data to sample data pre-service object, it is the necessary links setting up hard measurement output model, its Main Function has: 1. utilize pivot analysis (PCA) decoupling zero and reduce sample data to obtain the input auxiliary variable of important pivot and following model, ignore the auxiliary variable that irrelevant essence is wanted, thus minimizing data dimension, reduce the complexity of model treatment; 2. reject the outlier do not conformed to body data, to improve the reliability of data, outlier is mainly derived from fault or the maintenance of instrument, and its existence very easily causes departing from of follow-up modeling even failed; 3. the noise in data genaration and gatherer process is removed in filtering, noise produces the interference of instrument and sensor and impact mainly due to the problem of instrument own and surrounding environment, it is degrading the quality of data, even flood target signature, bring difficulty to subsequent analysis process.Obvious lifting is obtained through pretreated data reliability.

Step 2: the foundation of local linear smoothing forecast model and training thereof;

(1) determination of local linear smoothing (LWPR) model.The regression model of the standard that remains that LWPR algorithm is considered:

y＝f(x)+∈(5)

Wherein, x represents the input data that d ties up, and y represents standardized output, and ∈ is 0 average random noise simultaneously.And if only if at data point x _cdata in certain limit are just paid attention to.Meanwhile, with lower order polynomial expressions, modeling is carried out to these local datas.For the consideration of computational complexity and Model approximation power balance, adopt the linear model such as formula (6):

y＝β ^Tx+∈(6)。

(2) weight initialization.In order to measure the locality set of each point, weight w _igaussian kernel is utilized to obtain:

w _i＝exp(-0.5(x _i-x _c) ^TD(x _i-x _c))，W＝diag{w ₁，...，w _M}(7)

Wherein D is positive semidefinite distance matrix, which determines data point x _cthe size and shape of neighbouring data set.

(3) the model prediction of output.A given new data entry point, a calculating K linear input model also obtains predicted value total output is then carry out linear averaging according to weights:

$\hat{y}=\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{w}_k{\hat{y}}_k/\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{w}_k---\left(8\right).$

(4) model training.Utilize the cost function shown in formula (8) to adjust matrix D by gradient descent method thus learn to obtain weights:

$J=\frac{1}{{\mathrm{\&Sigma;}}_{i=1}^M}\underset{i=1}{\overset{M}{\mathrm{\&Sigma;}}}{w}_i(y_i-{\hat{y}}_{i,-i}^2)+\frac{\mathrm{\&gamma;}}{N}\underset{i,j=1}{\overset{N}{\mathrm{\&Sigma;}}}{D}_{\mathrm{ij}}^2---\left(9\right)$

Wherein M is the number of data point in data acquisition.Part I is the cross check error of cost function partial model, can ensure the generalization ability of model with this.And Part II is penalty term, ensure the atrophy that acceptance domain can not be unlimited when running into large training set.Parameter γ can be obtained by the mode of gathering through test.

(5) if cost function does not meet the demands, then adjust matrix D and then amendment weights, enter (3) step, so circulate, until cost function meets the demands, then model training terminates, and weights are determined.

According to effective sample data characteristics, set up the local linear plan model having higher asymptotic efficiency and adaptive faculty, input quantity is the pivot that joint obtains and namely mainly inputs auxiliary variable, the predicted value of the linear input model of calculating K, then carries out linear weighted function according to weights average.These weights utilize gaussian kernel to obtain, relevant one and positive semidefinite distance matrix.Adjustment positive semidefinite distance matrix, until met the demands by the cross check error cost function that forms of prediction output valve and sample data measured value, thus determines weights.Cost function introduces cross check error to ensure the generalization ability of model, also adds penalty term to ensure to be unlikely to unlimited atrophy when running into large training set simultaneously.Model is set up and is trained flow process as shown in Figure 2.

Step 3:BOD parameter online soft sensor exports

The Locallinearregressionmodel trained can be used for online BOD parameter hard measurement.The main auxiliary variable detected in real time needs to carry out pre-service equally, as normalization, pivot analysis, determines whether outlier, noise filtering etc.Its weights of the model trained are determined, then formula (8) can be utilized to predict output valve.

The present invention be advantageous in that the flexible measurement method of proposition adopts the mode of data-driven, the auxiliary variable relevant to BOD parameter is obtained according to the change of operating condition in sewage disposal system, pre-service is carried out to the data collected, then adopts local linear smoothing algorithm (LWPR) on-line prediction BOD parameter.BOD parameter soft measurement method proposed by the invention can be implemented in line prediction, and only need detect some auxiliary variables easily measured, and cost is lower.Although BOD parameter soft measurement method has obtained extensive concern and research at present, but the flexible measurement method that the present invention proposes ensure that the validity of data and eliminates redundant information in process of data preprocessing, simplify forecast model, make set up Locallinearregressionmodel can on-line prediction.Therefore, from data processing and forecast model two aspect, the present invention ensures that proposed method is simple, effective simultaneously.

Fig. 3 gives active sludge treatment process, mainly comprise pre-service, just heavy, aeration, two heavy and sludge reflux 5 parts, utilize the effects such as the cohesion of the micropopulation in aeration tank inner suspension, flowing, absorption, oxygenolysis to remove dirty Organic substance in water.Sewage is after pre-service and coagulation, preliminary sedimentation tank water outlet enters aeration tank, under the effect of active sludge, carry out aerobic and anaerobic biodegradation, the organic substance decomposing of the dissolving in sewage and colloidal state is carried out denitrogenation dephosphorizing reaction, the mixing water outlet of aeration tank enters second pond and carries out Separation of Solid and Liquid, and top primary water enters receiving water body.Activated sludge process will produce a large amount of mud, except the active sludge of backflow, after the mud mixing of excess sludge and preliminary sedimentation tank, carries out digesting, the sludge handling process such as dehydration.

BOD ₅reflect that water body is by the overall target of Organic Pollution degree, BOD ₅not only with suspended sediment concentration, the variablees such as chemical oxygen demand (COD) are closely related, are also subject to other disturbing factor, as temperature, and flow distribution, pH etc., therefore, BOD ₅with suspended sediment concentration, chemical oxygen demand (COD) etc. have very strong nonlinear relationship.Wherein, the auxiliary variable selected of embodiment is in table 1.

Table 1 auxiliary variable table

Sampling obtains 526 groups of data, comprises 19 row inputs and a row output (i.e. BOD ₅).Wherein 400 groups of data are used to training pattern, and other 126 groups of data are used for verifying.As shown in Figure 4, after PCA and Jolleffi parameter processing, we can detect outlier data group clearly.Meanwhile, data also can reduce to 14 dimensions (14 pivots) from original 19 dimensions, thus reduce the complicacy of soft-sensing model.

Fig. 5 is the design sketch of the method prediction BOD parameter that the present invention proposes, and average error is 4.95%, more satisfactory, and this algorithm calculated amount is little.Therefore, method proposed by the invention is applicable to the complex process of this multivariate of wastewater treatment, strong coupling.

The above is only to better embodiment of the present invention, not any pro forma restriction is done to the present invention, every any simple modification done above embodiment according to technical spirit of the present invention, equivalent variations and modification, all belong in the scope of technical solution of the present invention.

Claims (3)

1. a biochemical oxygen demand parameter online soft sensor method, is characterized in that carrying out according to following steps:

Step 1: sample data pre-service, obtains effective sample data;

(1) sample data normalized, makes sample data be between [-1,1];

(2) pivot analysis decoupling zero and dimensionality reduction is utilized;

(3) Jolliffe parameter is utilized to reject the outlier do not conformed to body data, to improve the reliability of data:

(4) noise in median filter removal data genaration and gatherer process is utilized;

Step 2: the foundation of local linear smoothing forecast model and training thereof;

(1) Locallinearregressionmodel is determined;

(2) weight initialization:

(3) the model prediction of output, a given new data entry point, a calculating K linear input model also obtains predicted value;

(4) model training, utilizes cost function to adjust positive semidefinite Distance matrix D by gradient descent method and obtains weights;

(5) if cost function does not meet the demands, then adjust matrix D and then amendment weights, get back to (3) step, until cost function meets the demands, then model training terminates, and weights are determined;

Step 3:BOD parameter online soft sensor exports;

Effective sample data are carried out online BOD parameter hard measurement by the local linear smoothing forecast model trained.

2. according to biochemical oxygen demand parameter online soft sensor method a kind of described in claim 1, it is characterized in that: in described online BOD parameter hard measurement, pre-service is carried out to the auxiliary variable detected in real time, comprises as normalization, pivot analysis, determine whether outlier, noise filtering.

3., according to biochemical oxygen demand parameter online soft sensor method a kind of described in claim 1, it is characterized in that: in described step 2, weight initialization, weights ω _igaussian kernel is utilized to obtain:

ω _i＝exp(-0.5(x _i-x _c) ^TD(x _i-x _c))，W＝diag{ω ₁，...，ω _M}

Wherein D is positive semidefinite distance matrix, which determines data point x _cthe size and shape of neighbouring data set;

The model prediction of output, a given new data entry point, a calculating K linear input model also obtains predicted value total output is then carry out linear averaging according to weights:

$\hat{y}=\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{\mathrm{\&omega;}}_k{\hat{y}}_k/\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{\mathrm{\&omega;}}_k---\left(8\right)$

Model training, utilizes the cost function shown in formula (8) to adjust matrix D by gradient descent method thus learns to obtain weights:

$J=\frac{1}{{\mathrm{\&Sigma;}}_{i=1}^M}\underset{i=1}{\overset{M}{\mathrm{\&Sigma;}}}{\mathrm{\&omega;}}_i(y_i-{\hat{y}}_{i,-i}^2)+\frac{\mathrm{\&gamma;}}{N}\underset{i,j=1}{\overset{N}{\mathrm{\&Sigma;}}}{D}_{\mathrm{ij}}^2---\left(9\right)$

Wherein M is the number of data point in data acquisition, and parameter γ is obtained by the mode of gathering through test.