CN101794345A - Data processing method and device - Google Patents

Abstract

The invention discloses a data processing method and a device, and relates to the technical field of intelligent traffic systems. The invention aims to solve the problem that the medium filtering smoothing technology in the prior art has the detect of random prediction process, so that the prediction precision is low, thereby failing to satisfy the requirements for actual prediction. The embodiment of the invention provides a data processing method which comprises the following steps: acquiring historical data information; carrying out data preprocessing on the historical data information; carrying out data combination and filling on the preprocessed historical data information; and carrying out dynamic exponential smoothing process on the historical data information. The embodiment of the invention can enhance the prediction precision and satisfy the requirements for actual prediction.

Description

A kind of data processing method and device

Technical field

The present invention relates to the intelligent transportation system technical field, relate in particular to a kind of data processing method and device.

Background technology

Advanced transportation information service systems (Advanced Traffic Information System, ATIS) be based upon on the perfect information network basis, this system can obtain all kinds of transport information by sensor or the data transmission set that is equipped in road, car, transfer stop, parking lot and forecast center, carries out overall treatment according to the described data that get access to.This system can provide Real-time Road traffic congestion information comprehensively and accurately to society in real time.But the data accessed by described equipment can not cover all roads fully, fill up thereby need carry out real time data by the similar inquiry of historical data, and available historical data are predicted after by analysis.

In order to improve the availability of dynamic information, the function that needs the information prediction of increase system, need carry out independent analysis to the historical road condition data of past certain hour in the cycle, obtain the variation tendency of every road in the traffic of historical data in the cycle, the mode by interface offers transportation information service systems and uses.Yet in the prior art, adopt median filter smoothness of image to handle usually historical data is handled, thereby the purpose of realization data prediction.

In realizing process of the present invention, the inventor finds that there are the following problems at least in the prior art: because the median filter smoothness of image treatment technology forecasting process that prior art adopted comparatively at random, make precision of prediction lower, can't satisfy the needs of actual prediction.

Summary of the invention

Embodiments of the invention provide a kind of data processing method and device.

For achieving the above object, embodiments of the invention adopt following technical scheme:

A kind of data processing method comprises:

Obtain historical data information;

Described historical data information is carried out the data pre-service;

Described pretreated historical data information is carried out data to be merged and fills up;

Described data are merged and fill up after historical data information carry out the dynamic index smoothing processing.

A kind of data processing equipment comprises:

Information acquisition unit is used to obtain historical data information;

Pretreatment unit is used for described historical data information is carried out the data pre-service;

Data merge shim, are used for that described pretreated historical data information is carried out data and merge and fill up;

The data smoothing unit is used for described data merging and the historical data information after filling up is carried out the dynamic index smoothing processing.

Data processing method that the embodiment of the invention provides and device are by obtaining historical data information; Described historical data information is carried out the data pre-service; Described pretreated historical data information is carried out data to be merged and fills up; Described data are merged and fill up after historical data information carry out the dynamic index smoothing processing.Compared with prior art, the dynamic index smoothing processing of employing of the present invention can be so that precision of prediction be higher and can satisfy the needs of actual prediction.

Description of drawings

A kind of data processing method process flow diagram that Fig. 1 provides for the embodiment of the invention;

In a kind of data processing method that Fig. 2 provides for the embodiment of the invention described historical data information is carried out the implementation procedure process flow diagram of the pretreated step of data;

In a kind of data processing method that Fig. 3 provides for the embodiment of the invention described pretreated historical data information carried out that data merge and the implementation procedure process flow diagram flow chart of the step filled up;

In a kind of data processing method that Fig. 4 provides for the embodiment of the invention described data are merged and fill up after historical data information carry out the implementation procedure process flow diagram of the step of dynamic index smoothing processing;

A kind of data processing equipment structural representation that Fig. 5 provides for the embodiment of the invention.

Embodiment

Below in conjunction with accompanying drawing embodiment of the invention data processing method and device are described in detail.

As shown in Figure 1, be a kind of data processing method that the embodiment of the invention provides, this method comprises:

101: obtain historical data information;

102: described historical data information is carried out the data pre-service; This step mainly is described historical data to be carried out the rejecting of abnormal data, improves the quality of historical data.

103: described pretreated historical data information is carried out data merge and fill up;

104: described data are merged and fill up after historical data information carry out the dynamic index smoothing processing.

Because described historical data exists various interference can produce a collection of abnormal data in collection and processing output procedure, if being carried out the data analysis meeting, described abnormal data has influence on the accuracy that predicts the outcome at last, so need carry out pre-service to described historical data, eliminate described abnormal data.Can adopt a kind of statistical method to come the rejecting abnormalities vehicle speed value in the embodiment of the invention based on the time band.

The implementation procedure of the pretreated step of data is provided described historical data information in a kind of data processing method that provides for the embodiment of the invention as shown in Figure 2; If the value of the time dimension of described historical data is 00:00-23:59; Per 5 minutes is a time period; The time band is meant one day vehicle speed value of certain road; Wherein, to be a time band half an hour.Described historical data pre-service is rejected some abnormal datas based on continuous urban history road condition data more than month, and its concrete implementation procedure comprises:

201: described historical data information is carried out the time band divide; Wherein, described historical data can be the historical vehicle speed value of reading from database; Described time band divide be to one day the speed of a motor vehicle of particular link dividing half an hour, thereby tentatively obtain 48 times bands.

202: the time band according to dividing merges verification; The specific implementation process of this step is:

Being with the T that successively uses F check and two samples to check the described ready-portioned time judges whether and can be with the merging time; Wherein, described F check is used to judge whether the variance of two time bands to be tested equates; The T check of described pair of sample is used to judge whether the average of two time bands to be tested equates; If the T check of described F check and described pair of sample all by merge two adjacent times bands.

The formula of structure F test statistics is: $F=\frac{S_1^2}{S_2^2}~F(n_1-1,n_2-1);$

Wherein, $S_1^2=\frac{1}{n_1-1}\underset{i=1}{\overset{n_1}{\mathrm{\Σ}}}{(X_i-\stackrel{\‾}{X})}^2,S_2^2=\frac{1}{n_2-1}\underset{i=1}{\overset{n_2}{\mathrm{\Σ}}}{(Y_i-\stackrel{\‾}{Y})}^2;$

X _i, Y _iBe respectively each vehicle speed value of two normal populations;

X, Y are sample average.

The region of rejection of F check is:

$W=\{F<F_{\frac{\mathrm{\&alpha;}}{2}}(n_1-1,n_2-1)$ Or $F>F_{1-\frac{\mathrm{\&alpha;}}{2}}(n_1-1,n_2-1)\}$

α is an insolation level.

The formula of structure T test statistics is: $T=\frac{\stackrel{\&OverBar;}{X}-\stackrel{\&OverBar;}{Y}}{S_W\&CenterDot;\sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}~t(n_1+n_2-2);$

Wherein, n ₁, n ₂Be respectively the effective value number of two normal populations;

${s_W}^2=\frac{(n_1-1){S_1}^2+(n_2-1){S_2}^2}{n_1+n_2-2}$ (S ₁ ², S ₂ ²Be respectively the variance of two normal populations)

The region of rejection of T check is: $W=\left\{\right|T|>t_{1-\frac{\mathrm{\&alpha;}}{2}}(n_1+n_2-2)\}.$

It should be noted that if band of described time satisfies the merging condition, then it merged that the time band after will merging is then handled as a time band; If described time band does not satisfy the merging condition, then it is still carried out follow-up processing according to the time band of dividing after its pre-service.

203: carry out unusual verification with described through the time band that merges verification, provide unusual check results; The implementation procedure of this step is: traversal is through merging the time band of handling, described effective time band is carried out the T check of U check or single sample respectively, if described effective time band is not by described check, think that then these data are abnormal data, record described abnormal data in the unusual check results.Wherein, described U check is applicable to fully big situation of sample size.

It should be noted that the U check is used under the very big situation of sample number, condition is that speed of a motor vehicle effective value number is greater than 30 herein.The formula of structure U test statistics is: $U=\frac{x_i-\mathrm{\&mu;}}{\mathrm{\&sigma;}/\sqrt{n}}$

Wherein, x _iRepresent the speed of a motor vehicle variable in this time band, the speed of a motor vehicle average of μ representative time band, the speed of a motor vehicle variance of σ representative time band, the effective value number of n representative time band;

The region of rejection of U check is: W={|U|＞μ _α/2;

Single sample T check is used under the situation of small sample amount, and condition is that speed of a motor vehicle effective value number is less than or equal to 30 herein.The formula of structure T test statistics is: $T=\frac{x_i-\stackrel{\&OverBar;}{X}}{s/\sqrt{n}};$

Wherein, x _iRepresent the speed of a motor vehicle variable in this time band, the speed of a motor vehicle average of μ representative time band, the speed of a motor vehicle variance of s representative time band, the effective value number of n representative time band;

The region of rejection of T check is: W={|T|＞t _α(n-1) }.

204:, abnormal data is rejected according to described unusual check results.

As shown in Figure 3, in a kind of data processing method that provides for the embodiment of the invention described pretreated historical data information carried out that data merge and the implementation procedure of the step filled up, this process comprises:

301: receive the described historical data information of rejecting through abnormal data;

302: the data in the identical moment in the described historical data information are merged processing; Concretely, be exactly that the data on the identical time point merge to identical week characteristic day, simply use the method for arithmetic mean to merge processing after, obtain one group of data.

303: whether the data that detect after described merging is handled exist no datat on the moment point;

304:, then carry out data filling and handle if there is no datat on the moment point.Wherein, the described data filling that carries out is handled and can be adopted the method for least square method to fill up, and for example: each two temporal data is done sample point before and after can getting the time point that needs fill data; Its specific implementation process is as follows:

Setting up regression model is $\hat{Y}={\widehat{\mathrm{\&beta;}}}_0+{\widehat{\mathrm{\&beta;}}}_{1}x;$

Wherein: x need to represent the time point of padding data;

Represent the data that to fill up on this time point;

$\left\{\begin{array}{c}{\widehat{\mathrm{\&beta;}}}_0=\widehat{\stackrel{\&OverBar;}{y}}-\stackrel{\&OverBar;}{x}{\widehat{\mathrm{\&beta;}}}_1\\ {\widehat{\mathrm{\&beta;}}}_1{=L}_{\mathrm{xy}}/L_{\mathrm{xx}}\end{array}\right.;$

Wherein, $L_{\mathrm{xy}}\stackrel{\mathrm{def}}{=}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}(x_i-\stackrel{\&OverBar;}{x})(y_i-\stackrel{\&OverBar;}{y})=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{x}_i{y}_i-n\stackrel{\&OverBar;}{x}\stackrel{\&OverBar;}{y},L_{\mathrm{xx}}\stackrel{\mathrm{def}}{=}\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{(x_i-\stackrel{\&OverBar;}{x})}^2=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{x_i}^2-n{\stackrel{\&OverBar;}{x}}^2;$

(x _i, y _i) be each two temporal data value before and after the x time point of choosing.

As shown in Figure 4, in a kind of data processing method that provides for the embodiment of the invention described data are merged and fill up after historical data information carry out the implementation procedure of the step of dynamic index smoothing processing, the initial value that this process is established smoothing factor α is α ₀, the initial value of exponential smoothing and control accuracy ε.Wherein, described control accuracy is used for judging whether being approximate optimal solution.This process specifically comprises:

401: receive the described historical data information of filling up processing through number;

402: the initial value α that obtains smoothing factor α ₀, exponential smoothing initial value and control accuracy ε;

403:, obtain the exponential smoothing numerical value of described next moment point of exponential smoothing initial value according to the initial value and the described exponential smoothing initial value of described smoothing factor; Wherein, described exponential smoothing numerical value is that computing formula by exponential smoothing obtains.Described exponential smoothing is a kind of information processing method that is come by the development of moving average method.This method does not need to store the time series data of n phase, and gives recent real data with bigger flexible strategy, gives data at a specified future date with less flexible strategy, and the flexible strategy index of coincidence rule of each issue certificate.The exponential smoothing computing formula is:

${\hat{y}}_{t+1}=\mathrm{\&alpha;}{y}_t+(1-\mathrm{\&alpha;}){\hat{y}}_t---(1-1)$

In the formula, because the data in the embodiment of the invention are per 5 minutes among a 0:00-23:59 data, t value 1-288, expression begins per 5 minutes time points from 0:00, t=1, expression 0:00; T=288, expression 23:55;

--smooth value of 5 minutes behind the-period t;

y _t---period t actual value;

---period t smooth value;

α----smoothing factor claims weighting factor again, and span is 0≤α≤1.

Following formula shows: the exponential smoothing prediction is constantly the predicted value of last issue to be revised with predicated error, and obtains the predicted value of current period.

Be

And y _tThe weighted arithmetic mean number, along with the size variation of α value, the decision

Right with yt

Influence degree.

Have by the phase and review character, include the influence of full issue certificate.

404:, judge whether optimum smoothing factor of described smoothing factor according to described exponential smoothing data and the control accuracy that gets access to; Concrete implementation procedure is:

Calculate $\&dtri;\mathrm{SSE}\left({\mathrm{\&alpha;}}_0\right)=(\mathrm{dSSE}/\mathrm{d\&alpha;}){|}_{\mathrm{\&alpha;}={\mathrm{\&alpha;}}_0};$ If $\left|\right|\&dtri;\mathrm{SSE}\left({\mathrm{\&alpha;}}_0\right)\left|\right|\&le;\mathrm{\&epsiv;},$ α then ₀It is exactly approximate optimal solution; Otherwise not optimum solution, then begin the smoothing factor search.

405: if optimum smoothing factor is then proceeded the exponential smoothing of next moment point and handled;

406:, then carry out the smoothing factor search if not optimum smoothing factor.The specific implementation process of this step is as follows:

Step 1: for described smoothing factor is set up squared prediction error and minimum Optimization Model;

$\min \mathrm{SSE}=\underset{t=1}{\overset{n}{\mathrm{\&Sigma;}}}{e}_t^2=\underset{t=1}{\overset{n}{\mathrm{\&Sigma;}}}{(y_t-{\hat{y}}_t)}^2---(1-2)$

Described exponential smoothing computing formula (1-1) is given

After the assignment, following formula can be expressed as

$\min \mathrm{SSE}=\underset{t=1}{\overset{n}{\mathrm{\&Sigma;}}}{[y_t-(\mathrm{\&alpha;}{y}_{t-1}+(1-\mathrm{\&alpha;})y_{t-1}^{^})]}^2---(1-3)$

Successively will

Value bring following formula into, obtain after the arrangement:

$\min \mathrm{SSE}=\underset{t=1}{\overset{n}{\mathrm{\&Sigma;}}}{[y_t-\mathrm{\&alpha;}\underset{j=1}{\overset{t-1}{\mathrm{\&Sigma;}}}{(1-\mathrm{\&alpha;})}^{j-1}{y}_{t-j}-{(1-\mathrm{\&alpha;})}^{t-1}{\hat{y}}_1]}^2---(1-4)$

Because when the t value is bigger,

In fact very little, can be approximated to be 0.Then target equation (1-4) can be reduced to:

$\min \mathrm{SSE}=\underset{t=1}{\overset{n}{\mathrm{\&Sigma;}}}{[y_t-\mathrm{\&alpha;}\underset{j=1}{\overset{t-1}{\mathrm{\&Sigma;}}}{(1-\mathrm{\&alpha;})}^{j-1}{y}_{t-j}]}^2---(1-5)$

Find the solution above-mentioned nonlinear optimization model (1-5) and can obtain best α value.Ask the minimum value of following formula that a lot of methods are arranged, the steepest decline process of iteration of designed following fast convergence rate in embodiments of the present invention, easily carrying out: i.e. optimum gradient method.

Described optimum gradient method is the direction of search of the negative gradient direction of application target function as each step iteration.Because each step is all got the optimal step size of negative gradient direction, so be called optimum gradient method.Use optimum gradient method, it is the fastest that its target function value was descended in former steps, so be called method of steepest descent.

For example: the gradient of a n dimension nonlinear function f (X) is defined as:

$\&dtri;f\left(x\right)=\frac{\&PartialD;f}{\&PartialD;X}={[\frac{\&PartialD;f}{\&PartialD;x_1},\frac{\&PartialD;f}{{\&PartialD;x}_2},...,\frac{\&PartialD;f}{\&PartialD;x_n}]}^T$

This is a n dimension local derviation vector.

The vector of unit length of gradient is: $S=\frac{\&dtri;f\left(X\right)}{\left|\right|\&dtri;f\left(X\right)\left|\right|}$

As can be seen, gradient direction is the normal direction of function just, the negative gradient direction with-S in the same way.Wherein

Be called gradient Mould, also be norm.The computing formula of mould is

$\left|\right|\&dtri;f\left(X\right)\left|\right|=\sqrt{{\left(\frac{\&PartialD;f}{\&PartialD;x_1}\right)}^2+{\left(\frac{\&PartialD;f}{\&PartialD;x_2}\right)}^2+...+{\left(\frac{\&PartialD;f}{\&PartialD;x_n}\right)}^2}$

The character of gradient direction has: it is the fastest to descend along negative gradient directivity function value, is direction of steepest descent.The iterative formula of optimum gradient method:

Schilling X _K+1=X _k+ λ _kS _k

Wherein, vector of unit length $S_k=-\frac{\&dtri;f\left(X_k\right)}{\left|\right|\&dtri;f\left(X_k\right)\left|\right|};$

λ _kBe optimal step size.Therefore, the concrete definition to optimum gradient method is: the optimization numerical method of getting the optimal step size search all the time along the negative gradient direction.

By the iterative formula of optimum gradient method, the formula of function minimization can be arranged

$\underset{\mathrm{\&lambda;}}{\min }(X_k+\mathrm{\&lambda;}{S}_k)=f(X_k+{\mathrm{\&lambda;}}_k{S}_k)=f\left(X_{k+1}\right)$

With f (X) at X _kBe launched into Taylor series near the point $f\left(X\right)=f\left(X_k\right)+{\&dtri;}^{T}f\left(X_k\right)\mathrm{\&Delta;X}+\frac{1}{2}\mathrm{\&Delta;}{X}^T\mathrm{A\&Delta;X};$

Wherein, $A=\left(\begin{array}{c}\frac{{\&PartialD;}^{2}f}{\&PartialD;{x_1}^2},\frac{{\&PartialD;}^{2}f}{\&PartialD;x_1\&PartialD;x_2},....\frac{{\&PartialD;}^{2}f}{\&PartialD;x_1\&PartialD;x_n}\\ \frac{{\&PartialD;}^{2}f}{\&PartialD;x_2\&PartialD;x_1},\frac{{\&PartialD;}^{2}f}{\&PartialD;{x_2}^2},....\frac{{\&PartialD;}^{2}f}{\&PartialD;x_2\&PartialD;x_n}\\ \&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\&CenterDot;\\ \frac{{\&PartialD;}^{2}f}{\&PartialD;x_n\&PartialD;x_1},\frac{{\&PartialD;}^{2}f}{\&PartialD;x_n\&PartialD;x_2},....\frac{{\&PartialD;}^{2}f}{\&PartialD;{x_n}^2}\end{array}\right);$

With Δ X=λ S _kThe substitution following formula has

$f\left(X\right)=f\left(X_k\right)+{\&dtri;}^{T}f\left(X_k\right)\mathrm{\&lambda;}{S}_k+\frac{1}{2}{\left(\mathrm{\&lambda;}{S}_k\right)}^{T}A\left(\mathrm{\&lambda;}{S}_k\right)$

And

$\frac{\&PartialD;f}{\&PartialD;\mathrm{\&lambda;}}{|}_{\mathrm{\&lambda;}={\mathrm{\&lambda;}}_k}={\&dtri;}^{T}f\left(X_k\right)S_k+\mathrm{\&lambda;}{S_k}^{T}A{S}_k=0$

So optimal step size can be expressed as

${\mathrm{\&lambda;}}_k=-\frac{{\&dtri;}^{T}f\left(X\right){\mathrm{AS}}_k}{{S_k}^{T}A{S}_k}$

Since in the embodiment of the invention for asking $\min \mathrm{SSE}=\underset{t=1}{\overset{n}{\mathrm{\&Sigma;}}}{[y_t-\mathrm{\&alpha;}\underset{j=1}{\overset{t-1}{\mathrm{\&Sigma;}}}{(1-\mathrm{\&alpha;})}^{j-1}{y}_{t-j}]}^2$ Minimum value, this function is the one dimension function that contains variable α.So α of derivation one dimension _kValue, derivation is as follows:

α _k+1＝α _k+λS _k

Wherein $s_k=-\frac{\&dtri;f\left({\mathrm{\&alpha;}}_k\right)}{\left|\right|\&dtri;f\left({\mathrm{\&alpha;}}_k\right)\left|\right|},$ λ is a step-length.

$\underset{\mathrm{\&lambda;}}{\min }f({\mathrm{\&alpha;}}_k+\mathrm{\&lambda;}{s}_k)=f({\mathrm{\&alpha;}}_k+{\mathrm{\&lambda;}}_k{s}_k)=f\left({\mathrm{\&alpha;}}_{k+1}\right)$

With f (x) at α _kBe launched into Taylor series near the point $f\left(x\right)=f\left({\mathrm{\&alpha;}}_k\right)+\&dtri;f\left({\mathrm{\&alpha;}}_k\right)\mathrm{\&Delta;x}+\frac{1}{2}{f}^{\&prime;\&prime;}\left({\mathrm{\&alpha;}}_k\right)\mathrm{\&Delta;}{x}^2;$

Because of Δ x=λ s _k, bring following formula into, have

$f\left(x\right)=f\left({\mathrm{\&alpha;}}_k\right)+\&dtri;f\left({\mathrm{\&alpha;}}_k\right)\mathrm{\&lambda;}{s}_k+\frac{1}{2}{f}^{\&prime;\&prime;}\left(x\right){\left(\mathrm{\&lambda;}{s}_k\right)}^2$

And $\frac{\&PartialD;f}{\&PartialD;\mathrm{\&lambda;}}{|}_{\mathrm{\&lambda;}={\mathrm{\&lambda;}}_k}=\&dtri;f\left({\mathrm{\&alpha;}}_k\right)s_k+\mathrm{\&lambda;}{f}^{\&prime;\&prime;}\left({\mathrm{\&alpha;}}_k\right){s_k}^2=0$

So optimal step size can be expressed as

${\mathrm{\&lambda;}}_k=-\frac{\&dtri;f\left({\mathrm{\&alpha;}}_k\right)}{f^{\&prime;\&prime;}\left({\mathrm{\&alpha;}}_k\right)s_k}$

Like this, utilize the mode of optimum gradient method, from α ₀The edge of setting out

Direction is carried out linear search, asks optimal step size λ _K-1Thereby with the formula of optimum gradient method,

Get access to ${\mathrm{\&alpha;}}_k={\mathrm{\&alpha;}}_{k-1}-{\mathrm{\&lambda;}}_{k-1}\&dtri;\mathrm{SSE}\left({\mathrm{\&alpha;}}_{k-1}\right),k\&GreaterEqual;1;$

If $\left|\right|\&dtri;\mathrm{SSE}\left({\mathrm{\&alpha;}}_k\right)\left|\right|\&le;\mathrm{\&epsiv;},$ α then _kBe exactly approximate optimal solution, output α _kAnd turn to step 2, otherwise turn to step 1.

Step 2:, obtain described optimum smoothing factor according to the Optimization Model of described foundation.To be similar to optimum solution α exactly concretely _kBring the exponential smoothing model into, and be used for prediction.

As shown in Figure 5, be a kind of data processing equipment that the embodiment of the invention provides, this device comprises:

Information acquisition unit 501 is used to obtain historical data information;

Pretreatment unit 502 is used for described historical data information is carried out the data pre-service;

Data merge shim 503, are used for that described pretreated historical data information is carried out data and merge and fill up;

Data smoothing unit 504 is used for described data merging and the historical data information after filling up is carried out the dynamic index smoothing processing.

Wherein, described pretreatment unit comprises:

The time band is divided subelement, is used for that described historical data information is carried out the time band and divides;

Merge the syndrome unit, be used for merging verification according to the time band of dividing;

Unusual syndrome unit is used for carrying out unusual verification with described through the time band that merges verification, provides unusual check results;

The abnormality value removing subelement is used for according to described unusual check results abnormal data being rejected.

Described data merge shim, comprising:

Data message receives subelement, is used to receive the described historical data information of rejecting through abnormal data;

Data merge subelement, are used for the data in the described identical moment of historical data information are merged processing;

The data check subelement, whether the data that are used to detect after described merging is handled exist no datat on the moment point;

The data filling subelement if be used for existing no datat on the moment point, then carry out data filling and handles.

Described data merge shim, comprising:

Data message receives subelement, is used to receive the described historical data information of rejecting through abnormal data;

Data merge subelement, are used for the data in the described identical moment of historical data information are merged processing;

The data check subelement, whether the data that are used to detect after described merging is handled exist no datat on the moment point;

The data filling subelement if be used for existing no datat on the moment point, then carry out data filling and handles.

Described data smoothing unit comprises:

Data message receives subelement, is used to receive the described historical data information of filling up processing through number;

The parameter acquiring subelement is used to obtain the initial value of smoothing factor, exponential smoothing initial value and control accuracy;

The exponential smoothing value is obtained subelement, is used for initial value and described exponential smoothing initial value according to described smoothing factor, obtains the exponential smoothing numerical value of described next moment point of exponential smoothing initial value;

Optimum smoothing factor judgment sub-unit is used for the exponential smoothing data and the control accuracy that get access to according to described, judges whether optimum smoothing factor of described smoothing factor;

The smoothing processing subelement is used for if optimum smoothing factor is then proceeded the exponential smoothing of next moment point and handled;

Smoothing factor search subelement is used for if not optimum smoothing factor, then carries out the smoothing factor search.

It should be noted that described smoothing factor search subelement, may further include:

Optimization Model is set up subdivision, is used to described smoothing factor to set up squared prediction error and minimum Optimization Model;

Optimum smoothing factor obtains subdivision, is used for the Optimization Model according to described foundation, obtains described optimum smoothing factor.

Data processing method that the embodiment of the invention provides and device are by obtaining historical data information; Described historical data information is carried out the data pre-service; Described pretreated historical data information is carried out data to be merged and fills up; Described data are merged and fill up after historical data information carry out the dynamic index smoothing processing.Compared with prior art, the dynamic index smoothing processing of employing of the present invention can be so that precision of prediction be higher and can satisfy the needs of actual prediction.

Through the above description of the embodiments, one of ordinary skill in the art will appreciate that: realize that all or part of step in the foregoing description method is to instruct relevant hardware to finish by program, described program can be stored in the computer read/write memory medium, this program is when carrying out, comprise step as above-mentioned method embodiment, described storage medium, as: FLASH, ROM/RAM, magnetic disc, CD etc.

The above; only be the specific embodiment of the present invention, but protection scope of the present invention is not limited thereto, anyly is familiar with those skilled in the art in the technical scope that the present invention discloses; can expect easily changing or replacing, all should be encompassed within protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion by described protection domain with claim.

Claims (10)

1. a data processing method is characterized in that, comprising:

Obtain historical data information;

Described historical data information is carried out the data pre-service;

Described pretreated historical data information is carried out data to be merged and fills up;

Described data are merged and fill up after historical data information carry out the dynamic index smoothing processing.

2. data processing method according to claim 1 is characterized in that, described historical data information is carried out the pretreated step of data, comprising:

Described historical data information is carried out the time band divides;

Time band according to dividing merges verification;

Carry out unusual verification with described through the time band that merges verification, provide unusual check results;

According to described unusual check results, abnormal data is rejected.

3. data processing method according to claim 2 is characterized in that, described pretreated historical data information is carried out the step that data merge and fill up, and comprising:

Receive the described historical data information of rejecting through abnormal data;

The data in the identical moment in the described historical data information are merged processing;

Whether the data that detect after described merging is handled exist no datat on the moment point;

If there is no datat on the moment point, then carries out data filling and handle.

4. data processing method according to claim 3 is characterized in that, described data are merged and fill up after historical data information carry out the step of dynamic index smoothing processing, comprising:

Receive the described historical data information of filling up processing through number;

Obtain the initial value of smoothing factor, exponential smoothing initial value and control accuracy;

According to the initial value and the described exponential smoothing initial value of described smoothing factor, obtain the exponential smoothing numerical value of described next moment point of exponential smoothing initial value;

According to described exponential smoothing data and the control accuracy that gets access to, judge whether optimum smoothing factor of described smoothing factor;

If optimum smoothing factor is then proceeded the exponential smoothing of next moment point and is handled;

If not optimum smoothing factor, then carry out the smoothing factor search.

5. data processing method according to claim 4 is characterized in that, the step of described smoothing factor search comprises:

For described smoothing factor is set up squared prediction error and minimum Optimization Model;

According to the Optimization Model of described foundation, obtain described optimum smoothing factor.

6. a data processing equipment is characterized in that,

Information acquisition unit is used to obtain historical data information;

Pretreatment unit is used for described historical data information is carried out the data pre-service;

Data merge shim, are used for that described pretreated historical data information is carried out data and merge and fill up;

The data smoothing unit is used for described data merging and the historical data information after filling up is carried out the dynamic index smoothing processing.

7. data processing equipment according to claim 6 is characterized in that, described pretreatment unit comprises:

The time band is divided subelement, is used for that described historical data information is carried out the time band and divides;

Merge the syndrome unit, be used for merging verification according to the time band of dividing;

Unusual syndrome unit is used for carrying out unusual verification with described through the time band that merges verification, provides unusual check results;

The abnormality value removing subelement is used for according to described unusual check results abnormal data being rejected.

8. data processing equipment according to claim 7 is characterized in that, described data merge shim, comprising:

Data message receives subelement, is used to receive the described historical data information of rejecting through abnormal data;

Data merge subelement, are used for the data in the described identical moment of historical data information are merged processing;

The data check subelement, whether the data that are used to detect after described merging is handled exist no datat on the moment point;

The data filling subelement if be used for existing no datat on the moment point, then carry out data filling and handles.

9. data processing equipment according to claim 8 is characterized in that, described data smoothing unit comprises:

Data message receives subelement, is used to receive the described historical data information of filling up processing through number;

The parameter acquiring subelement is used to obtain the initial value of smoothing factor, exponential smoothing initial value and control accuracy;

The exponential smoothing value is obtained subelement, is used for initial value and described exponential smoothing initial value according to described smoothing factor, obtains the exponential smoothing numerical value of described next moment point of exponential smoothing initial value;

Optimum smoothing factor judgment sub-unit is used for the exponential smoothing data and the control accuracy that get access to according to described, judges whether optimum smoothing factor of described smoothing factor;

The smoothing processing subelement is used for if optimum smoothing factor is then proceeded the exponential smoothing of next moment point and handled;

Smoothing factor search subelement is used for if not optimum smoothing factor, then carries out the smoothing factor search.

10. data processing equipment according to claim 9 is characterized in that, described smoothing factor search subelement comprises:

Optimization Model is set up subdivision, is used to described smoothing factor to set up squared prediction error and minimum Optimization Model;

Optimum smoothing factor obtains subdivision, is used for the Optimization Model according to described foundation, obtains described optimum smoothing factor.

Images