CN104035779A - Method for handling missing values during data stream decision tree classification - Google Patents

Abstract

The invention belongs to the technical field of data stream mining, and particularly relates to a method for handling missing values during data stream decision tree classification. The method includes reading data samples in data streams and updating sliding windows; updating missing handlers if the missing handlers corresponding to attributes are available when the detected attributes in the current data samples have the missing values, or adaptively selecting and creating missing handlers according to characteristics of data if the missing handlers corresponding to the attributes are not available; supplementing the missing values in the data samples by the aid of the missing handlers to obtain complete data samples, training the complete data samples according to a Hoeffding decision tree classification process and returning data stream decision tree classification results. Compared with existing methods, the method has the advantages that the method is superior in time performance, the classification accuracy of decision tree models can be sufficiently guaranteed, accordingly, the time expenditure can be reduced, the time performance can be improved, the data stream classification handling speeds can be increased, and requirements of actual data stream handling application can be met.

Description

Missing values disposal route in a kind of data stream decision tree classification

Technical field

The invention belongs to Mining Data Stream Technology field, be specifically related to the missing values disposal route in a kind of data stream decision tree classification.

Background technology

Along with the arrival of large data age, how application system at a high speed and continuously produce data stream, excavates useful information from data stream, has become the focus that technician is concerned about.Data stream Decision Tree Technologies is the important research direction during data stream is excavated, and this technology can be applied to a lot of aspects such as network invasion monitoring and credit card fraud.Can there is missing values because of reasons such as HP M, faulty sensor or manual operation errors in the data stream in reality.In data stream decision tree classification, the missing values in data stream can cause and have a strong impact on classification accuracy.But data stream can only be scanned once in mining process, cannot before mining process, take in advance to process the measure of missing values.

Document [1] is (with reference to Domingos P, Hulten G.Mining high-speed data streams[C] //Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2000:71-80.) Hoeffding Decision-Tree Method has been proposed, utilize Hoeffding to define the data sample in reason incremental learning data stream.Hoeffding Decision-Tree Method is assigned to leaf node according to the decision tree when front construction by data sample, and leaf node defines the definite optimum Split Attribute of reason according to sample information and the Hoeffding of storage, and division becomes internal node then.Dynamically construct decision tree by continuous repetition said process, until decision tree reaches stable.

Document [2] is (with reference to Yang H, Fong S.Aerial root classifiers for predicting missing values in data stream decision tree classification[C] // 2011SIAM International Conference on Data Mining (SDM2011) .2011:28-30.) ARC (Aerial Root Classifiers) method has been proposed, on the basis of Hoeffding Decision-Tree Method, increase missing values treatment mechanism.ARC method utilizes moving window to preserve up-to-date data sample, in the time disappearance property value being detected, utilizing the sample in moving window is the property value that this attribute is set up sub-classifier prediction disappearance, and then constructs decision tree according to Hoeffding Decision-Tree Method.ARC method has designed update mechanism simultaneously, for solving the problem that sub-classifier is out-of-date.Attribute metric during according to decision tree split vertexes is each attribute assignment weight, is added the error rate of the corresponding sub-classifier of each attribute, thereby obtains overall error rate by weight.In the time that overall error rate exceedes default threshold value, select successively the attribute of weight maximum to upgrade its corresponding sub-classifier, until overall error rate meets the demands.

But, the time performance of ARC method significantly declines in the time that the characteristic attribute of data sample is more, and the important measurement index of time performance during to be data stream excavate, and has therefore had a strong impact on the time performance of ARC method, transfer efficiency is reduced, affected the using value in reality.

Summary of the invention

The technology of the present invention is dealt with problems: overcome the deficiencies in the prior art, missing values disposal route in a kind of data stream decision tree classification is provided, according to the adaptively selected missing values disposal route of data characteristics, adopt improved Bayesian Classification Model, optimize update mechanism simultaneously, thereby reduce time overhead, promote time performance, improve the classification processing speed of data stream, thereby meet the application of actual data stream processing.

Technical scheme of the present invention is: the missing values disposal route in a kind of data stream decision tree classification, the steps include:

Step 1: the data sample in reading data flow, and use the moving window W of fixed capacity to preserve the most newly arrived data sample;

Step 2: the attribute X in current data sample _iwhile there is missing values, set up or Update attribute X _icorresponding disappearance processor.If attribute X _idisappearance processor exist, skip to step 4 upgrade disappearance processor, otherwise enter step 3 set up disappearance processor;

Step 3: in calculating moving window W, similar sample is about attribute X _istandard deviation sigma (X _i), if σ is (X _i) be no more than threshold value σ _m, choice for use mode or mean value replace missing values, predict missing values otherwise set up sub-classifier.Set up disappearance processor and skip to step 5 according to the method;

Step 4: calculate the weighting total false rate E of disappearance processor, if E exceedes threshold value beta, select weight maximum and error rate e _i> β ^*disappearance processor upgrade, until E is lower than threshold value beta;

Step 5: utilize disappearance processor complementary properties X _imissing values, obtain complete data sample;

Step 6: train complete data sample according to Hoeffding Decision-Tree Method, dynamically construct decision-tree model, and attribute metric during according to decision tree division leaf node is each attribute X _iupgrade weight;

Step 7: return data stream decision tree classification result.

Described step 3 is by the standard deviation of computational data sample, adaptively selected different missing values disposal route, the missing values of attribute in supplementary data sample.

Further, for s data sample, data attribute X={X ₁, X ₂..., X _n, make x _ijrepresent attribute X _iproperty value in j sample.As attribute X _iduring for Category Attributes, sample standard deviation σ (X _i) be:

$\mathrm{\σ}\left(X_i\right)=\sqrt{\frac{1}{s-1}\underset{j=1}{\overset{s}{\mathrm{\Σ}}}\mathrm{diff}(x_{\mathrm{ij}},M_i)},$

$\mathrm{diff}(x_{\mathrm{ij}},M_i)=\begin{array}{c}\left\{\begin{array}{cc}1& (x_{\mathrm{ij}}\≠M_i)\\ 0& (x_{\mathrm{ij}}=M_i)\end{array}\right.,\end{array}$

Wherein, M _irepresent attribute X _iat the mode that calculates property value in sample; As attribute X _iduring for connection attribute, sample standard deviation σ (X _i) be:

$\mathrm{\σ}\left(X_i\right)=\sqrt{\frac{1}{s-1}\underset{j=1}{\overset{s}{\mathrm{\Σ}}}{(x_{\mathrm{ij}}-{\mathrm{\μ}}_i)}^2},$

${\mathrm{\μ}}_i=\underset{j=1}{\overset{s}{\mathrm{\Σ}}}{x}_{\mathrm{ij}},$

Wherein, μ _irepresent attribute X _iat the mean value that calculates property value in sample.

Further, σ _mfor predefined acceptable maximum sample standard deviation.As σ (X _i)≤σ _mtime, select attribute X _ithe mode M of the property value in similar sample _ior average value mu _ireplace missing values; As σ (X _i) > σ _mtime, utilize all Sample Establishing sub-classifiers in moving window W to predict missing values, and work as X _iduring for Category Attributes, set up improved Bayesian Classification Model as sub-classifier, work as X _iduring for connection attribute, set up regressive prediction model as sub-classifier.

Further, while setting up the sub-classifier of Category Attributes, select a kind of improved bayes classification method as forecast model.As the attribute X of data sample _iwhen disappearance, obtain attribute X according to bayes classification method _ieach property value x _ijposteriority conditional probability.Improved bayes classification method is selected different property value x according to this probability size _ijas disappearance attribute X _ipredicted value, instead of select the property value of posteriority conditional probability maximum.

Described step 4 lacks the weighting total false rate E of processor by calculating, judging whether needs to upgrade disappearance processor, thereby solves the out-of-date problem of disappearance processor.

Further, the attribute metric during according to decision tree split vertexes for each attribute X _icorresponding disappearance processor distribution weights omega _i:

${\mathrm{\ω}}_i=\frac{\stackrel{\‾}{G}\left(X_i\right)}{\underset{j=1}{\overset{n}{\mathrm{\Σ}}}\stackrel{\‾}{G}\left(X_j\right)},$

Be added the error rate e of each disappearance processor by weight _i, obtain total false rate E:

$E=\underset{i=1}{\overset{n}{\mathrm{\Σ}}}{\mathrm{\ω}}_i{e}_i.$

In the time that total false rate E exceedes predefined threshold value beta, need disappearance processor to upgrade.Select successively weight maximum and error rate e _i> β ^*disappearance processor upgrade, until E≤β, wherein β ^*to ensure not out-of-date threshold value and the β of disappearance processor ^*< β.

Compared with prior art, good effect of the present invention is: compared with missing values disposal route in existing data stream decision tree classification, the method that the present invention proposes has more excellent time performance, and can fully ensure the classification accuracy of decision-tree model, thereby reduction time overhead, promote time performance, improve the classification processing speed of data stream, thereby meet the application of actual data stream processing.

Brief description of the drawings

Fig. 1 is the main flow chart of the inventive method;

Fig. 2 is the process flow diagram of setting up disappearance processor;

Fig. 3 is the process flow diagram that upgrades disappearance processor.

Embodiment

Below in conjunction with Figure of description, the specific embodiment of the present invention is described in detail.

The main flow chart of the inventive method is as shown in Figure 1:

(1) adaptively selected and foundation disappearance processor

Adaptively selected and set up disappearance processor idiographic flow as shown in Figure 2, the steps include:

Step 1: the attribute X in current data sample detected _ithere is missing values;

Step 2: read all samples similar with current data sample in moving window W, calculate in similar sample about attribute X _istandard deviation sigma (X _i);

Step 3: preset σ _mfor acceptable maximum sample standard deviation, if σ is (X _i) be no more than threshold value σ _m, enter step 4, otherwise enter step 5;

Step 4: select the average method of substitution to set up disappearance processor, work as X _iduring for Category Attributes, use attribute X _ithe mode of the property value in the similar sample of moving window W replaces missing values, works as X _iduring for connection attribute, use attribute X _ithe mean value of the property value in the similar sample of moving window W replaces missing values;

Step 5: chooser sorter predicted method is set up disappearance processor, works as X _iduring for Category Attributes, set up improved Bayesian Classification Model prediction missing values, work as X _iduring for connection attribute, set up regressive prediction model and predict missing values;

In described step 3, for s data sample, data attribute X={X ₁, X ₂..., X _n, make x _ijrepresent attribute X _iproperty value in j sample.As attribute X _iduring for Category Attributes, sample standard deviation σ (X _i) be:

$\mathrm{\&sigma;}\left(X_i\right)=\sqrt{\frac{1}{s-1}\underset{j=1}{\overset{s}{\mathrm{\&Sigma;}}}\mathrm{diff}(x_{\mathrm{ij}},M_i)},$

$\mathrm{diff}(x_{\mathrm{ij}},M_i)=\begin{array}{c}\left\{\begin{array}{cc}1& (x_{\mathrm{ij}}\&NotEqual;M_i)\\ 0& (x_{\mathrm{ij}}=M_i)\end{array}\right.,\end{array}$

Wherein, M _irepresent attribute X _iat the mode that calculates the property value in sample.As attribute X _iduring for connection attribute, sample standard deviation σ (X _i) be:

$\mathrm{\&sigma;}\left(X_i\right)=\sqrt{\frac{1}{s-1}\underset{j=1}{\overset{s}{\mathrm{\&Sigma;}}}{(x_{\mathrm{ij}}-{\mathrm{\&mu;}}_i)}^2},$

${\mathrm{\&mu;}}_i=\underset{j=1}{\overset{s}{\mathrm{\&Sigma;}}}{x}_{\mathrm{ij}},$

Wherein, μ _irepresent attribute X _iat the mean value that calculates the property value in sample.

In described step 5, adopt the forecast model of a kind of improved Bayesian Classification Model as Category Attributes.As the Category Attributes X of data sample _iwhile there is missing values, set up improved Bayesian Classification Model NB _i(X') → X _ipredict attribute X _imissing values, wherein X'=X-X _i+ C, C is the classification under data sample.Suppose attribute X _ithere is v value, according to Bayesian Classification Model NB _iobtain posteriority conditional probability P (x _ij| X') j=1,2 ..., v, wherein x _ijfor attribute X _iv value.Obtaining the posteriority conditional probability P (x of each property value _ij| X') after, different property value x selected according to this probability size _ijas attribute X _ipredicted value, instead of select the property value of maximum probability.

(2) upgrade disappearance processor

Upgrade the idiographic flow of disappearance processor as shown in Figure 3, the steps include:

Step 1: attribute X _ithe error rate of corresponding disappearance processor is e _i, weight is ω _i, be added the error rate of all disappearance processors by weight, calculate the weighting total false rate E of disappearance processor;

Step 2: presetting β is maximum weighted total false rate, if E exceedes threshold value beta and has the disappearance processor not upgrading, enters step 3, otherwise enters step 4;

Step 3: preset β ^*for ensureing the not out-of-date error rate of disappearance processor, and β ^*< β, selects weight maximum and error rate to exceed β ^*disappearance processor upgrade, and again perform step 1;

Step 4: exit more new technological process.

In described step 1, the weights omega of disappearance processor _iattribute metric during according to decision tree split vertexes calculates, each attribute X _iattribute metric be attribute X _ithe weights omega of corresponding disappearance processor _ifor:

${\mathrm{\&omega;}}_i=\frac{\stackrel{\&OverBar;}{G}\left(X_i\right)}{\underset{j=1}{\overset{n}{\mathrm{\&Sigma;}}}\stackrel{\&OverBar;}{G}\left(X_j\right)},$

The weighting total false rate E of all disappearance processors is:

$E=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{\mathrm{\&omega;}}_i{e}_i.$

(3) overall design of the inventive method

Specific implementation step of the present invention is:

Step 1: the data sample in reading data flow, and use the moving window W of fixed capacity to preserve the most newly arrived data sample;

Step 2: as the attribute X in data sample _iwhile there is missing values, set up or Update attribute X _icorresponding disappearance processor.If disappearance processor exists, skip to step 4 and upgrade disappearance processor, set up disappearance processor otherwise enter step 3;

Step 3: set up disappearance processor and skip to step 5 according to flow process shown in Fig. 2 and step;

Step 4: upgrade disappearance processor according to flow process shown in Fig. 3 and step;

Step 5: utilize disappearance processor complementary properties X _imissing values, obtain complete data sample;

Step 6: train complete data sample according to Hoeffding Decision-Tree Method, dynamically construct decision-tree model, and attribute metric during according to decision tree division leaf node is each attribute X _iupgrade weight;

Step 7: return data stream decision tree classification result.

Claims (5)

1. the missing values disposal route in data stream decision tree classification, its feature is as follows at performing step:

Step 1: the data sample in reading data flow, and use the moving window W of fixed capacity to preserve the most newly arrived data sample;

Step 2: the attribute X in current data sample _iwhile there is missing values, set up or Update attribute X _icorresponding disappearance processor, if attribute X _idisappearance processor exist, skip to step 4 upgrade disappearance processor, otherwise enter step 3 set up disappearance processor;

Step 3: in calculating moving window W, similar sample is about attribute X _istandard deviation sigma (X _i), if σ is (X _i) be no more than threshold value σ _m, choice for use mode or mean value replace missing values, predict missing values otherwise set up sub-classifier, set up disappearance processor and skip to step 5 according to the method;

Step 4: calculate the weighting total false rate E of disappearance processor, if E exceedes threshold value beta, select weight maximum and error rate e _i> β ^*disappearance processor upgrade, until E is lower than threshold value beta;

Step 5: utilize disappearance processor complementary properties X _imissing values, obtain complete data sample;

Step 6: train complete data sample according to Hoeffding Decision-Tree Method, dynamically construct decision-tree model, and attribute metric during according to decision tree division leaf node is each attribute X _iupgrade weight;

Step 7: return data stream decision tree classification result.

2. the missing values disposal route in a kind of data stream decision tree classification according to claim 1, is characterized in that: in described step 3, calculate in moving window W similar sample about attribute X _istandard deviation sigma (X _i) method be:

For s data sample, data attribute X={X ₁, X ₂..., X _n, make x _ijrepresent attribute X _iproperty value in j sample, as attribute X _iduring for Category Attributes, sample standard deviation σ (X _i) be:

$\mathrm{\&sigma;}\left(X_i\right)=\sqrt{\frac{1}{s-1}\underset{j=1}{\overset{s}{\mathrm{\&Sigma;}}}\mathrm{diff}(x_{\mathrm{ij}},M_i)},$

$\mathrm{diff}(x_{\mathrm{ij}},M_i)=\begin{array}{c}\left\{\begin{array}{cc}1& (x_{\mathrm{ij}}\&NotEqual;M_i)\\ 0& (x_{\mathrm{ij}}=M_i)\end{array}\right.,\end{array}$

Wherein, M _irepresent attribute X _iat the mode that calculates property value in sample; As attribute X _iduring for connection attribute, sample standard deviation σ (X _i) be:

$\mathrm{\&sigma;}\left(X_i\right)=\sqrt{\frac{1}{s-1}\underset{j=1}{\overset{s}{\mathrm{\&Sigma;}}}{(x_{\mathrm{ij}}-{\mathrm{\&mu;}}_i)}^2},$

${\mathrm{\&mu;}}_i=\underset{j=1}{\overset{s}{\mathrm{\&Sigma;}}}{x}_{\mathrm{ij}},$

Wherein, μ _irepresent attribute X _iat the mean value that calculates property value in sample.

3. the missing values disposal route in a kind of data stream decision tree classification according to claim 1, is characterized in that: the method for setting up missing values processor in described step 3 is: preset σ _mfor acceptable maximum sample standard deviation, as σ (X _i)≤σ _mtime, select attribute X _ithe mode M of the property value in similar sample _ior average value mu _ireplace missing values; As σ (X _i) > σ _mtime, utilize all Sample Establishing sub-classifiers in moving window W to predict missing values, and work as X _iduring for Category Attributes, set up improved Bayesian Classification Model as sub-classifier, work as X _iduring for connection attribute, set up regressive prediction model as sub-classifier.

4. the missing values disposal route in a kind of data stream decision tree classification according to claim 3, it is characterized in that: describedly set up improved Bayesian Classification Model and as the method for sub-classifier be: while setting up the sub-classifier of Category Attributes, adopt a kind of improved bayes classification method as forecast model, as the attribute X of data sample _iwhen disappearance, obtain attribute X according to bayes classification method _ieach property value x _ijposteriority conditional probability, improved bayes classification method is selected different property value x according to this probability size _ijas disappearance attribute X _ipredicted value, instead of select the property value of posteriority conditional probability maximum.

5. the missing values disposal route in a kind of data stream decision tree classification according to claim 1, is characterized in that: in described step 4, specific implementation process is:

Attribute metric during according to decision tree split vertexes for each attribute X _idisappearance processor distribution weights omega _i:

${\mathrm{\&omega;}}_i=\frac{\stackrel{\&OverBar;}{G}\left(X_i\right)}{\underset{j=1}{\overset{n}{\mathrm{\&Sigma;}}}\stackrel{\&OverBar;}{G}\left(X_j\right)},$

Be added the error rate e of each disappearance processor by weight _i, obtain total false rate E:

$E=\underset{i=1}{\overset{n}{\mathrm{\&Sigma;}}}{\mathrm{\&omega;}}_i{e}_i,$

In the time that total false rate E exceedes predefined threshold value beta, need disappearance processor to upgrade, select successively weight maximum and error rate e _i> β ^*disappearance processor upgrade, until E≤β, wherein β ^*to ensure not out-of-date threshold value and the β of disappearance processor ^*< β.