CN106126739A - A kind of device processing business association data - Google Patents

Abstract

The invention discloses a kind of device processing business association data, module is excavated including data quality management module and useful data, wherein quality management module includes that preliminary treatment submodule, data describe submodule, quality testing submodule and data quality grading management submodule, and useful data excavates module and includes that data prediction submodule, useful data build submodule, useful data correction submodule and useful data layer digging submodule.

Description

A kind of device processing business association data

Technical field

The present invention relates to business datum field, be specifically related to a kind of device processing business association data.

Background technology

Data are exactly numerical value, and namely we are by the result observed, test or calculate.Data have a variety of, Simple is exactly numeral.Data can also be word, image, sound etc..Data may be used for scientific research, design, verification etc.. Data background is recipient's information preparation for particular data, i.e. understands the rule of phy symbol sequence as recipient, and knows When each symbol in road and the directivity target of symbol combination or implication, just can obtain the information of one group of data institute load.Data As the carrier of information, certainly want the main information comprised in analytical data, and the principal character of analytical data.Data are load Or the phy symbol by certain regularly arranged combination of record information.

In the data message currently used, having substantial portion of data is to be issued by manager, and root Modify by manager according to the suggestion of user or the demand of manager self, for the magnanimity information of this part, as What can preferably carry out quality management and excavation, the most therefrom finds useful information, is one and needs solution badly Problem.

Summary of the invention

For the problems referred to above, the present invention provides a kind of device processing business association data.

The purpose of the present invention realizes by the following technical solutions:

A kind of device processing business association data, is characterized in that, digs including data quality management module and useful data Pick module, wherein quality management module include preliminary treatment submodule, data describe submodule, quality testing submodule and Quality of data administration by different levels submodule, useful data excavates module and includes that data prediction submodule, useful data build submodule Block, useful data correction submodule and useful data layer digging submodule；

Preliminary treatment submodule, for being acquired business association data, is characterized in that, comprise the following steps:

The data item multiple to be collected belonging to same business is placed by D1 continuously, and total the waiting belonging to different business is adopted Collection data item is placed between the data item that two kinds of business are the most corresponding；

D2 uses some collecting thread groups to start the every of data acquisition sequential scheduling data item queue placement, completes The data acquisition of business association.

Preferably, it is characterized in that, described D1 comprises the following steps:

D1 is by each data collection service respectively corresponding gatherer process, and multiple gathers identical for collection period Journey is divided in same gatherer process queue；

Multiple gatherer processes that each gatherer process queue is comprised by d2 have relation according to the data item between gatherer process It is acquired process sequence；

Each gatherer process is had relation according to the data item between this gatherer process and close gatherer process and carries out this by d3 In gatherer process, data item sequence, obtains final data item queue；

Wherein, the most corresponding collecting thread group of described each gatherer process queue.

Preferably, it is characterised in that described data item is collected smallest standalone unit.

Preferably,

(1) data describe submodule

The attribute of attribute and data influencer by introducing data itself describes data, the attribute number of data itself According to size, date created, comprise picture number, related data amount represents, wherein, related data amount be current data point to other The summation of other data of data and sensing current data；The attribute of data influence person influencer network clustering coefficientCarry out table Show,Obtained by following methods:

Building data influence person and describe network, for each data, influencer includes multiple user and a pipe Reason person, each of which influencer all represents a node, and user may browse through data, it is also possible to data propose the suggestion of amendment, And data both can have been modified by manager voluntarily, it is also possible to modify according to user's suggestion,

Then influencer network clustering coefficientIt is defined as:

$\stackrel{\‾}{K}=\frac{{\mathrm{m\σ}}_1+{\mathrm{l\σ}}_2+n({\mathrm{\δ}}_1\×{\mathrm{\σ}}_3+{\mathrm{\δ}}_2\×{\mathrm{\σ}}_4)}{m+l+n}\×\sqrt{1-{\left(\frac{m-l}{m}\right)}^3}$

In formula, σ₁Representing that user often browses the factor of influence that a secondary data applies, m represents that user browses total degree；σ₂Represent User often proposes the factor of influence that suggestion for revision applies, and l represents that user advises total degree；σ₃Represent that manager is often certainly The factor of influence that row amendment one secondary data applies, σ₄Represent that manager often advises revising the impact that a secondary data applies according to user The factor, δ₁And δ₂It is respectively σ₃And σ₄Weights, n represents that manager revises total degree；Frequency system is revised for user Number, for representing user's satisfaction to data, this coefficient shows that the most greatly user is the most frequent to the amendment of data；

(2) quality testing submodule

Use " three grades of evaluation models " that the quality of data is evaluated, first split data into three classes according to size of data, Then its quality of data is evaluated by all other attribute in addition to size of data of synthetic data, and concrete grammar is as follows:

Sample data is divided into quality data, middle qualitative data and low quality data, if size of data is more than threshold value T₁, then these data belong to quality data, if size of data is more than threshold values T₂But it is less than threshold values T₁, then these data belong to middle matter Amount data, if size of data is less than threshold values T₂, then these data belong to low quality data, T₁＞ T₂And T₁、T₂Span be [1KB, 1MB]；Further quality data and low quality are divided into different brackets, choose all other attribute composition of data Vector, and the average of each data attribute of each grade is calculated according to sample data, set up corresponding average for each grade Vector, new data vector X=(x₁,…,x_N) represent, the mean vector of certain grade Y=(y₁,…,y_N) represent, N represents All other attribute number of data in addition to size of data, two vectorial similarities similarity function R (X, Y) represent:

$R(X,Y)=\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{x_i}{|}^2}+\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{y_i}{|}^2}$

R (X, Y) value is the least, then show that similarity is the biggest, otherwise, then similarity is the least, each data calculate respectively with not The similarity of the mean vector of ad eundem, thus confirm its credit rating；

(3) quality of data administration by different levels submodule

Data are divided into different quality grade, according to data level different pairs by after quality testing submodule According to carrying out administration by different levels；

Preferably,

(1) data prediction submodule

Data are divided into different field, determine client's desired data field according to user's request, use above-mentioned three grade High-quality High-level Data in field is screened by evaluation model, forms new tables of data K；

(2) useful data builds submodule

Through the data of pretreatment, each data fields contains different classification, introduces correlation coefficient P and screens useful number According to classification:

$P=\frac{\frac{Z_s}{Z}-\mathrm{\ρ}}{1-\mathrm{\ρ}}$

In formula, Z_sRepresent the quantity that in new data table K mono-classification, data double-way points to, i.e. for data A and B, can Point to B from A, also can point to A, Z from B and represent the related data amount in tables of data K mono-classification,Wherein N represents one The sum of data in classification；

(3) useful data correction submodule

Useful data in use, can be affected by artificial destruction and user two aspects of voting, according to this two The revised correlation coefficient of aspect is P '；Concurrently set threshold value T, and T ∈ (0,0.1], if P ' is ＞ T, then show that this classification is to have Use data；When qualified useful data cannot be obtained from quality data, successively at middle qualitative data and low quality number Qualified useful data is made a look up according to, and after all data search, if the P ' finally given is maximum Value less than T, although or the maximum of P ' more than T but its absolute value with the difference of threshold values T less than setting value C, show nothing Although method finds useful data or can find useful data but the useful data degree of association obtained is already below expection, then Now automatically manager is sent prompting, revise or increase related data；Take C=T/5；

(4) useful data layer digging module

First scan data table K, it is assumed that maximum and the minima of P ' are respectively P '_maxWith P '_min, tables of data K is split BecomeIndividual Non-overlapping Domain, P mining goes out Local frequent itemset, and wherein int is bracket function；Then profit Use priori character, connect Local frequent itemset and obtain overall candidate；Scanning K counts the reality of each candidate and props up again Degree of holding is to determine global frequentItemset.

The concrete correction formula being modified according to artificial destruction and user's ballot in useful data correction submodule is:

P '=P × (1-Y) × (1+H)

In formula, Y represents the data probability by artificial destruction, and H represents that ballot user accounts for the ratio of total number of persons.

Have the beneficial effect that data are described by introducing network clustering coefficient, considered the attribute of data itself with The attribute of data influence person, improves the accuracy rate of classification, revises the introducing of coefficient of frequency by user simultaneously and reduces manually Intervene, it is achieved that the target of the efficient detection quality of data；Use three grades of evaluation models, saved memory space, improve calculating Efficiency；Use brand-new similarity function, be exaggerated the effect of bigger relative error so that credit rating more science is accurate； Introduce data correction submodule correlation coefficient is modified, it is possible to fully overcome artificial destruction and user's ballot shadow to data Ring；The association rule mining divided based on region application is combined with the classification of useful data, it is only necessary to after classifying at three grades A tables of data in carry out layer digging, only when current data table does not has satisfactory data, just can at the next one Excavating in tables of data, amount of calculation declines to a great extent, and the excavation of these data can associate useful data classification, excavates purposiveness more By force.

Accompanying drawing explanation

The invention will be further described to utilize accompanying drawing, but the embodiment in accompanying drawing does not constitute any limit to the present invention System, for those of ordinary skill in the art, on the premise of not paying creative work, it is also possible to obtain according to the following drawings Other accompanying drawing.

Fig. 1 is the structured flowchart of a kind of device processing business association data.

Reference: quality management module-1；Useful data excavates module-2；Preliminary treatment submodule-11；Data describe Submodule-12；Quality testing submodule-13；Quality of data administration by different levels submodule-14；Data prediction submodule- 21；Useful data builds submodule-22；Useful data correction submodule-23；Useful data layer digging submodule-24.

Detailed description of the invention

The invention will be further described with the following Examples.

Embodiment 1:

A kind of device processing business association data as shown in Figure 1, including data quality management module 1 and useful data Excavating module 2, wherein quality management module 1 includes that preliminary treatment submodule 11, data describe submodule 12, quality testing Submodule 13 and quality testing submodule 14, useful data excavates module 2 and includes data prediction submodule 21, useful number According to building submodule 22, useful data correction submodule 23 and useful data layer digging submodule 24.

Preliminary treatment submodule 11, for being acquired business association data, is characterized in that, comprise the following steps:

The data item multiple to be collected belonging to same business is placed by D1 continuously, and total the waiting belonging to different business is adopted Collection data item is placed between the data item that two kinds of business are the most corresponding；

D2 uses some collecting thread groups to start the every of data acquisition sequential scheduling data item queue placement, completes The data acquisition of business association.

Preferably, it is characterized in that, described D1 comprises the following steps:

D1 is by each data collection service respectively corresponding gatherer process, and multiple gathers identical for collection period Journey is divided in same gatherer process queue；

Multiple gatherer processes that each gatherer process queue is comprised by d2 have relation according to the data item between gatherer process It is acquired process sequence；

Each gatherer process is had relation according to the data item between this gatherer process and close gatherer process and carries out this by d3 In gatherer process, data item sequence, obtains final data item queue；

Wherein, the most corresponding collecting thread group of described each gatherer process queue.

Preferably, it is characterised in that described data item is collected smallest standalone unit.

Preferably,

(1) data describe submodule 12:

The attribute of attribute and data influencer by introducing data itself describes data, the attribute number of data itself According to size, date created, comprise picture number, related data amount represents, wherein, related data amount be current data point to other The summation of other data of data and sensing current data；The attribute of data influence person influencer network clustering coefficientCarry out table Show,Obtained by following methods:

Building data influence person and describe network, for each data, influencer includes multiple user and a pipe Reason person, each of which influencer all represents a node, and user may browse through data, it is also possible to data propose the suggestion of amendment, And data both can have been modified by manager voluntarily, it is also possible to modify according to user's suggestion,

Then influencer network clustering coefficientIt is defined as:

$\stackrel{\‾}{K}=\frac{{\mathrm{m\σ}}_1+{\mathrm{l\σ}}_2+n({\mathrm{\δ}}_1\×{\mathrm{\σ}}_3+{\mathrm{\δ}}_2\×{\mathrm{\σ}}_4)}{m+l+n}\×\sqrt{1-{\left(\frac{m-l}{m}\right)}^3}$

In formula, σ₁Representing that user often browses the factor of influence that a secondary data applies, m represents that user browses total degree；σ₂Represent User often proposes the factor of influence that suggestion for revision applies, and l represents that user advises total degree；σ₃Represent that manager is often certainly The factor of influence that row amendment one secondary data applies, σ₄Represent that manager often advises revising the impact that a secondary data applies according to user The factor, δ₁And δ₂It is respectively σ₃And σ₄Weights, n represents that manager revises total degree；Frequency system is revised for user Number, for representing user's satisfaction to data, this coefficient shows that the most greatly user is the most frequent to the amendment of data.

(2) quality testing submodule 13:

Use " three grades of evaluation models " that the quality of data is evaluated, first split data into three classes according to size of data, Then its quality of data is evaluated by all other attribute in addition to size of data of synthetic data, and concrete grammar is as follows:

Sample data is divided into quality data, middle qualitative data and low quality data, if size of data is more than threshold value T₁, then these data belong to quality data, if size of data is more than threshold values T₂But it is less than threshold values T₁, then these data belong to middle matter Amount data, if size of data is less than threshold values T₂, then these data belong to low quality data, T₁＞ T₂And T₁、T₂Span be [1KB, 1MB]；Further quality data and low quality are divided into different brackets, choose all other attribute composition of data Vector, and the average of each data attribute of each grade is calculated according to sample data, set up corresponding average for each grade Vector, new data vector X=(x₁,…,x_N) represent, the mean vector of certain grade Y=(y₁,…,y_N) represent, N represents All other attribute number of data in addition to size of data, two vectorial similarities similarity function R (X, Y) represent:

$R(X,Y)=\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{x_i}{|}^2}+\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{y_i}{|}^2}$

R (X, Y) value is the least, then show that similarity is the biggest, otherwise, then similarity is the least, each data calculate respectively with not The similarity of the mean vector of ad eundem, thus confirm its credit rating.

(3) quality testing submodule 14:

Data are divided into different quality grade, according to data level different pairs by after quality testing submodule According to carrying out administration by different levels.

Preferably,

(1) data prediction submodule

Data are divided into different field, determine client's desired data field according to user's request, use above-mentioned three grade High-quality High-level Data in field is screened by evaluation model, forms new tables of data K；

(2) useful data builds submodule

Through the data of pretreatment, each data fields contains different classification, introduces correlation coefficient P and screens useful number According to classification:

$P=\frac{\frac{Z_s}{Z}-\mathrm{\ρ}}{1-\mathrm{\ρ}}$

In formula, Z_sRepresent the quantity that in new data table K mono-classification, data double-way points to, i.e. for data A and B, can Point to B from A, also can point to A, Z from B and represent the related data amount in tables of data K mono-classification,Wherein N represents one The sum of data in classification；

(3) useful data correction submodule

Useful data in use, can be affected by artificial destruction and user two aspects of voting, according to this two The revised correlation coefficient of aspect is P '；Concurrently set threshold value T, and T ∈ (0,0.1], if P ' is ＞ T, then show that this classification is to have Use data；When qualified useful data cannot be obtained from quality data, successively at middle qualitative data and low quality number Qualified useful data is made a look up according to, and after all data search, if the P ' finally given is maximum Value less than T, although or the maximum of P ' more than T but its absolute value with the difference of threshold values T less than setting value C, show nothing Although method finds useful data or can find useful data but the useful data degree of association obtained is already below expection, then Now automatically manager is sent prompting, revise or increase related data；Take C=T/5；

(4) useful data layer digging module

First scan data table K, it is assumed that maximum and the minima of P ' are respectively P '_maxWith P '_min, tables of data K is split BecomeIndividual Non-overlapping Domain, P mining goes out Local frequent itemset, and wherein int is bracket function；Then profit Use priori character, connect Local frequent itemset and obtain overall candidate；Scanning K counts the reality of each candidate and props up again Degree of holding is to determine global frequentItemset.

The concrete correction formula being modified according to artificial destruction and user's ballot in useful data correction submodule is:

P '=P × (1-Y) × (1+H)

In formula, Y represents the data probability by artificial destruction, and H represents that ballot user accounts for the ratio of total number of persons.

In the present embodiment, introduce network clustering coefficient and data be described, considered the attribute of data itself with The attribute of data influence person, improves the accuracy rate of classification, revises the introducing of coefficient of frequency by user simultaneously and reduces manually Intervene, it is achieved that the target of the efficient detection quality of data；Use three grades of evaluation models, saved memory space, improve calculating Efficiency；Use brand-new similarity function, be exaggerated the effect of bigger relative error so that credit rating more science is accurate； Introduce data correction submodule correlation coefficient is modified, it is possible to fully overcome artificial destruction and user's ballot shadow to data Ringing, take C=T/5, prompting scope of data increases by 5%, but amount of calculation adds 3.7%；The association rule that will divide based on region Then excavate application to combine with the classification of useful data, it is only necessary in three grades of sorted tables of data, carry out layering dig Pick, only when current data table does not has satisfactory data, just can excavate in next tables of data, computationally intensive Width declines, and the excavation of these data can associate useful data classification, excavates purposiveness higher.

Embodiment 2:

A kind of device processing business association data as shown in Figure 1, including data quality management module 1 and useful data Excavating module 2, wherein quality management module 1 includes that preliminary treatment submodule 11, data describe submodule 12, quality testing Submodule 13 and quality testing submodule 14, useful data excavates module 2 and includes data prediction submodule 21, useful number According to building submodule 22, useful data correction submodule 23 and useful data layer digging submodule 24.

Preliminary treatment submodule 11, for being acquired business association data, is characterized in that, comprise the following steps:

The data item multiple to be collected belonging to same business is placed by D1 continuously, and total the waiting belonging to different business is adopted Collection data item is placed between the data item that two kinds of business are the most corresponding；

D2 uses some collecting thread groups to start the every of data acquisition sequential scheduling data item queue placement, completes The data acquisition of business association.

Preferably, it is characterized in that, described D1 comprises the following steps:

D1 is by each data collection service respectively corresponding gatherer process, and multiple gathers identical for collection period Journey is divided in same gatherer process queue；

Multiple gatherer processes that each gatherer process queue is comprised by d2 have relation according to the data item between gatherer process It is acquired process sequence；

Each gatherer process is had relation according to the data item between this gatherer process and close gatherer process and carries out this by d3 In gatherer process, data item sequence, obtains final data item queue；

Wherein, the most corresponding collecting thread group of described each gatherer process queue.

Preferably, it is characterised in that described data item is collected smallest standalone unit.

Preferably,

(1) data describe submodule 12:

The attribute of attribute and data influencer by introducing data itself describes data, the attribute number of data itself According to size, date created, comprise picture number, related data amount represents, wherein, related data amount be current data point to other The summation of other data of data and sensing current data；The attribute of data influence person influencer network clustering coefficientCarry out table Show,Obtained by following methods:

Building data influence person and describe network, for each data, influencer includes multiple user and a pipe Reason person, each of which influencer all represents a node, and user may browse through data, it is also possible to data propose the suggestion of amendment, And data both can have been modified by manager voluntarily, it is also possible to modify according to user's suggestion,

Then influencer network clustering coefficientIt is defined as:

$\stackrel{\‾}{K}=\frac{{\mathrm{m\σ}}_1+{\mathrm{l\σ}}_2+n({\mathrm{\δ}}_1\×{\mathrm{\σ}}_3+{\mathrm{\δ}}_2\×{\mathrm{\σ}}_4)}{m+l+n}\×\sqrt{1-{\left(\frac{m-l}{m}\right)}^3}$

In formula, σ₁Representing that user often browses the factor of influence that a secondary data applies, m represents that user browses total degree；σ₂Represent User often proposes the factor of influence that suggestion for revision applies, and l represents that user advises total degree；σ₃Represent that manager is often certainly The factor of influence that row amendment one secondary data applies, σ₄Represent that manager often advises revising the impact that a secondary data applies according to user The factor, δ₁And δ₂It is respectively σ₃And σ₄Weights, n represents that manager revises total degree；Frequency system is revised for user Number, for representing user's satisfaction to data, this coefficient shows that the most greatly user is the most frequent to the amendment of data.

(2) quality testing submodule 13:

Use " three grades of evaluation models " that the quality of data is evaluated, first split data into three classes according to size of data, Then its quality of data is evaluated by all other attribute in addition to size of data of synthetic data, and concrete grammar is as follows:

Sample data is divided into quality data, middle qualitative data and low quality data, if size of data is more than threshold value T₁, then these data belong to quality data, if size of data is more than threshold values T₂But it is less than threshold values T₁, then these data belong to middle matter Amount data, if size of data is less than threshold values T₂, then these data belong to low quality data, T₁＞ T₂And T₁、T₂Span be [1KB, 1MB]；Further quality data and low quality are divided into different brackets, choose all other attribute composition of data Vector, and the average of each data attribute of each grade is calculated according to sample data, set up corresponding average for each grade Vector, new data vector X=(x₁,…,x_N) represent, the mean vector of certain grade Y=(y₁,…,y_N) represent, N represents All other attribute number of data in addition to size of data, two vectorial similarities similarity function R (X, Y) represent:

$R(X,Y)=\sqrt{{\Σ}_{i=1}^N|\frac{x_i-y_i}{x_i}{|}^2}+\sqrt{{\Σ}_{i=1}^N|\frac{x_i-y_i}{y_i}{|}^2}$

R (X, Y) value is the least, then show that similarity is the biggest, otherwise, then similarity is the least, each data calculate respectively with not The similarity of the mean vector of ad eundem, thus confirm its credit rating.

(3) quality testing submodule 14:

Data are divided into different quality grade, according to data level different pairs by after quality testing submodule According to carrying out administration by different levels.

Preferably,

(1) data prediction submodule

Data are divided into different field, determine client's desired data field according to user's request, use above-mentioned three grade High-quality High-level Data in field is screened by evaluation model, forms new tables of data K；

(2) useful data builds submodule

Through the data of pretreatment, each data fields contains different classification, introduces correlation coefficient P and screens useful number According to classification:

$P=\frac{\frac{Z_s}{Z}-\mathrm{\ρ}}{1-\mathrm{\ρ}}$

In formula, Z_sRepresent the quantity that in new data table K mono-classification, data double-way points to, i.e. for data A and B, can Point to B from A, also can point to A, Z from B and represent the related data amount in tables of data K mono-classification,Wherein N represents one The sum of data in classification；

(3) useful data correction submodule

Useful data in use, can be affected by artificial destruction and user two aspects of voting, according to this two The revised correlation coefficient of aspect is P '；Concurrently set threshold value T, and T ∈ (0,0.1], if P ' is ＞ T, then show that this classification is to have Use data；When qualified useful data cannot be obtained from quality data, successively at middle qualitative data and low quality number Qualified useful data is made a look up according to, and after all data search, if the P ' finally given is maximum Value less than T, although or the maximum of P ' more than T but its absolute value with the difference of threshold values T less than setting value C, show nothing Although method finds useful data or can find useful data but the useful data degree of association obtained is already below expection, then Now automatically manager is sent prompting, revise or increase related data；Take C=T/5；

(4) useful data layer digging module

First scan data table K, it is assumed that maximum and the minima of P ' are respectively P '_maxWith P '_min, tables of data K is split BecomeIndividual Non-overlapping Domain, P mining goes out Local frequent itemset, and wherein int is bracket function；Then profit Use priori character, connect Local frequent itemset and obtain overall candidate；Scanning K counts the reality of each candidate and props up again Degree of holding is to determine global frequentItemset.

The concrete correction formula being modified according to artificial destruction and user's ballot in useful data correction submodule is:

P '=P × (1-Y) × (1+H)

In formula, Y represents the data probability by artificial destruction, and H represents that ballot user accounts for the ratio of total number of persons.

In the present embodiment, introduce network clustering coefficient and data be described, considered the attribute of data itself with The attribute of data influence person, improves the accuracy rate of classification, revises the introducing of coefficient of frequency by user simultaneously and reduces manually Intervene, it is achieved that the target of the efficient detection quality of data；Use three grades of evaluation models, saved memory space, improve calculating Efficiency；Use brand-new similarity function, be exaggerated the effect of bigger relative error so that credit rating more science is accurate； Introduce data correction submodule correlation coefficient is modified, it is possible to fully overcome artificial destruction and user's ballot shadow to data Ringing, take C=T/6, prompting scope of data increases by 4%, but amount of calculation adds 3.3%；The association rule that will divide based on region Then excavate application to combine with the classification of useful data, it is only necessary in three grades of sorted tables of data, carry out layering dig Pick, only when current data table does not has satisfactory data, just can excavate in next tables of data, computationally intensive Width declines, and the excavation of these data can associate useful data classification, excavates purposiveness higher.

Embodiment 3:

A kind of device processing business association data as shown in Figure 1, including data quality management module 1 and useful data Excavating module 2, wherein quality management module 1 includes that preliminary treatment submodule 11, data describe submodule 12, quality testing Submodule 13 and quality testing submodule 14, useful data excavates module 2 and includes data prediction submodule 21, useful number According to building submodule 22, useful data correction submodule 23 and useful data layer digging submodule 24.

Preliminary treatment submodule 11, for being acquired business association data, is characterized in that, comprise the following steps:

The data item multiple to be collected belonging to same business is placed by D1 continuously, and total the waiting belonging to different business is adopted Collection data item is placed between the data item that two kinds of business are the most corresponding；

D2 uses some collecting thread groups to start the every of data acquisition sequential scheduling data item queue placement, completes The data acquisition of business association.

Preferably, it is characterized in that, described D1 comprises the following steps:

D1 is by each data collection service respectively corresponding gatherer process, and multiple gathers identical for collection period Journey is divided in same gatherer process queue；

Multiple gatherer processes that each gatherer process queue is comprised by d2 have relation according to the data item between gatherer process It is acquired process sequence；

Each gatherer process is had relation according to the data item between this gatherer process and close gatherer process and carries out this by d3 In gatherer process, data item sequence, obtains final data item queue；

Wherein, the most corresponding collecting thread group of described each gatherer process queue.

Preferably, it is characterised in that described data item is collected smallest standalone unit.

Preferably,

(1) data describe submodule 12:

The attribute of attribute and data influencer by introducing data itself describes data, the attribute number of data itself According to size, date created, comprise picture number, related data amount represents, wherein, related data amount be current data point to other The summation of other data of data and sensing current data；The attribute of data influence person influencer network clustering coefficientCarry out table Show,Obtained by following methods:

Building data influence person and describe network, for each data, influencer includes multiple user and a pipe Reason person, each of which influencer all represents a node, and user may browse through data, it is also possible to data propose the suggestion of amendment, And data both can have been modified by manager voluntarily, it is also possible to modify according to user's suggestion,

Then influencer network clustering coefficientIt is defined as:

$\stackrel{\‾}{K}=\frac{{\mathrm{m\σ}}_1+{\mathrm{l\σ}}_2+n({\mathrm{\δ}}_1\×{\mathrm{\σ}}_3+{\mathrm{\δ}}_2\×{\mathrm{\σ}}_4)}{m+l+n}\×\sqrt{1-{\left(\frac{m-l}{m}\right)}^3}$

In formula, σ₁Representing that user often browses the factor of influence that a secondary data applies, m represents that user browses total degree；σ₂Represent User often proposes the factor of influence that suggestion for revision applies, and l represents that user advises total degree；σ₃Represent that manager is often certainly The factor of influence that row amendment one secondary data applies, σ₄Represent that manager often advises revising the impact that a secondary data applies according to user The factor, δ₁And δ₂It is respectively σ₃And σ₄Weights, n represents that manager revises total degree；Frequency system is revised for user Number, for representing user's satisfaction to data, this coefficient shows that the most greatly user is the most frequent to the amendment of data.

(2) quality testing submodule 13:

Use " three grades of evaluation models " that the quality of data is evaluated, first split data into three classes according to size of data, Then its quality of data is evaluated by all other attribute in addition to size of data of synthetic data, and concrete grammar is as follows:

Sample data is divided into quality data, middle qualitative data and low quality data, if size of data is more than threshold value T₁, then these data belong to quality data, if size of data is more than threshold values T₂But it is less than threshold values T₁, then these data belong to middle matter Amount data, if size of data is less than threshold values T₂, then these data belong to low quality data, T₁＞ T₂And T₁、T₂Span be [1KB, 1MB]；Further quality data and low quality are divided into different brackets, choose all other attribute composition of data Vector, and the average of each data attribute of each grade is calculated according to sample data, set up corresponding average for each grade Vector, new data vector X=(x₁,…,x_N) represent, the mean vector of certain grade Y=(y₁,…,y_N) represent, N represents All other attribute number of data in addition to size of data, two vectorial similarities similarity function R (X, Y) represent:

$R(X,Y)=\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{x_i}{|}^2}+\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{y_i}{|}^2}$

R (X, Y) value is the least, then show that similarity is the biggest, otherwise, then similarity is the least, each data calculate respectively with not The similarity of the mean vector of ad eundem, thus confirm its credit rating.

(3) quality testing submodule 14:

Data are divided into different quality grade, according to data level different pairs by after quality testing submodule According to carrying out administration by different levels.

Preferably,

(1) data prediction submodule

Data are divided into different field, determine client's desired data field according to user's request, use above-mentioned three grade High-quality High-level Data in field is screened by evaluation model, forms new tables of data K；

(2) useful data builds submodule

Through the data of pretreatment, each data fields contains different classification, introduces correlation coefficient P and screens useful number According to classification:

$P=\frac{\frac{Z_s}{Z}-\mathrm{\ρ}}{1-\mathrm{\ρ}}$

In formula, Z_sRepresent the quantity that in new data table K mono-classification, data double-way points to, i.e. for data A and B, can Point to B from A, also can point to A, Z from B and represent the related data amount in tables of data K mono-classification,Wherein N represents one The sum of data in classification；

(3) useful data correction submodule

Useful data in use, can be affected by artificial destruction and user two aspects of voting, according to this two The revised correlation coefficient of aspect is P '；Concurrently set threshold value T, and T ∈ (0,0.1], if P ' is ＞ T, then show that this classification is to have Use data；When qualified useful data cannot be obtained from quality data, successively at middle qualitative data and low quality number Qualified useful data is made a look up according to, and after all data search, if the P ' finally given is maximum Value less than T, although or the maximum of P ' more than T but its absolute value with the difference of threshold values T less than setting value C, show nothing Although method finds useful data or can find useful data but the useful data degree of association obtained is already below expection, then Now automatically manager is sent prompting, revise or increase related data；Take C=T/5；

(4) useful data layer digging module

First scan data table K, it is assumed that maximum and the minima of P ' are respectively P '_maxWith P '_min, tables of data K is split BecomeIndividual Non-overlapping Domain, P mining goes out Local frequent itemset, and wherein int is bracket function；Then profit Use priori character, connect Local frequent itemset and obtain overall candidate；Scanning K counts the reality of each candidate and props up again Degree of holding is to determine global frequentItemset.

The concrete correction formula being modified according to artificial destruction and user's ballot in useful data correction submodule is:

P '=P × (1-Y) × (1+H)

In formula, Y represents the data probability by artificial destruction, and H represents that ballot user accounts for the ratio of total number of persons.

In the present embodiment, introduce network clustering coefficient and data be described, considered the attribute of data itself with The attribute of data influence person, improves the accuracy rate of classification, revises the introducing of coefficient of frequency by user simultaneously and reduces manually Intervene, it is achieved that the target of the efficient detection quality of data；Use three grades of evaluation models, saved memory space, improve calculating Efficiency；Use brand-new similarity function, be exaggerated the effect of bigger relative error so that credit rating more science is accurate； Introduce data correction submodule correlation coefficient is modified, it is possible to fully overcome artificial destruction and user's ballot shadow to data Ringing, take C=T/7, prompting scope of data increases by 3.5%, but amount of calculation adds 3%；The association rule that will divide based on region Then excavate application to combine with the classification of useful data, it is only necessary in three grades of sorted tables of data, carry out layering dig Pick, only when current data table does not has satisfactory data, just can excavate in next tables of data, computationally intensive Width declines, and the excavation of these data can associate useful data classification, excavates purposiveness higher.

Embodiment 4:

A kind of device processing business association data as shown in Figure 1, including data quality management module 1 and useful data Excavating module 2, wherein quality management module 1 includes that preliminary treatment submodule 11, data describe submodule 12, quality testing Submodule 13 and quality testing submodule 14, useful data excavates module 2 and includes data prediction submodule 21, useful number According to building submodule 22, useful data correction submodule 23 and useful data layer digging submodule 24.

Preliminary treatment submodule 11, for being acquired business association data, is characterized in that, comprise the following steps:

The data item multiple to be collected belonging to same business is placed by D1 continuously, and total the waiting belonging to different business is adopted Collection data item is placed between the data item that two kinds of business are the most corresponding；

D2 uses some collecting thread groups to start the every of data acquisition sequential scheduling data item queue placement, completes The data acquisition of business association.

Preferably, it is characterized in that, described D1 comprises the following steps:

D1 is by each data collection service respectively corresponding gatherer process, and multiple gathers identical for collection period Journey is divided in same gatherer process queue；

Multiple gatherer processes that each gatherer process queue is comprised by d2 have relation according to the data item between gatherer process It is acquired process sequence；

Each gatherer process is had relation according to the data item between this gatherer process and close gatherer process and carries out this by d3 In gatherer process, data item sequence, obtains final data item queue；

Wherein, the most corresponding collecting thread group of described each gatherer process queue.

Preferably, it is characterised in that described data item is collected smallest standalone unit.

Preferably,

(1) data describe submodule 12:

The attribute of attribute and data influencer by introducing data itself describes data, the attribute number of data itself According to size, date created, comprise picture number, related data amount represents, wherein, related data amount be current data point to other The summation of other data of data and sensing current data；The attribute of data influence person influencer network clustering coefficientCarry out table Show,Obtained by following methods:

Building data influence person and describe network, for each data, influencer includes multiple user and a pipe Reason person, each of which influencer all represents a node, and user may browse through data, it is also possible to data propose the suggestion of amendment, And data both can have been modified by manager voluntarily, it is also possible to modify according to user's suggestion,

Then influencer network clustering coefficientIt is defined as:

$\stackrel{\‾}{K}=\frac{{\mathrm{m\σ}}_1+{\mathrm{l\σ}}_2+n({\mathrm{\δ}}_1\×{\mathrm{\σ}}_3+{\mathrm{\δ}}_2\×{\mathrm{\σ}}_4)}{m+l+n}\×\sqrt{1-{\left(\frac{m-l}{m}\right)}^3}$

In formula, σ₁Representing that user often browses the factor of influence that a secondary data applies, m represents that user browses total degree；σ₂Represent User often proposes the factor of influence that suggestion for revision applies, and l represents that user advises total degree；σ₃Represent that manager is often certainly The factor of influence that row amendment one secondary data applies, σ₄Represent that manager often advises revising the impact that a secondary data applies according to user The factor, δ₁And δ₂It is respectively σ₃And σ₄Weights, n represents that manager revises total degree；Frequency system is revised for user Number, for representing user's satisfaction to data, this coefficient shows that the most greatly user is the most frequent to the amendment of data.

(2) quality testing submodule 13:

Use " three grades of evaluation models " that the quality of data is evaluated, first split data into three classes according to size of data, Then its quality of data is evaluated by all other attribute in addition to size of data of synthetic data, and concrete grammar is as follows:

Sample data is divided into quality data, middle qualitative data and low quality data, if size of data is more than threshold value T₁, then these data belong to quality data, if size of data is more than threshold values T₂But it is less than threshold values T₁, then these data belong to middle matter Amount data, if size of data is less than threshold values T₂, then these data belong to low quality data, T₁＞ T₂And T₁、T₂Span be [1KB, 1MB]；Further quality data and low quality are divided into different brackets, choose all other attribute composition of data Vector, and the average of each data attribute of each grade is calculated according to sample data, set up corresponding average for each grade Vector, new data vector X=(x₁,…,x_N) represent, the mean vector of certain grade Y=(y₁,…,y_N) represent, N represents All other attribute number of data in addition to size of data, two vectorial similarities similarity function R (X, Y) represent:

$R(X,Y)=\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{x_i}{|}^2}+\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{y_i}{|}^2}$

R (X, Y) value is the least, then show that similarity is the biggest, otherwise, then similarity is the least, each data calculate respectively with not The similarity of the mean vector of ad eundem, thus confirm its credit rating.

(3) quality testing submodule 14:

Data are divided into different quality grade, according to data level different pairs by after quality testing submodule According to carrying out administration by different levels.

Preferably,

(1) data prediction submodule

Data are divided into different field, determine client's desired data field according to user's request, use above-mentioned three grade High-quality High-level Data in field is screened by evaluation model, forms new tables of data K；

(2) useful data builds submodule

Through the data of pretreatment, each data fields contains different classification, introduces correlation coefficient P and screens useful number According to classification:

$P=\frac{\frac{Z_s}{Z}-\mathrm{\ρ}}{1-\mathrm{\ρ}}$

In formula, Z_sRepresent the quantity that in new data table K mono-classification, data double-way points to, i.e. for data A and B, can Point to B from A, also can point to A, Z from B and represent the related data amount in tables of data K mono-classification,Wherein N represents one The sum of data in classification；

(3) useful data correction submodule

Useful data in use, can be affected by artificial destruction and user two aspects of voting, according to this two The revised correlation coefficient of aspect is P '；Concurrently set threshold value T, and T ∈ (0,0.1], if P ' is ＞ T, then show that this classification is to have Use data；When qualified useful data cannot be obtained from quality data, successively at middle qualitative data and low quality number Qualified useful data is made a look up according to, and after all data search, if the P ' finally given is maximum Value less than T, although or the maximum of P ' more than T but its absolute value with the difference of threshold values T less than setting value C, show nothing Although method finds useful data or can find useful data but the useful data degree of association obtained is already below expection, then Now automatically manager is sent prompting, revise or increase related data；Take C=T/5；

(4) useful data layer digging module

First scan data table K, it is assumed that maximum and the minima of P ' are respectively P '_maxWith P '_min, tables of data K is split BecomeIndividual Non-overlapping Domain, P mining goes out Local frequent itemset, and wherein int is bracket function；Then profit Use priori character, connect Local frequent itemset and obtain overall candidate；Scanning K counts the reality of each candidate and props up again Degree of holding is to determine global frequentItemset.

The concrete correction formula being modified according to artificial destruction and user's ballot in useful data correction submodule is:

P '=P × (1-Y) × (1+H)

In formula, Y represents the data probability by artificial destruction, and H represents that ballot user accounts for the ratio of total number of persons.

In the present embodiment, introduce network clustering coefficient and data be described, considered the attribute of data itself with The attribute of data influence person, improves the accuracy rate of classification, revises the introducing of coefficient of frequency by user simultaneously and reduces manually Intervene, it is achieved that the target of the efficient detection quality of data；Use three grades of evaluation models, saved memory space, improve calculating Efficiency；Use brand-new similarity function, be exaggerated the effect of bigger relative error so that credit rating more science is accurate； Introduce data correction submodule correlation coefficient is modified, it is possible to fully overcome artificial destruction and user's ballot shadow to data Ringing, take C=T/8, prompting scope of data increases by 3%, but amount of calculation adds 2.7%；The association rule that will divide based on region Then excavate application to combine with the classification of useful data, it is only necessary in three grades of sorted tables of data, carry out layering dig Pick, only when current data table does not has satisfactory data, just can excavate in next tables of data, computationally intensive Width declines, and the excavation of these data can associate useful data classification, excavates purposiveness higher.

Embodiment 5:

A kind of device processing business association data as shown in Figure 1, including data quality management module 1 and useful data Excavating module 2, wherein quality management module 1 includes that preliminary treatment submodule 11, data describe submodule 12, quality testing Submodule 13 and quality testing submodule 14, useful data excavates module 2 and includes data prediction submodule 21, useful number According to building submodule 22, useful data correction submodule 23 and useful data layer digging submodule 24.

Preliminary treatment submodule 11, for being acquired business association data, is characterized in that, comprise the following steps:

The data item multiple to be collected belonging to same business is placed by D1 continuously, and total the waiting belonging to different business is adopted Collection data item is placed between the data item that two kinds of business are the most corresponding；

D2 uses some collecting thread groups to start the every of data acquisition sequential scheduling data item queue placement, completes The data acquisition of business association.

Preferably, it is characterized in that, described D1 comprises the following steps:

D1 is by each data collection service respectively corresponding gatherer process, and multiple gathers identical for collection period Journey is divided in same gatherer process queue；

Multiple gatherer processes that each gatherer process queue is comprised by d2 have relation according to the data item between gatherer process It is acquired process sequence；

Each gatherer process is had relation according to the data item between this gatherer process and close gatherer process and carries out this by d3 In gatherer process, data item sequence, obtains final data item queue；

Wherein, the most corresponding collecting thread group of described each gatherer process queue.

Preferably, it is characterised in that described data item is collected smallest standalone unit.

Preferably,

(1) data describe submodule 12:

The attribute of attribute and data influencer by introducing data itself describes data, the attribute number of data itself According to size, date created, comprise picture number, related data amount represents, wherein, related data amount be current data point to other The summation of other data of data and sensing current data；The attribute of data influence person influencer network clustering coefficientCarry out table Show,Obtained by following methods:

Building data influence person and describe network, for each data, influencer includes multiple user and a pipe Reason person, each of which influencer all represents a node, and user may browse through data, it is also possible to data propose the suggestion of amendment, And data both can have been modified by manager voluntarily, it is also possible to modify according to user's suggestion,

Then influencer network clustering coefficientIt is defined as:

$\stackrel{\‾}{K}=\frac{{\mathrm{m\σ}}_1+{\mathrm{l\σ}}_2+n({\mathrm{\δ}}_1\×{\mathrm{\σ}}_3+{\mathrm{\δ}}_2\×{\mathrm{\σ}}_4)}{m+l+n}\×\sqrt{1-{\left(\frac{m-l}{m}\right)}^3}$

In formula, σ₁Representing that user often browses the factor of influence that a secondary data applies, m represents that user browses total degree；σ₂Represent User often proposes the factor of influence that suggestion for revision applies, and l represents that user advises total degree；σ₃Represent that manager is often certainly The factor of influence that row amendment one secondary data applies, σ₄Represent that manager often advises revising the impact that a secondary data applies according to user The factor, δ₁And δ₂It is respectively σ₃And σ₄Weights, n represents that manager revises total degree；Frequency system is revised for user Number, for representing user's satisfaction to data, this coefficient shows that the most greatly user is the most frequent to the amendment of data.

(2) quality testing submodule 13:

Use " three grades of evaluation models " that the quality of data is evaluated, first split data into three classes according to size of data, Then its quality of data is evaluated by all other attribute in addition to size of data of synthetic data, and concrete grammar is as follows:

Sample data is divided into quality data, middle qualitative data and low quality data, if size of data is more than threshold value T₁, then these data belong to quality data, if size of data is more than threshold values T₂But it is less than threshold values T₁, then these data belong to middle matter Amount data, if size of data is less than threshold values T₂, then these data belong to low quality data, T₁＞ T₂And T₁、T₂Span be [1KB, 1MB]；Further quality data and low quality are divided into different brackets, choose all other attribute composition of data Vector, and the average of each data attribute of each grade is calculated according to sample data, set up corresponding average for each grade Vector, new data vector X=(x₁,…,x_N) represent, the mean vector of certain grade Y=(y₁,…,y_N) represent, N represents All other attribute number of data in addition to size of data, two vectorial similarities similarity function R (X, Y) represent:

$R(X,Y)=\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{x_i}{|}^2}+\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{y_i}{|}^2}$

R (X, Y) value is the least, then show that similarity is the biggest, otherwise, then similarity is the least, each data calculate respectively with not The similarity of the mean vector of ad eundem, thus confirm its credit rating.

(3) quality testing submodule 14:

Data are divided into different quality grade, according to data level different pairs by after quality testing submodule According to carrying out administration by different levels.

Preferably,

(1) data prediction submodule

Data are divided into different field, determine client's desired data field according to user's request, use above-mentioned three grade High-quality High-level Data in field is screened by evaluation model, forms new tables of data K；

(2) useful data builds submodule

Through the data of pretreatment, each data fields contains different classification, introduces correlation coefficient P and screens useful number According to classification:

$P=\frac{\frac{Z_s}{Z}-\mathrm{\ρ}}{1-\mathrm{\ρ}}$

In formula, Z_sRepresent the quantity that in new data table K mono-classification, data double-way points to, i.e. for data A and B, can Point to B from A, also can point to A, Z from B and represent the related data amount in tables of data K mono-classification,Wherein N represents one The sum of data in classification；

(3) useful data correction submodule

Useful data in use, can be affected by artificial destruction and user two aspects of voting, according to this two The revised correlation coefficient of aspect is P '；Concurrently set threshold value T, and T ∈ (0,0.1], if P ' is ＞ T, then show that this classification is to have Use data；When qualified useful data cannot be obtained from quality data, successively at middle qualitative data and low quality number Qualified useful data is made a look up according to, and after all data search, if the P ' finally given is maximum Value less than T, although or the maximum of P ' more than T but its absolute value with the difference of threshold values T less than setting value C, show nothing Although method finds useful data or can find useful data but the useful data degree of association obtained is already below expection, then Now automatically manager is sent prompting, revise or increase related data；Take C=T/5；

(4) useful data layer digging module

First scan data table K, it is assumed that maximum and the minima of P ' are respectively P '_maxWith P '_min, tables of data K is split BecomeIndividual Non-overlapping Domain, P mining goes out Local frequent itemset, and wherein int is bracket function；Then profit Use priori character, connect Local frequent itemset and obtain overall candidate；Scanning K counts the reality of each candidate and props up again Degree of holding is to determine global frequentItemset.

The concrete correction formula being modified according to artificial destruction and user's ballot in useful data correction submodule is:

P '=P × (1-Y) × (1+H)

In formula, Y represents the data probability by artificial destruction, and H represents that ballot user accounts for the ratio of total number of persons.

In the present embodiment, introduce network clustering coefficient and data be described, considered the attribute of data itself with The attribute of data influence person, improves the accuracy rate of classification, revises the introducing of coefficient of frequency by user simultaneously and reduces manually Intervene, it is achieved that the target of the efficient detection quality of data；Use three grades of evaluation models, saved memory space, improve calculating Efficiency；Use brand-new similarity function, be exaggerated the effect of bigger relative error so that credit rating more science is accurate； Introduce data correction submodule correlation coefficient is modified, it is possible to fully overcome artificial destruction and user's ballot shadow to data Ringing, take C=T/9, prompting scope of data increases by 2.7%, but amount of calculation adds 2.5%；The association that will divide based on region Rule digging application combines with the classification of useful data, it is only necessary to carries out layering in three grades of sorted tables of data and digs Pick, only when current data table does not has satisfactory data, just can excavate in next tables of data, computationally intensive Width declines, and the excavation of these data can associate useful data classification, excavates purposiveness higher.

Last it should be noted that, above example is only in order to illustrate technical scheme, rather than the present invention is protected Protecting the restriction of scope, although having made to explain to the present invention with reference to preferred embodiment, those of ordinary skill in the art should Work as understanding, technical scheme can be modified or equivalent, without deviating from the reality of technical solution of the present invention Matter and scope.

Claims (5)

1. process a device for business association data, it is characterized in that, excavate including data quality management module and useful data Module, wherein quality management module includes that preliminary treatment submodule, data describe submodule, quality testing submodule sum According to quality grading manage submodule, useful data excavate module include data prediction submodule, useful data build submodule, Useful data correction submodule and useful data layer digging submodule；

Preliminary treatment submodule, for being acquired business association data, is characterized in that, comprise the following steps:

The data item multiple to be collected belonging to same business is placed by D1 continuously, belongs to the total number to be collected of different business It is placed between the data item that two kinds of business are the most corresponding according to item；

D2 uses some collecting thread groups to start the every of data acquisition sequential scheduling data item queue placement, finishing service The data acquisition of association.

A kind of device processing business association data the most according to claim 1, is characterized in that, described D1 includes following step Rapid:

D1 is by the most corresponding for each data collection service gatherer process, and is drawn by multiple gatherer processes identical for collection period Divide in same gatherer process queue；

Multiple gatherer processes that each gatherer process queue is comprised by d2 have relation according to the data item between gatherer process to be carried out Gatherer process sorts；

Each gatherer process is had relation according to the data item between this gatherer process and close gatherer process and carries out this collection by d3 In process, data item sequence, obtains final data item queue；

Wherein, the most corresponding collecting thread group of described each gatherer process queue.

A kind of device processing business association data the most according to claim 1, it is characterised in that described data item is quilt The smallest standalone unit gathered.

A kind of device processing business association data the most according to claim 1, is characterized in that,

(1) data describe submodule

The attribute of attribute and data influencer by introducing data itself describes data, and the attribute data of data itself are big Little, date created, comprise picture number, related data amount represents, wherein, related data amount is other data that current data is pointed to Summation with other data pointing to current data；The attribute of data influence person influencer network clustering coefficientRepresent, Obtained by following methods:

Building data influence person and describe network, for each data, influencer includes multiple user and a manager, Each of which influencer all represents a node, and user may browse through data, it is also possible to data propose the suggestion of amendment, and manages Data both can be modified by person voluntarily, it is also possible to modifies according to user's suggestion,

Then influencer network clustering coefficientIt is defined as:

$\stackrel{\‾}{K}=\frac{{\mathrm{m\σ}}_1+{\mathrm{l\σ}}_2+n({\mathrm{\δ}}_1\×{\mathrm{\σ}}_3+{\mathrm{\δ}}_2\×{\mathrm{\σ}}_4)}{m+l+n}\×\sqrt{1-{\left(\frac{m-l}{m}\right)}^3}$

In formula, σ₁Representing that user often browses the factor of influence that a secondary data applies, m represents that user browses total degree；σ₂Represent user Often proposing the factor of influence that suggestion for revision applies, l represents that user advises total degree；σ₃Represent that manager repaiies the most voluntarily Change the factor of influence that a secondary data applies, σ₄Represent manager often according to user advise revising a secondary data applies affect because of Son, δ₁And δ₂It is respectively σ₃And σ₄Weights, n represents that manager revises total degree；Coefficient of frequency is revised for user, For representing user's satisfaction to data, this coefficient shows that the most greatly user is the most frequent to the amendment of data；

(2) quality testing submodule

Use " three grades of evaluation models " that the quality of data is evaluated, first split data into three classes according to size of data, then Its quality of data is evaluated by all other attribute in addition to size of data of synthetic data, and concrete grammar is as follows:

Sample data is divided into quality data, middle qualitative data and low quality data, if size of data is more than threshold value T₁, then These data belong to quality data, if size of data is more than threshold values T₂But it is less than threshold values T₁, then these data belong to middle mass number According to, if size of data is less than threshold values T₂, then these data belong to low quality data, T₁＞ T₂And T₁、T₂Span be [1KB, 1MB]；Further quality data and low quality are divided into different brackets, choose all other attribute composition of vector of data, And the average of each data attribute according to the sample data each grade of calculating, set up corresponding mean vector for each grade, New data vector X=(x₁,…,x_N) represent, the mean vector of certain grade Y=(y₁,…,y_N) represent, N represents divisor According to all other attribute number of the outer data of size, two vectorial similarities similarity function R (X, Y) represent:

$R(X,Y)=\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{x_i}{|}^2}+\sqrt{{\mathrm{\Σ}}_{i=1}^N|\frac{x_i-y_i}{y_i}{|}^2}$

R (X, Y) value is the least, then show that similarity is the biggest, otherwise, then similarity is the least, and each data calculate respectively with the most equal The similarity of the mean vector of level, thus confirm its credit rating；

(3) quality of data administration by different levels submodule

Data are divided into different quality grade by after quality testing submodule, according to data level different pairs according to entering Row administration by different levels.

A kind of device processing business association data the most according to claim 1, is characterized in that,

(1) data prediction submodule

Data are divided into different field, determine client's desired data field according to user's request, use above-mentioned three grade to evaluate High-quality High-level Data in field is screened by model, forms new tables of data K；

(2) useful data builds submodule

Through the data of pretreatment, each data fields contains different classification, introduces correlation coefficient P screening useful data and divides Class:

$P=\frac{\frac{Z_s}{Z}-\mathrm{\ρ}}{1-\mathrm{\ρ}}$

In formula, Z_sRepresent the quantity that in new data table K mono-classification, data double-way points to, i.e. for data A and B, can refer to from A To B, also can point to A, Z from B and represent the related data amount in tables of data K mono-classification,During wherein N represents a classification The sum of data；

(3) useful data correction submodule

Useful data in use, can be affected, according to these two aspects by artificial destruction and user two aspects of voting Revised correlation coefficient is P '；Concurrently set threshold value T, and T ∈ (0,0.1], if P ' is ＞ T, then show that this classification is useful number According to；When qualified useful data cannot be obtained from quality data, successively in middle qualitative data and low quality data Make a look up qualified useful data, and after all data search, if the P ' maximum finally given is little In T, although or the maximum of P ' more than T but its absolute value with the difference of threshold values T less than setting value C, show to look for To useful data or although useful data can find but the useful data degree of association obtained is already below expection, the most now Automatically manager is sent prompting, revise or increase related data；Take C=T/5；

(4) useful data layer digging module

First scan data table K, it is assumed that maximum and the minima of P ' are respectively P '_maxWith P '_min, tables of data K is divided intoIndividual Non-overlapping Domain, P mining goes out Local frequent itemset, and wherein int is bracket function；Then utilize Priori character, connects Local frequent itemset and obtains overall candidate；Scanning K counts the actual support of each candidate again Degree is to determine global frequentItemset；

The concrete correction formula being modified according to artificial destruction and user's ballot in useful data correction submodule is:

P '=P × (1-Y) × (1+H)

In formula, Y represents the data probability by artificial destruction, and H represents that ballot user accounts for the ratio of total number of persons.