CN106778795A - A kind of sorting technique and device based on incremental learning - Google Patents

Abstract

The embodiment of the invention discloses a kind of sorting technique based on incremental learning, methods described includes：Build grader and determine characteristic of division vector；According to the grader and characteristic of division vector, the data sample to increasing newly is trained；According to the relevant parameter obtained by training, the data sample is deleted and retained.Accordingly, the embodiment of the invention also discloses a kind of sorter based on incremental learning.Using the present invention, it is possible to achieve in incremental learning, useless data and the data for remaining with actively are deleted, so as to improve the adaptability and accuracy rate of cognitive computation model.

Description

A kind of sorting technique and device based on incremental learning

Technical field

The present invention relates to technical field of data processing, more particularly to a kind of sorting technique based on incremental learning and Device.

Background technology

The fast development of the emerging technologies such as cloud computing, Internet of Things promotes the scale of data just with unprecedented speed Degree increases, and the big data epoch have arrived.How timely and effectively to obtain valuable in the complex data of magnanimity The information of value, depend on one kind can Active Learning, it is and voluntarily valuable in analyze data according to demand Information, actively carries out model --- the cognitive computation model that data calculate treatment.It should be understood that, cognition is calculated Model can fast and accurately find valuable information, extract effective information and by these valuable letters Breath is organized, and provides an effective solution.

More universal cognitive computation model, SVMs (Support Vector are applied as a kind of Machine, SVM) sorting algorithm uses batch processing algorithm, and due to batch processing algorithm, need will be whole Individual data set is loaded among internal memory, and the amount of storage of internal memory is limited so that the algorithm is not suitable for big data Computing, it is impossible to be applied to real-time occasion higher.Meanwhile, can also cause the needs when grader is built Take a substantial amount of time, cause the data for reaching below not in time to cause to lose due to treatment, cause Serious consequence.

The content of the invention

The embodiment of the invention provides a kind of sorting technique based on incremental learning and device, it is possible to achieve increasing In amount study, useless data and the data for remaining with actively are deleted, so as to improve cognitive computation model Adaptability and accuracy rate.

Embodiment of the present invention first aspect provides a kind of sorting technique based on incremental learning, including：

Build grader and determine characteristic of division vector；

According to the grader and characteristic of division vector, the data sample to increasing newly is trained；

According to the relevant parameter obtained by training, the data sample is deleted and retained.

In the first possible implementation of first aspect, it is described structure grader and determine characteristic of division to Amount, including：

Determine the quantity of grader to be built；

Build each grader；

Determine the characteristic of division vector of each grader.

It is described special according to the grader and the classification in second possible implementation of first aspect Vector is levied, the data sample to increasing newly is trained, including：

1. by stochastic gradient descent SGD algorithms, newly-increased data sample subset B is randomly selected₁It is trained；

2. preliminary classification device Γ is passed through¹Judge the sample set B₁The correctness of classification, and according to judged result By the sample set B₁It is divided into test errors collection B_errCollection B correct with test_ok；

3. the test errors collection B is judged_errWhether it is empty set,

If so, new batch of data sample is then extracted by the SGD algorithms be trained,

If it is not, then by the set of supporting vector SV in original data sampleWith the sample set B₁Enter Row merges to obtain new setWith new grader Γ², and by the setIn remove the set Data sample outside remaining data sample collection B correct with the test_okMerge to obtain the grader Γ²Incremental data sample set B₁′；

Repeat above-mentioned 1., 2. and 3. three steps.

With reference to second possible implementation of first aspect, in the third possible implementation, described According to the relevant parameter obtained by training, the data sample is deleted and retained, including：

According to formula (1), formula (2) and formula (3), forgetting factor α is tried to achieve_i, wherein, α_iRepresent number I-th ratio of data sample supporting vector SV, T after being trained through T times according to sample_iRepresent total frequency of training, r_i Represent i-th number of times of the trained rear supporting vector SV of data sample, the test errors collection B_errInterior every number According to the r of sample_i=0, the correct collection B of test_okThe r of interior each data sample_i=1；

According to based on the forgetting factor α_iPrediction incremental learning mechanism, the data sample is deleted And reservation.

With reference to the third possible implementation of first aspect, in the 4th kind of possible implementation, described According to based on the forgetting factor α_iPrediction incremental learning mechanism, the data sample is deleted and retained, Including：

Tri- threshold values of β, γ and δ are set；

Compare the forgetting factor α_iWith the magnitude relationship of β, γ and δ；

According to result of the comparison, the data sample is deleted and retained.

It is described to set in the 5th kind of possible implementation with reference to the 4th kind of possible implementation of first aspect Put after tri- threshold values of β, γ and δ, also include：

After often being trained through 10 times, each data sample is tried to achieve according to formula (4) and is weighed with the error of set threshold value Value, wherein, e_iError weights are represented, P represents set threshold value；

e_i=P- α_i(1≤i≤10) (4)

Select the forgetting factor α of the error maximum weight_iAs new threshold value；

According to the forgetting factor α_i, adapt to the value of adjustment β, γ and δ.

With reference to the 4th kind of possible implementation of first aspect, in the 6th kind of possible implementation, described According to result of the comparison, the data sample is deleted and retained, including：

As the forgetting factor α_iWhen=0, retain the forgetting factor α_iCorresponding data sample；

As the ＜ α of the forgetting factor 0_iDuring ＜ β, the forgetting factor α is deleted_iCorresponding data sample；

As the forgetting factor β≤α_iDuring ＜ δ, data sample of the selection more than γ is used as data sample next time Collection is tested；

As the forgetting factor δ ＜ α_iDuring ＜ 1, by the forgetting factor α_iCorresponding data sample is used as next time Set of data samples is tested.

Embodiment of the present invention second aspect provides a kind of sorter based on incremental learning, including：

Initialization module, for building grader and determining characteristic of division vector；

Data training module, for according to the grader and characteristic of division vector, to the data for increasing newly Sample is trained；

Data processing module, for according to the relevant parameter obtained by training, being deleted to the data sample Except and retain.

In the first possible implementation of second aspect, the initialization module, specifically for：

Determine the quantity of grader to be built；

Build each grader；

Determine the characteristic of division vector of each grader.

In second possible implementation of second aspect, the data training module, specifically for：

1. by stochastic gradient descent SGD algorithms, newly-increased data sample subset B is randomly selected₁It is trained；

2. preliminary classification device Γ is passed through¹Judge the sample set B₁The correctness of classification, and according to judged result By the sample set B₁It is divided into test errors collection B_errCollection B correct with test_ok；

3. the test errors collection B is judged_errWhether it is empty set,

If so, new batch of data sample is then extracted by the SGD algorithms be trained,

If it is not, then by the set of supporting vector SV in original data sampleWith the sample set B₁Enter Row merges to obtain new setWith new grader Γ², and by the setIn remove the set Data sample outside remaining data sample collection B correct with the test_okMerge to obtain the grader Γ²Incremental data sample set B₁′；

Repeat above-mentioned 1., 2. and 3. three steps.

With reference to second possible implementation of second aspect, in the third possible implementation, the number Include according to processing module：

Parameter calculation unit, for according to formula (1), formula (2) and formula (3), trying to achieve forgetting factor α_i, wherein, α_iRepresent data sample i-th ratio of data sample supporting vector SV, T after T training_i Represent total frequency of training, r_iRepresent i-th number of times of the trained rear supporting vector SV of data sample, the survey Examination Error Set B_errThe r of interior each data sample_i=0, the correct collection B of test_okThe r of interior each data sample_i=1；

Data processing unit, for according to based on the forgetting factor α_iPrediction incremental learning mechanism, to institute Data sample is stated to be deleted and retained.

With reference to the third possible implementation of second aspect, in the 4th kind of possible implementation, the number According to processing unit, specifically for：

Tri- threshold values of β, γ and δ are set；

Compare the forgetting factor α_iWith the magnitude relationship of β, γ and δ；

According to result of the comparison, the data sample is deleted and retained.

With reference to the 4th kind of possible implementation of second aspect, in the 5th kind of possible implementation, the number Also include threshold adjustment unit according to processing module, be used for：

After often being trained through 10 times, each data sample is tried to achieve according to formula (4) and is weighed with the error of set threshold value Value, wherein, e_iError weights are represented, P represents set threshold value；

e_i=P- α_i(1≤i≤10) (4)

Select the forgetting factor α of the error maximum weight_iAs new threshold value；

According to the forgetting factor α_i, adapt to the value of adjustment β, γ and δ.

With reference to the 4th kind of possible implementation of second aspect, in the 6th kind of possible implementation, the number According to processing unit, also particularly useful for：

As the forgetting factor α_iWhen=0, retain the forgetting factor α_iCorresponding data sample；

As the ＜ α of the forgetting factor 0_iDuring ＜ β, the forgetting factor α is deleted_iCorresponding data sample；

As the forgetting factor β≤α_iDuring ＜ δ, data sample of the selection more than γ is used as data sample next time Collection is tested；

As the forgetting factor δ ＜ α_iDuring ＜ 1, by the forgetting factor α_iCorresponding data sample is used as next time Set of data samples is tested.

Therefore, the embodiment of the present invention first builds grader and determines characteristic of division vector, further according to classification Device and characteristic of division vector are trained to the data sample for increasing newly, and then according to the correlation ginseng obtained by training It is several that data sample is deleted and retained, it is possible to achieve in incremental learning, actively delete useless data And the data for remaining with, so as to improve the adaptability and accuracy rate of cognitive computation model.

Brief description of the drawings

In order to illustrate more clearly the embodiments of the present invention, below will be to needed for embodiment or description of the prior art The accompanying drawing to be used is briefly described, it should be apparent that, drawings in the following description are only of the invention Some embodiments, for those of ordinary skill in the art, on the premise of not paying creative work, Other accompanying drawings can also be obtained according to these accompanying drawings.

Fig. 1 is a kind of schematic flow sheet of sorting technique based on incremental learning provided in an embodiment of the present invention；

Fig. 2 is the schematic flow sheet of another sorting technique based on incremental learning provided in an embodiment of the present invention；

Fig. 3 is a kind of structural representation of sorter based on incremental learning provided in an embodiment of the present invention；

Fig. 4 is a kind of structural representation of data processing module provided in an embodiment of the present invention；

Fig. 5 is the structural representation of another sorter based on incremental learning provided in an embodiment of the present invention.

Specific embodiment

Below in conjunction with the accompanying drawing in the embodiment of the present invention, the technical scheme in the embodiment of the present invention is carried out clearly Chu, it is fully described by, it is clear that described embodiment is only a part of embodiment of the invention, rather than Whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art are not making creation Property work under the premise of the every other embodiment that is obtained, belong to the scope of protection of the invention.

Sorting technique based on incremental learning provided in an embodiment of the present invention, is applied to the learning training of big data Algorithm, except can apply to batch data process field, such as bio-identification, signal identification and detection, The multiple fields such as image recognition, can also be applied to classification, recurrence, the cluster, example of magnanimity real-time stream Such as portable medical in economic data classification analysis, the Internet of things system in real-time securities trading and ecommerce The analysis of the real-time streams such as real-time diagnosis, the Real-Time Traffic Volume prediction of wisdom traffic.

Fig. 1 is a kind of schematic flow sheet of the sorting technique based on incremental learning in the embodiment of the present invention.As schemed The flow of the sorting technique based on incremental learning in shown the present embodiment can include：

S101, builds grader and determines characteristic of division vector.

Specifically, the data sample amount for processing as needed, determines the quantity of grader to be built, and then The grader of respective numbers is built, and determines the characteristic of division vector of each grader.Wherein, training step It is as follows：

1. with grader by the tagsort of all set of data samples；

2. the relevant value of each characteristic vector in each grader is calculated respectively according to previously selected kernel function；

3. calculating covariance matrix space according to the relevant value for obtaining carries out Householder conversion；

4. characteristic of division coefficient is calculated；

5. the model parameter of grader is obtained.

S102, according to the grader and characteristic of division vector, the data sample to increasing newly is trained.

Specifically, according to grader and characteristic of division vector, the step of being trained to newly-increased data sample Including：

1. by stochastic gradient descent SGD algorithms, newly-increased data sample subset B is randomly selected₁It is trained；

2. preliminary classification device Γ is passed through¹Judge the sample set B₁The correctness of classification, and according to judged result By the sample set B₁It is divided into test errors collection B_errCollection B correct with test_ok；

3. the test errors collection B is judged_errWhether it is empty set,

If so, new batch of data sample is then extracted by the SGD algorithms be trained,

If it is not, then by the set of supporting vector SV in original data sampleWith the sample set B₁Enter Row merges to obtain new setWith new grader Γ², and by the setIn remove the set Data sample outside remaining data sample collection B correct with the test_okMerge to obtain the grader Γ²Incremental data sample set B₁′；

Repeat above-mentioned 1., 2. and 3. three steps.

S103, according to the relevant parameter obtained by training, is deleted and is retained to the data sample.

Specifically, first, according to formula (1), formula (2) and formula (3), trying to achieve forgetting factor α_i, Wherein, α_iRepresent data sample i-th ratio of data sample supporting vector SV, T after T training_iRepresent Total frequency of training, r_iI-th number of times of the trained rear supporting vector SV of data sample is represented, the test is wrong Collect B by mistake_errThe r of interior each data sample_i=0, the i.e. data sample are normal data, and the test correctly collects B_ok The r of interior each data sample_i=1, the i.e. data sample are abnormal data.It is pointed out that by improved KKT (Karush Kuhn Tucker) theorem understands that formula (1) and formula (2) are to meet optimal super flat The condition in face.

Then, according to based on the forgetting factor α_iPrediction incremental learning mechanism, the data sample is entered Row is deleted and retained.During implementing, first set tri- threshold values of β, γ and δ, such as β=0.3, γ=0.4, δ=0.7, then compare forgetting factor α_iWith the magnitude relationship of β, γ and δ, and then according to result of the comparison, by pre- If rule is deleted and retained to data sample.Wherein, preset rules are as follows：

As the forgetting factor α_iWhen=0, retain the forgetting factor α_iCorresponding data sample, reason is, Newly-increased data sample is error sample, and these samples do not influence original grader, but may be new dividing Class, so being retained as a subset of the test sample of training next time；

As the ＜ α of the forgetting factor 0_iDuring ＜ β, the forgetting factor α is deleted_iCorresponding data sample, reason is, It is very low by the ratio of data sample SV after repeatedly training, it is not a kind of new classification, institute by the data sample To be deleted, the training speed of intrusion detection is improve while the storage for reducing initial data；

As the forgetting factor β≤α_iDuring ＜ δ, data sample of the selection more than γ is used as data sample next time Collection is tested, the convergence rate that SV can be accelerated to search for；

As the forgetting factor δ ＜ α_iDuring ＜ 1, by the forgetting factor α_iCorresponding data sample is used as next time Set of data samples is tested.

It is further alternative, after often being trained through 10 times, each data sample and institute are tried to achieve according to formula (4) If the error weights of threshold value, the forgetting factor α of Select Error maximum weight_iAs new threshold value, and according to something lost Forget factor-alpha_i, adapt to the value of adjustment β, γ and δ.Wherein, e_iError weights are represented, P represents set threshold value

e_i=P- α_i(1≤i≤10) (4)

Therefore, the embodiment of the present invention first builds grader and determines characteristic of division vector, further according to classification Device and characteristic of division vector are trained to the data sample for increasing newly, and then according to the correlation ginseng obtained by training It is several that data sample is deleted and retained, it is possible to achieve in incremental learning, actively delete useless data And the data for remaining with, so as to improve the adaptability and accuracy rate of cognitive computation model.

Fig. 2 is a kind of schematic flow sheet of the sorting technique based on incremental learning, the party in the embodiment of the present invention Method is applied to the intrusion detection scene of the network information, and the network flow data that can be directed to magnanimity carries out invasion inspection Survey.The flow of the sorting technique based on incremental learning in the present embodiment can include as shown in the figure：

S201, builds Attack Classification device and determines characteristic of division vector.

Specifically, the network flow data sample size for processing as needed, determines Attack Classification device to be built Quantity, and then build the Attack Classification device of respective numbers, and determine the characteristic of division of each Attack Classification device Vector.Wherein, training step is as follows：

1. Attack Classification device is used by the tagsort of all-network data on flows sample set；

2. the relevant of each characteristic vector in each Attack Classification device is calculated according to previously selected kernel function respectively Value；

3. calculating covariance matrix space according to the relevant value for obtaining carries out Householder conversion；

4. characteristic of division coefficient is calculated；

5. the model parameter of Attack Classification device is obtained.

S202, sets tri- threshold values of β, γ and δ.

Such as β=0.3, γ=0.4, δ=0.7.

S203, according to the Attack Classification device and characteristic of division vector, to the network flow data for increasing newly Sample is trained, while asking for forgetting factor.

Specifically, according to Attack Classification device and characteristic of division vector, the network flow data sample to increasing newly enters The step of row training, includes：

1. by stochastic gradient descent SGD algorithms, newly-increased network flow data sample set B is randomly selected₁ It is trained；

2. initial Attack Classification device Γ is passed through¹Judge the sample set B₁The correctness of classification, and according to judgement Result is by the sample set B₁It is divided into test errors collection B_errCollection B correct with test_ok；

3. the test errors collection B is judged_errWhether it is empty set,

If so, new a collection of network flow data sample is then extracted by the SGD algorithms be trained,

If it is not, then by the set of supporting vector SV in original network flow data sampleWith the sample Subset B₁Merge to obtain new setWith new attack grader Γ², and by the setIn Except the setNetwork flow data sample outside rest network data on flows sample it is correct with the test Collection B_okMerge to obtain the Attack Classification device Γ²Incremental data sample set B₁′；

Repeat above-mentioned 1., 2. and 3. three steps.

Further, according to formula (1), formula (2) and formula (3), forgetting factor α is tried to achieve_i, wherein, α_iRepresent network flow data sample i-th network flow data sample supporting vector SV after T training Ratio, T_iRepresent total frequency of training, r_iRepresent i-th trained rear supporting vector SV of network flow data sample Number of times, the test errors collection B_errThe r of interior each network flow data sample_i=0, i.e. the network traffics number It is proper network data on flows according to sample, the test correctly collects B_okInterior each network flow data sample r_i=1, i.e. the network flow data sample are attack network flow data.It is pointed out that by improved KKT (Karush Kuhn Tucker) theorem understands that formula (1) and formula (2) are to meet optimal super flat The condition in face.

S204, asks for the error weights of each network flow data sample and set threshold value.

Specifically, the error weights of each network flow data sample and set threshold value are tried to achieve according to formula (4), Wherein, e_iError weights are represented, P represents set threshold value.

e_i=P- α_i(1≤i≤10) (4)

S205, selects the forgetting factor of the error maximum weight as new threshold value.

S206, according to the forgetting factor, adapts to the value of adjustment β, γ and δ.

The magnitude relationship of S207, relatively more described forgetting factor and β, γ and δ.

S208, according to result of the comparison, is deleted and is retained to the network flow data sample.

Specifically, according to result of the comparison, network flow data sample is deleted and protected by preset rules Stay.Wherein, preset rules are as follows：

As the forgetting factor α_iWhen=0, retain the forgetting factor α_iMap network data on flows sample, it is former Because being that newly-increased network flow data sample is error sample, and these samples do not influence original Attack Classification Device, but may be new classification, it is possible to it is a kind of new attack classification, so being retained as next instruction The a subset of experienced test sample；

As the ＜ α of the forgetting factor 0_iDuring ＜ β, the forgetting factor α is deleted_iMap network data on flows sample, Reason is, very low by the ratio of network flow data sample SV after repeatedly training, by the network traffics number It is not a kind of new attack classification according to sample, so being deleted, reduces the storage of original network traffic data While improve the training speed of intrusion detection；

As the forgetting factor β≤α_iDuring ＜ δ, network flow data sample of the selection more than γ is used as next time Network flow data sample set is tested, the convergence rate that SV can be accelerated to search for；

As the forgetting factor δ ＜ α_iDuring ＜ 1, by the forgetting factor α_iMap network data on flows sample conduct Network flow data sample set next time is tested, and reason is that the network flow data sample is to attack Hit data sample.

Therefore, the embodiment of the present invention first builds Attack Classification device and determines characteristic of division vector, further according to Attack Classification device and characteristic of division vector are trained to the network flow data sample for increasing newly, and then according to instruction Relevant parameter obtained by practicing is deleted and retained to network flow data sample, it is possible to achieve in increment In habit, useless network flow data and the network flow data for remaining with actively are deleted, so as to improve The adaptability and accuracy rate of cognitive computation model.

Fig. 3 is a kind of structural representation of the sorter based on incremental learning in the embodiment of the present invention.As schemed The sorter based on incremental learning in the shown embodiment of the present invention can at least include initialization module 310, Data training module 320 and data processing module 330, wherein：

Initialization module 310, for building grader and determining characteristic of division vector.

Specifically, the data sample amount for processing as needed, determines the quantity of grader to be built, and then The grader of respective numbers is built, and determines the characteristic of division vector of each grader.Wherein, training step It is as follows：

1. with grader by the tagsort of all set of data samples；

2. the relevant value of each characteristic vector in each grader is calculated respectively according to previously selected kernel function；

3. calculating covariance matrix space according to the relevant value for obtaining carries out Householder conversion；

4. characteristic of division coefficient is calculated；

5. the model parameter of grader is obtained.

Data training module 320, for according to the grader and characteristic of division vector, to the number for increasing newly It is trained according to sample.

Specifically, according to grader and characteristic of division vector, the step of being trained to newly-increased data sample Including：

1. by stochastic gradient descent SGD algorithms, newly-increased data sample subset B is randomly selected₁It is trained；

2. preliminary classification device Γ is passed through¹Judge the sample set B₁The correctness of classification, and according to judged result By the sample set B₁It is divided into test errors collection B_errCollection B correct with test_ok；

3. the test errors collection B is judged_errWhether it is empty set,

If so, new batch of data sample is then extracted by the SGD algorithms be trained,

If it is not, then by the set of supporting vector SV in original data sampleWith the sample set B₁Enter Row merges to obtain new setWith new grader Γ², and by the setIn remove the set Data sample outside remaining data sample collection B correct with the test_okMerge to obtain the grader Γ²Incremental data sample set B₁′；

Repeat above-mentioned 1., 2. and 3. three steps.

Data processing module 330, for according to the relevant parameter obtained by training, being carried out to the data sample Delete and retain.In implementing, data processing module 330 can as shown in Figure 4 further include parameter Computing unit 331 and data processing unit 332, wherein：

Parameter calculation unit 331, for according to formula (1), formula (2) and formula (3), trying to achieve forgetting Factor-alpha_i, wherein, α_iRepresent data sample i-th ratio of data sample supporting vector SV after T training Rate, T_iRepresent total frequency of training, r_iI-th number of times of the trained rear supporting vector SV of data sample is represented, The test errors collection B_errThe r of interior each data sample_i=0, the correct collection B of test_okInterior each data sample R_i=1.

Data processing unit 332, for according to based on the forgetting factor α_iPrediction incremental learning mechanism, it is right The data sample is deleted and retained.During implementing, tri- threshold values of β, γ and δ are first set, Such as β=0.3, γ=0.4, δ=0.7, then compare forgetting factor α_iWith the magnitude relationship of β, γ and δ, Jin Ergen According to result of the comparison, data sample is deleted and retained by preset rules.Wherein, preset rules are as follows：

As the forgetting factor α_iWhen=0, retain the forgetting factor α_iCorresponding data sample, reason is, Newly-increased data sample is error sample, and these samples do not influence original grader, but may be new dividing Class, so being retained as a subset of the test sample of training next time；

As the ＜ α of the forgetting factor 0_iDuring ＜ β, the forgetting factor α is deleted_iCorresponding data sample, reason is, It is very low by the ratio of data sample SV after repeatedly training, it is not a kind of new classification, institute by the data sample To be deleted, the training speed of intrusion detection is improve while the storage for reducing initial data；

As the forgetting factor β≤α_iDuring ＜ δ, data sample of the selection more than γ is used as data sample next time Collection is tested, the convergence rate that SV can be accelerated to search for；

As the forgetting factor δ ＜ α_iDuring ＜ 1, by the forgetting factor α_iCorresponding data sample is used as next time Set of data samples is tested.

Fig. 4 is referred to, data processing module 330 can also include threshold adjustment unit 333 as shown in the figure, use In：

After often being trained through 10 times, each data sample is tried to achieve according to formula (4) and is weighed with the error of set threshold value Value；

e_i=P- α_i(1≤i≤10) (4)

Select the forgetting factor α of the error maximum weight_iAs new threshold value；

According to the forgetting factor α_i, adapt to the value of adjustment β, γ and δ.

Fig. 5 is the structural representation of sorter of the another kind based on incremental learning in the embodiment of the present invention, As shown in figure 5, the sorter that should be based on incremental learning can include：At least one processor 501, for example CPU, at least one communication bus 502, at least one network interface 503, memory 504.Wherein, lead to Letter bus 502 is used to realize the connection communication between these components；Memory 504 can be that high-speed RAM is deposited Reservoir, or non-volatile memory (non-volatile memory), for example, at least one disk storage Device.Optionally, memory 504 can also be at least one storage dress for being located remotely from aforementioned processor 501 Put.Batch processing code is stored in memory 504, processor 501 is used to call storage in memory x04 Program code, perform following operation：

Build grader and determine characteristic of division vector；

According to the grader and characteristic of division vector, the data sample to increasing newly is trained；

According to the relevant parameter obtained by training, the data sample is deleted and retained.

Optionally, processor 501 builds grader and determines that the vectorial concrete operations of characteristic of division are：

Determine the quantity of grader to be built；

Build each grader；

Determine the characteristic of division vector of each grader.

Again optional, processor 501 is vectorial according to the grader and the characteristic of division, to the number for increasing newly The concrete operations being trained according to sample are：

1. by stochastic gradient descent SGD algorithms, newly-increased data sample subset B is randomly selected₁It is trained；

2. preliminary classification device Γ is passed through¹Judge the sample set B₁The correctness of classification, and according to judged result By the sample set B₁It is divided into test errors collection B_errCollection B correct with test_ok；

3. the test errors collection B is judged_errWhether it is empty set,

If so, new batch of data sample is then extracted by the SGD algorithms be trained,

If it is not, then by the set of supporting vector SV in original data sampleWith the sample set B₁Enter Row merges to obtain new setWith new grader Γ², and by the setIn remove the set Data sample outside remaining data sample collection B correct with the test_okMerge to obtain the grader Γ²Incremental data sample set B₁′；

Repeat above-mentioned 1., 2. and 3. three steps.

Further, processor 501 is carried out according to the relevant parameter obtained by training to the data sample Delete and the concrete operations of reservation are：

According to formula (1), formula (2) and formula (3), forgetting factor α is tried to achieve_i, wherein, α_iRepresent number I-th ratio of data sample supporting vector SV, T after being trained through T times according to sample_iRepresent total frequency of training, r_i Represent i-th number of times of the trained rear supporting vector SV of data sample, the test errors collection B_errInterior every number According to the r of sample_i=0, the correct collection B of test_okThe r of interior each data sample_i=1；

According to based on the forgetting factor α_iPrediction incremental learning mechanism, the data sample is deleted And reservation.

Further, processor 501 is according to based on the forgetting factor α_iPrediction incremental learning mechanism, The concrete operations that the data sample is deleted and retained are：

Tri- threshold values of β, γ and δ are set；

Compare the forgetting factor α_iWith the magnitude relationship of β, γ and δ；

According to result of the comparison, the data sample is deleted and retained.

Optionally, after processor 501 sets tri- threshold values of β, γ and δ, also perform：

After often being trained through 10 times, each data sample is tried to achieve according to formula (4) and is weighed with the error of set threshold value Value；

e_i=P- α_i(1≤i≤10) (4)

Select the forgetting factor α of the error maximum weight_iAs new threshold value；

According to the forgetting factor α_i, adapt to the value of adjustment β, γ and δ.

Again optional, processor 501 is deleted and retained according to result of the comparison to the data sample Concrete operations be：

As the forgetting factor α_iWhen=0, retain the forgetting factor α_iCorresponding data sample；

As the ＜ α of the forgetting factor 0_iDuring ＜ β, the forgetting factor α is deleted_iCorresponding data sample；

As the forgetting factor β≤α_iDuring ＜ δ, data sample of the selection more than γ is used as data sample next time Collection is tested；

As the forgetting factor δ ＜ α_iDuring ＜ 1, by the forgetting factor α_iCorresponding data sample is used as next time Set of data samples is tested.

Therefore, the embodiment of the present invention first builds grader and determines characteristic of division vector, further according to classification Device and characteristic of division vector are trained to the data sample for increasing newly, and then according to the correlation ginseng obtained by training It is several that data sample is deleted and retained, it is possible to achieve in incremental learning, actively delete useless data And the data for remaining with, so as to improve the adaptability and accuracy rate of cognitive computation model.

One of ordinary skill in the art will appreciate that all or part of flow in realizing above-described embodiment method, Computer program be can be by instruct the hardware of correlation to complete, described program can be stored in a calculating In machine read/write memory medium, the program is upon execution, it may include such as the flow of the embodiment of above-mentioned each method. Wherein, described storage medium can for magnetic disc, CD, read-only memory (Read-Only Memory, ) or random access memory (Random Access Memory, RAM) etc. ROM.

Above disclosed is only present pre-ferred embodiments, can not limit the present invention's with this certainly Interest field, therefore the equivalent variations made according to the claims in the present invention, still belong to the scope that the present invention is covered.

Claims (14)

1. a kind of sorting technique based on incremental learning, it is characterised in that methods described includes：

Build grader and determine characteristic of division vector；

According to the grader and characteristic of division vector, the data sample to increasing newly is trained；

According to the relevant parameter obtained by training, the data sample is deleted and retained.

2. the method for claim 1, it is characterised in that the structure grader simultaneously determines that classification is special Vector is levied, including：

Determine the quantity of grader to be built；

Build each grader；

Determine the characteristic of division vector of each grader.

3. the method for claim 1, it is characterised in that described according to the grader and described point Category feature vector, the data sample to increasing newly is trained, including：

1. by stochastic gradient descent SGD algorithms, newly-increased data sample subset B is randomly selected₁It is trained；

2. preliminary classification device Γ is passed through¹Judge the sample set B₁The correctness of classification, and according to judged result By the sample set B₁It is divided into test errors collection B_errCollection B correct with test_ok；

3. the test errors collection B is judged_errWhether it is empty set,

If so, new batch of data sample is then extracted by the SGD algorithms be trained,

If it is not, then by the set of supporting vector SV in original data sampleWith the sample set B₁Enter Row merges to obtain new setWith new grader Γ², and by the setIn remove the set Data sample outside remaining data sample collection B correct with the test_okMerge to obtain the grader Γ²Incremental data sample set B '₁；

Repeat above-mentioned 1., 2. and 3. three steps.

4. method as claimed in claim 3, it is characterised in that the correlation ginseng according to obtained by training Number, is deleted and is retained to the data sample, including：

According to formula (1), formula (2) and formula (3), forgetting factor α is tried to achieve_i, wherein, α_iRepresent number I-th ratio of data sample supporting vector SV, T after being trained through T times according to sample_iRepresent total frequency of training, r_i Represent i-th number of times of the trained rear supporting vector SV of data sample, the test errors collection B_errInterior every number According to the r of sample_i=0, the correct collection B of test_okThe r of interior each data sample_i=1；

$\mathrm{\ω}=\underset{i}{\Σ}{\mathrm{\α}}_i{y}_{i}H\left(x_i\right)---\left(1\right)$

$0\≤{\mathrm{\α}}_i\≤1,\underset{i}{\Σ}{\mathrm{\α}}_i{y}_i=0---\left(2\right)$

${\mathrm{\α}}_i=\frac{r_i}{T_i}---\left(3\right)$

According to based on the forgetting factor α_iPrediction incremental learning mechanism, the data sample is deleted And reservation.

5. method as claimed in claim 4, it is characterised in that the basis is based on the forgetting factor α_i Prediction incremental learning mechanism, the data sample is deleted and retained, including：

Tri- threshold values of β, γ and δ are set；

Compare the forgetting factor α_iWith the magnitude relationship of β, γ and δ；

According to result of the comparison, the data sample is deleted and retained.

6. method as claimed in claim 5, it is characterised in that tri- threshold values of described setting β, γ and δ Afterwards, also include：

After often being trained through 10 times, each data sample is tried to achieve according to formula (4) and is weighed with the error of set threshold value Value, wherein, e_iError weights are represented, P represents set threshold value；

e_i=P- α_i(1≤i≤10) (4)

Select the forgetting factor α of the error maximum weight_iAs new threshold value；

According to the forgetting factor α_i, adapt to the value of adjustment β, γ and δ.

7. method as claimed in claim 5, it is characterised in that described according to result of the comparison, to described Data sample is deleted and retained, including：

As the forgetting factor α_iWhen=0, retain the forgetting factor α_iCorresponding data sample；

As the ＜ α of the forgetting factor 0_iDuring ＜ β, the forgetting factor α is deleted_iCorresponding data sample；

As the forgetting factor β≤α_iDuring ＜ δ, data sample of the selection more than γ is used as data sample next time Collection is tested；

As the forgetting factor δ ＜ α_iDuring ＜ 1, by the forgetting factor α_iCorresponding data sample is used as next time Set of data samples is tested.

8. a kind of sorter based on incremental learning, it is characterised in that described device includes：

Initialization module, for building grader and determining characteristic of division vector；

Data training module, for according to the grader and characteristic of division vector, to the data for increasing newly Sample is trained；

Data processing module, for according to the relevant parameter obtained by training, being deleted to the data sample Except and retain.

9. device as claimed in claim 8, it is characterised in that the initialization module, specifically for：

Determine the quantity of grader to be built；

Build each grader；

Determine the characteristic of division vector of each grader.

10. device as claimed in claim 8, it is characterised in that the data training module, it is specific to use In：

1. by stochastic gradient descent SGD algorithms, newly-increased data sample subset B is randomly selected₁It is trained；

2. preliminary classification device Γ is passed through¹Judge the sample set B₁The correctness of classification, and according to judged result By the sample set B₁It is divided into test errors collection B_errCollection B correct with test_ok；

3. the test errors collection B is judged_errWhether it is empty set,

If so, new batch of data sample is then extracted by the SGD algorithms be trained,

If it is not, then by the set of supporting vector SV in original data sampleWith the sample set B₁Enter Row merges to obtain new setWith new grader Γ², and by the setIn remove the set Data sample outside remaining data sample collection B correct with the test_okMerge to obtain the grader Γ²Incremental data sample set B '₁；

Repeat above-mentioned 1., 2. and 3. three steps.

11. devices as claimed in claim 10, it is characterised in that the data processing module includes：

Parameter calculation unit, for according to formula (1), formula (2) and formula (3), trying to achieve forgetting factor α_i, wherein, α_iRepresent data sample i-th ratio of data sample supporting vector SV, T after T training_i Represent total frequency of training, r_iRepresent i-th number of times of the trained rear supporting vector SV of data sample, the survey Examination Error Set B_errThe r of interior each data sample_i=0, the correct collection B of test_okThe r of interior each data sample_i=1；

$\mathrm{\ω}=\underset{i}{\Σ}{\mathrm{\α}}_i{y}_{i}H\left(x_i\right)---\left(1\right)$

$0\≤{\mathrm{\α}}_i\≤1,\underset{i}{\Σ}{\mathrm{\α}}_i{y}_i=0---\left(2\right)$

${\mathrm{\α}}_i=\frac{r_i}{T_i}---\left(3\right)$

Data processing unit, for according to based on the forgetting factor α_iPrediction incremental learning mechanism, to institute Data sample is stated to be deleted and retained.

12. devices as claimed in claim 11, it is characterised in that the data processing unit, it is specific to use In：

Tri- threshold values of β, γ and δ are set；

Compare the forgetting factor α_iWith the magnitude relationship of β, γ and δ；

According to result of the comparison, the data sample is deleted and retained.

13. devices as claimed in claim 12, it is characterised in that the data processing module also includes threshold Value adjustment unit, is used for：

After often being trained through 10 times, each data sample is tried to achieve according to formula (4) and is weighed with the error of set threshold value Value, wherein, e_iError weights are represented, P represents set threshold value；

e_i=P- α_i(1≤i≤10) (4)

Select the forgetting factor α of the error maximum weight_iAs new threshold value；

According to the forgetting factor α_i, adapt to the value of adjustment β, γ and δ.

14. devices as claimed in claim 12, it is characterised in that the data processing unit, it is also specific For：

As the forgetting factor α_iWhen=0, retain the forgetting factor α_iCorresponding data sample；

As the ＜ α of the forgetting factor 0_iDuring ＜ β, the forgetting factor α is deleted_iCorresponding data sample；

As the forgetting factor β≤α_iDuring ＜ δ, data sample of the selection more than γ is used as data sample next time Collection is tested；

As the forgetting factor δ ＜ α_iDuring ＜ 1, by the forgetting factor α_iCorresponding data sample is used as next time Set of data samples is tested.