CN109272033A - A kind of online soft margin kernel-based learning method based on step size controlling - Google Patents
A kind of online soft margin kernel-based learning method based on step size controlling Download PDFInfo
- Publication number
- CN109272033A CN109272033A CN201811037902.7A CN201811037902A CN109272033A CN 109272033 A CN109272033 A CN 109272033A CN 201811037902 A CN201811037902 A CN 201811037902A CN 109272033 A CN109272033 A CN 109272033A
- Authority
- CN
- China
- Prior art keywords
- kernel
- sample
- function
- online
- soft margin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Abstract
The present invention relates to a kind of online soft margin kernel-based learning method (OSKL) based on step size controlling.By introducing Kernel Nonlinear Classifier, the influence of soft margin state modulator noise data is introduced, and there is the online kernel-based learning method of robustness based on the design of the basic framework of online gradient descent algorithm.The algorithm can reduce model memory space, effectively control influence of noise, the computation complexity of model modification is only (1) O, has strong real-time, the advantages such as is easily achieved, be the natural tool of processing and analysis data-flow problem.On-line learning algorithm of the present invention, overcome the conventional sorting methods based on batch system can not efficient process data flow the problem of, also overcome the problem of existing on-line learning algorithm such as Kernel Perceptron and Pegasos scheduling algorithm can not effectively inhibit influence of noise.
Description
Technical field
The invention belongs to data minings and machine learning field, are related to the method for data mining and data processing, specifically
It says, is related to a kind of online soft margin kernel-based learning method (OSKL) based on step size controlling.
Background technique
Classification problem is the classical problem of data mining Yu machine learning area research.It is traditional based on batch system
Classification method first collects data, constructs learning model based on data are collected, and select optimization algorithm solving model, is classified
Device.With the fast development of the technologies such as e-commerce, social media, mobile Internet, Internet of Things, more and more application scenarios
It needs to handle large-scale data stream in real time.Tradition is based on the classification method of batch system when handling large-scale data flow problem
There are many deficiencies such as computation complexity height, model modification low efficiency.On-line study is passed through based on the basic framework learnt point by point
The dynamic point-by-point learning data information of more new model, the primary computation complexity of model modification is only (1) O, has computation complexity
The advantages such as low, model modification is high-efficient, strong real-time are the natural tools of processing and analysis data-flow problem.In addition, extensive
Inevitably there is partial error label in label data, and these error labels can seriously affect the construction and effect of classifier.Therefore,
It needs to design a kind of data stream mining with error resilience performance.
Summary of the invention
It is an object of the invention to can not efficient process data flow point for the existing classification method based on batch system
Class problem, and on-line learning algorithm can not inhibit noise effect, propose a kind of online soft margin core based on step size controlling
Algorithm is practised, this method can reduce model memory space, and effectively control influence of noise significantly improves model modification efficiency, meets
The real-time demand of actual application problem.
An embodiment according to the present invention provides a kind of online soft margin kernel-based learning method based on step size controlling, contains
Following steps:
(1) initialization model parameter, decision function and model kernel function.
(2) data flow is acquired, the class label of categorised decision function prediction data flow sample is utilized.
(3) sample true tag is obtained, loss function is specified, calculates sample losses value.
(4) the update step-length of classifier decision function is calculated.
(5) it is based on online gradient descent algorithm basic framework, updates classifier decision function.
In learning algorithm according to an embodiment of the present invention, in step (1), the specific steps of model initialization are as follows:
Determine that training sample set and test sample set, initialization model threshold parameter C, two classification problems of initialization are determined
Plan function f0=0, specifying gaussian kernel function is model kernel function k ().
In learning algorithm according to an embodiment of the present invention, in step (2), categorised decision function prediction data flow is utilized
The specific steps of sample class label are as follows:
Data flow { (x is acquired in the form of one-by-onet,yt)}T=1,2 ..., xtIndicate t-th of sample input, ytIt indicates
T-th of sample exports (class label).Utilize decision function ft-1T-th of sample in prediction data streamLabel:
In learning algorithm according to an embodiment of the present invention, in step (3), the concrete operations stream of sample losses is calculated
Journey are as follows:
The most commonly used hinge function calculates the sample point (x as loss function in specified two classification problemst,yt)
Hinge loss:
In learning algorithm according to an embodiment of the present invention, in step (4), calculates and update step-length τtConcrete operations
Process are as follows: consider to determine update step-length τ based on following two o'clockt: 1. realized with confidence level as high as possible to current sample xtPoint
Correct classification, that is, reach zero loss (lt=0)；2. guaranteeing the stability of algorithm as far as possible, i.e. reduction decision function is updated
Fluctuation in journey.Optimal step size τtFor the solution of following optimization problems:
On the other hand, inevitably there are a large amount of error label data in extensive sampled data, these error labels can serious shadow
Ring the construction of decision function and the effect of corresponding classifier.For this purpose, we, which introduce the control of soft margin threshold parameter, updates step-length τt
≤ C, and then influence of the error label data to model is limited, guarantee the stability of classifier.Based on calculating institute in step (3)
Obtain sample point (xt,yt) hinge lose ltAnd step size controlling parameter C, it determines and updates step-length τtAre as follows:
In learning algorithm according to an embodiment of the present invention, in step (5), the specific behaviour of classifier decision function is updated
Make process are as follows:
Based on the update step-length τ calculated in step (4)t, under the basic framework of online gradient descent algorithm, to decision
Function ftIt is updated
Obtain new decision function ft。
The present invention relates to a kind of online soft margin kernel-based learning method based on step size controlling.Letter is lost by introducing hinge
Number, gaussian kernel function, soft margin threshold parameter C establish in line core Study strategies and methods, realize the on-line prediction to data flow.
This method makes the update of classifier decision function more smooth using soft margin threshold parameter, has robustness.Exist with classics
Line learning algorithm Kernel Perceptron is compared with Pegesos, and the algorithm OSKL proposed has been obviously improved nicety of grading.
Online classification algorithm OSKL of the present invention can flexibly handle the classification problem under data stream scenario, be based on batch processing skill with tradition
The static classification mode of art is compared, and computation complexity is considerably reduced, and reduces the model running time.
Detailed description of the invention
A kind of online soft margin kernel-based learning method schematic diagram based on step size controlling of attached drawing 1
Nicety of grading comparison schematic diagram of the 2 three kinds of algorithms of attached drawing in benchmark dataset
Average test nicety of grading comparison schematic diagram of the 3 three kinds of algorithms of attached drawing on noisy phonetic symbol label data set ijcnn
Average test nicety of grading comparison schematic diagram of the 4 three kinds of algorithms of attached drawing on noisy phonetic symbol label data set codrna
Average test nicety of grading comparison schematic diagram of the 5 three kinds of algorithms of attached drawing on noisy phonetic symbol label data set eegeye
Specific embodiment
Specific steps of the present invention are explained below in conjunction with attached drawing.
Embodiment one: it is carried out by taking the online classification experiment on master reference data set ijcnn, codrna, eegeye as an example
Explanation.As shown in Figure 1 for according to a kind of online soft margin kernel-based learning method based on step size controlling provided in an embodiment of the present invention
Schematic diagram, which contains following steps:
Step 1: initialization model parameter, decision function and model kernel function.The specific steps are that:
Initialization model threshold parameter C=0.05 initializes two classification problem decision function f0=0, specify Gaussian kernel letter
Number is model kernel function, that is, k (xi,xj)=exp (- ‖ xi-xj‖2/ d), wherein d is taken as the dimension of sample input x.
Step 2: acquisition data flow utilizes the class label of categorised decision function prediction data flow sample.Its specific steps
Are as follows: the form of one- by-one acquires data flow { (xt,yt)}T=1,2 ..., xtIndicate t-th of sample input, ytIndicate t-th of sample
This output (class label).Utilize decision function ft-1T-th of sample in prediction data streamLabel:
Step 3: obtaining sample true tag, specifies loss function, calculates sample losses value.The specific steps are that: it is specified
The most commonly used hinge function calculates the sample point (x as loss function in two classification problemst, yt) hinge loss:
Step 4: the update step-length of classifier decision function is calculated.The specific steps are that: introduce the control of soft margin threshold parameter
System updates step-length τt≤ C, and then influence of the error label data to model is limited, guarantee the stability of classifier.Based on step
(3) gained sample point (x is calculated int,yt) hinge lose ltAnd step size controlling parameter C, determine the update step of t step
Long τtAre as follows:
Step 5: being based on online gradient descent algorithm basic framework, updates classifier decision function.The specific steps are that:
Based on the update step-length τ calculated in step (4)t, under the basic framework of online gradient descent algorithm, to decision function ftIt carries out
It updates
Obtain new decision function ft。
Fig. 2 be using on-line learning algorithm of the present invention and existing on-line learning algorithm Kernel Perceptron and
Pegasos is predicted average online in benchmark dataset ijcnn, benchmark dataset codrna and benchmark dataset eegeye
Measuring accuracy comparison schematic diagram.As seen from Figure 2, on-line learning algorithm of the present invention is closed in above-mentioned 3 benchmark datasets
Average test precision is unanimously better than other two methods.
Embodiment two: on the basis of master reference data set ijcnn, codrna, eegeye, being added noise label,
Training online classification device on the data set of noisy phonetic symbol label.What is different from the first embodiment is that in the present embodiment, in step 1,
It randomly selects the 30% of data set and is used as test set, construct training set after remainder data addition noise label.Specifically, we will
Sample index distinguishes mould 20, mould 10, mould 5, and the sample point label that remainder is 0 is obtained noise label data multiplied by -1.
Fig. 3-5 is the training online classification device Kernel on data set ijcnn, codrna, eegeye of noisy phonetic symbol label
Perceptron, Pegasos and OSKL, and the average classification performance in the noiseless test data set of former 30% data set is (flat
Equal measuring accuracy, ACA).The experimental results showed that with the noise increasing with the training sample of mod20, mod10 and mod5 index
Greatly, three kinds of algorithm classification precision are declined, but the OSKL algorithm of my proposition can be controlled effectively in the case where containing noise and be made an uproar
Sound shadow is rung, and classifying quality is apparently higher than online classification device Kernel Perceptron and Pegasos.
Above-described embodiment is used to explain the present invention, rather than limits the invention, in spirit and right of the invention
It is required that protection scope in, to any modifications and changes for making of the present invention, both fall within protection scope of the present invention.
Claims (6)
1. a kind of online soft margin kernel-based learning method (OSKL) based on step size controlling, it is characterised in that contain following steps:
(1) initialization model parameter, decision function and model kernel function.
(2) data flow is acquired in the form of one-by-one, utilizes decision function ft-1To data flow sample xtLabel carry out it is pre-
It surveys.
(3) true tag y is obtainedtAfterwards, the hinge loss of the sample point is calculated
(4) it calculates and updates step-length τtIf: lt=0, then τt=0；If lt> 0, then
(5) classifier decision function f is updatedt=ft-1+τtytk(xt,·)。
2. a kind of online soft margin kernel-based learning method based on step size controlling according to claim 1, which is characterized in that
In step (1), concrete operations process are as follows: determine training sample set and test sample set, initialization model threshold parameter C,
Initialize two classification problem decision function f0=0, select gaussian kernel function as model kernel function k (), i.e. k (xi,xj)
=exp (- ‖ xi-xj‖2/ d), wherein d is taken as the dimension of sample input x.
3. a kind of online soft margin kernel-based learning method based on step size controlling according to claim 1, it is characterised in that: step
Suddenly in (two), the specific steps of categorised decision function prediction data flow sample class label are utilized are as follows: with the shape of one-by-one
Formula acquires data flow { (xt,yt)}T=1,2 ..., xtIndicate t-th of sample input, ytIndicate t-th of sample output (class label).
Utilize decision function ft-1T-th of sample in prediction data streamLabel:
4. a kind of online soft margin kernel-based learning method based on step size controlling according to claim 1, which is characterized in that
In step (3), the concrete operations process of sample losses is calculated are as follows: the most commonly used hinge function is made in specified two classification problems
For loss function, the sample point (x is calculatedt,yt) hinge loss:
5. a kind of online soft margin kernel-based learning method based on step size controlling according to claim 1, which is characterized in that
In step (4), calculates and update step-length τtConcrete operations process are as follows: based on following two o'clock consider determine update step-length τt.1. with
Confidence level as high as possible is realized to current sample xtThe correct classification of point, that is, reach zero loss (lt=0)；2. guaranteeing as far as possible
The stability of algorithm, the i.e. fluctuation of reduction decision function at no point in the update process.Optimal step size τ can be obtainedtAre as follows:
On the other hand, inevitably there are a large amount of error label data in extensive sampled data, these error labels can seriously affect certainly
The construction of plan function and the effect of corresponding classifier.For this purpose, we, which introduce the control of soft margin threshold parameter, updates step-length τt≤ C,
And then influence of the error label data to model is limited, guarantee the stability of classifier.Based on calculating gained sample in step (3)
This point (xt,yt) hinge lose ltAnd step size controlling parameter C, it determines and updates step-length τtAre as follows:
6. a kind of online soft margin kernel-based learning method based on step size controlling according to claim 1, which is characterized in that
In step (5), the concrete operations process of classifier decision function is updated are as follows: based on the update step-length τ calculated in step (4)t,
Under the basic framework of online gradient descent algorithm, to decision function ftIt is updated
Obtain new decision function ft。
The present invention relates to a kind of online soft margin kernel-based learning method based on step size controlling.By introducing hinge loss function, height
This kernel function, soft margin threshold parameter C are established in line core Study strategies and methods, realize the on-line prediction to data flow.This method
Make the update of classifier decision function more smooth using soft margin threshold parameter, there is robustness.With classical on-line study
Algorithm Kernel Perceptron is compared with Pegesos, and the algorithm OSKL proposed has been obviously improved nicety of grading.The present invention
Online classification algorithm OSKL can flexibly handle the classification problem under data stream scenario, with tradition based on the quiet of batch system
State mode classification is compared, and computation complexity is considerably reduced, and reduces the model running time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811037902.7A CN109272033B (en) | 2018-09-06 | 2018-09-06 | Online soft interval kernel learning algorithm based on step length control |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811037902.7A CN109272033B (en) | 2018-09-06 | 2018-09-06 | Online soft interval kernel learning algorithm based on step length control |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109272033A true CN109272033A (en) | 2019-01-25 |
| CN109272033B CN109272033B (en) | 2022-03-08 |
Family
ID=65187352
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811037902.7A Active CN109272033B (en) | 2018-09-06 | 2018-09-06 | Online soft interval kernel learning algorithm based on step length control |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109272033B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100124377A1 (en) * | 2008-11-19 | 2010-05-20 | Nec Laboratories America, Inc. | Linear spatial pyramid matching using sparse coding |
| US20110161944A1 (en) * | 2009-12-30 | 2011-06-30 | Samsung Electronics Co., Ltd. | Method and apparatus for transforming program code |
| CN105975907A (en) * | 2016-04-27 | 2016-09-28 | 江苏华通晟云科技有限公司 | SVM model pedestrian detection method based on distributed platform |
| CN106911580A (en) * | 2017-01-24 | 2017-06-30 | 中国科学院声学研究所 | A kind of TCP jamming control methods and system |
| CN107247968A (en) * | 2017-07-24 | 2017-10-13 | 东北林业大学 | Based on logistics equipment method for detecting abnormality under nuclear entropy constituent analysis imbalance data |
| CN107991877A (en) * | 2017-12-20 | 2018-05-04 | 东南大学 | A kind of Dynamic Model Identification method and system based on Recognition with Recurrent Neural Network |
-
2018
- 2018-09-06 CN CN201811037902.7A patent/CN109272033B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100124377A1 (en) * | 2008-11-19 | 2010-05-20 | Nec Laboratories America, Inc. | Linear spatial pyramid matching using sparse coding |
| US20110161944A1 (en) * | 2009-12-30 | 2011-06-30 | Samsung Electronics Co., Ltd. | Method and apparatus for transforming program code |
| CN105975907A (en) * | 2016-04-27 | 2016-09-28 | 江苏华通晟云科技有限公司 | SVM model pedestrian detection method based on distributed platform |
| CN106911580A (en) * | 2017-01-24 | 2017-06-30 | 中国科学院声学研究所 | A kind of TCP jamming control methods and system |
| CN107247968A (en) * | 2017-07-24 | 2017-10-13 | 东北林业大学 | Based on logistics equipment method for detecting abnormality under nuclear entropy constituent analysis imbalance data |
| CN107991877A (en) * | 2017-12-20 | 2018-05-04 | 东南大学 | A kind of Dynamic Model Identification method and system based on Recognition with Recurrent Neural Network |
Non-Patent Citations (2)
| Title |
|---|
| M.SAMI SOLIMAN 等: "Continuous query scheduler based on operators clustering", 《JOURNAL OF CENTRAL SOUTH UNIVERSITY OF TECHNOLOGY》 * |
| 丁立中 等: "基于正则化路径的支持向量机近似模型选择", 《计算机研究与发展》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109272033B (en) | 2022-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112069310B (en) | Text classification method and system based on active learning strategy | |
| EP3924893A1 (en) | Incremental training of machine learning tools | |
| CN110580496A (en) | Deep migration learning system and method based on entropy minimization | |
| KR102189688B1 (en) | Mehtod for extracting synonyms | |
| CN110349597B (en) | Voice detection method and device | |
| CN110875912A (en) | Network intrusion detection method, device and storage medium based on deep learning | |
| CN114841257B (en) | Small sample target detection method based on self-supervision comparison constraint | |
| CN111368920A (en) | Quantum twin neural network-based binary classification method and face recognition method thereof | |
| CN108710948B (en) | Transfer learning method based on cluster balance and weight matrix optimization | |
| CN110598869B (en) | Classification method and device based on sequence model and electronic equipment | |
| CN113723083B (en) | BERT model-based weighted passive supervision text emotion analysis method | |
| CN112418387A (en) | Quantum data processing method and apparatus | |
| JP2022530447A (en) | Chinese word division method based on deep learning, equipment, storage media and computer equipment | |
| CN112819020A (en) | Method and device for training classification model and classification method | |
| CN111127360A (en) | Gray level image transfer learning method based on automatic encoder | |
| Gopalakrishnan et al. | Sentiment analysis using simplified long short-term memory recurrent neural networks | |
| CN108154186B (en) | Pattern recognition method and device | |
| Fafalios et al. | Gradient boosting trees | |
| CN115905855A (en) | Improved meta-learning algorithm MG-copy | |
| JP2012181579A (en) | Pattern classification learning device | |
| CN113343801B (en) | Automatic wireless signal modulation and identification method based on lightweight convolutional neural network | |
| Fonseca et al. | Model-agnostic approaches to handling noisy labels when training sound event classifiers | |
| WO2021258482A1 (en) | Beauty prediction method and device based on migration and weak supervision, and storage medium | |
| CN109947945B (en) | Text data stream classification method based on word vector and integrated SVM | |
| CN116050419B (en) | Unsupervised identification method and system oriented to scientific literature knowledge entity |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Song Yunquan Inventor after: Li Yueling Inventor after: Yu Qi Inventor after: Lei Hejie Inventor after: Liang Xijun Inventor after: Gradual order Inventor before: Song Yunquan Inventor before: Lei Hejie Inventor before: Lv Cong Inventor before: Liang Xijun Inventor before: Gradual order |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |