CN103164709A - Method for optimizing support vector machine based on tabu search algorithm - Google Patents
Method for optimizing support vector machine based on tabu search algorithm Download PDFInfo
- Publication number
- CN103164709A CN103164709A CN 201210575338 CN201210575338A CN103164709A CN 103164709 A CN103164709 A CN 103164709A CN 201210575338 CN201210575338 CN 201210575338 CN 201210575338 A CN201210575338 A CN 201210575338A CN 103164709 A CN103164709 A CN 103164709A
- Authority
- CN
- China
- Prior art keywords
- parameter
- support vector
- vector machine
- algorithm
- tabu search
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to that selection of a kernel function and a parameter is considered first when a support vector machine is utilized to solve a specific problem. Although certain achievements on theoretical research and applications of the kernel function are achieved, a theory which instructs the selection of the parameter of the support vector machine is not formed. According to a method for optimizing a support vector machine based on a tabu search algorithm, a tabu search algorithm is utilized to conduct parameter optimization on the support vector machine based on the radial base kernel function. A classic tabu search algorithm is expanded, an eight-grid method is used for neighborhood generation, and the support vector machine can be automatically adjusted. Rate of convergence of the algorithm is improved on the premise of not losing accuracy, all surrounding locally-optimal solutions can be found by exploring locally-optimal places in a radioactive mode, and therefore global optimum can be achieved as far as possible. The algorithm is tested through a test function and a normal data set, the result shows that the improved algorithm can effectively find the global optimum and enable the support vector machine (SVM) to possess high classification accuracy rate.
Description
Technical field
The present invention relates to a kind of support vector machine optimization method, especially, relate to a kind of method based on the tabu search algorithm Support Vector Machines Optimized.
Background technology
Because support vector machine is widely used in various fields, its importance highlights day by day, and parameter selects directly to affect the quality of support vector machine identification target capabilities.The parameter of how to confirm support vector machine is the important content of research support vector machine, has also just let nature take its course into study hotspot.At present, have a lot of intelligent algorithms to be used to the optimization of support vector machine parameter, and tabu search algorithm is more and more favored with its higher finding the solution quality and efficient and obtain people in the powerful optimizing ability that many Combinatorial Optimization neighborhoods demonstrate.
The present invention is used for tabu search algorithm the optimization of support vector machine parameter, tabu search algorithm is had made some improvements and tests based on standard data set, and select to compare to show that algorithm of the present invention has performance preferably based on the support vector machine parameter in the structure risk upper bound.
Tabu search algorithm is avoided roundabout search by introducing taboo list with the taboo criterion, and absolves by the special pardon criterion excellent condition that some are avoided, and then guarantees the diversified efficient search that has, and finally realizes global optimization.At the document exercise question: " the SVM parameter based on the structure risk upper bound is selected ", author: Song Xiaoshan, Jiang Xiaoyu, Luo Jianhua, Wang Xi, scientific and technological Leader 2011, in 29 (08), provide the algorithm of structure risk supremum, proposed a kind of SVM parameter selection method based on the structure risk upper bound, can obtain the parameter that makes the SVM generalization stronger.
But the support vector machine adopting parameters is the unified standard of neither one still so far, and parameter is chosen most dependence experience and taked the method trying to gather, and is so not only time-consuming but also be difficult to obtain satisfied result.
Summary of the invention
When using support vector machine and solve particular problem, at first consider choosing and the selection of parameter of kernel function.Although obtained certain achievement about kernel function in theory research and application at present, not yet formed the theory that instructs the support vector machine parameter to choose.The technical problem to be solved in the present invention is: blindness and poor efficiency during for existing support vector machine Selecting All Parameters propose a kind of method based on the tabu search algorithm Support Vector Machines Optimized.By this algorithm, can be more targetedly and choose efficiently the support vector machine parameter, obtain near-optimization parameter in solution space by tabu search, and then make the classifying quality of support vector machine and global optimizing performance that to a certain extent lifting be arranged.
The present invention is achieved by the following technical solutions:
A kind of method based on the tabu search algorithm Support Vector Machines Optimized, raw data are classical trial function Shaffer ' s F6.
At first the first step uses tabu search algorithm that penalty factor and the kernel functional parameter of support vector machine are optimized.Specifically being divided into following components realizes:
First step A, the given algorithm parameter produces initial parameter c, γ at random, and the initialization taboo list also is set as sky with tab, respectively globally optimal solution best_glo and locally optimal solution best_loc are carried out initialization, putting escape state es was 0 (namely having shown the non-escape state that is initially in);
First step B, whether the evaluation algorithm end condition satisfies.If finish algorithm and export optimum results; Otherwise, continue following steps;
First step C calculates the number of times that parameter current is got, and sets accordingly radius of neighbourhood jump;
First step D records local optimum and records the step number that best_loc does not upgrade continuously, if this step number has surpassed predefined threshold value, can think that current solution is absorbed in local optimum.Algorithm begins to escape, and explores to obtain escaping a little, adds escape Candidate Set es_para, and putting escape state es is 1;
First step E judges whether to be in escape state (es=1), if radius of neighbourhood jump is set to 1, parameter is got the parameter in the escape Candidate Set successively, otherwise, continue following steps;
First step F produces the neighborhood solution of some, as candidate solution according to certain rule based on radius of neighbourhood jump;
First step G calculates the adaptation value of each neighborhood solution by SVM, namely should separate and produce corresponding classification accuracy rate, will sort from big to small according to accuracy to solution;
First step H makes a decision the element in Candidate Set successively: whether see accuracy corresponding to this parameter greater than the optimum record of local history, if greater than, replace the optimum record of local history, upgrade taboo list, with the starting point of this solution as next step search; Otherwise, step below continuing;
First step I judges that this parameter whether in taboo list, if do not exist, adds taboo list, with the starting point of this parameter as next step search; Otherwise judgement is next separates; If do not obtain greater than historical optimum record or the solution in taboo list not, with the starting point of optimal parameter as next step search;
First step J turns step (1.B).
Second step, the near-optimization parameter training that utilization obtains obtains optimum supporting vector machine model, and based on this trial function is tested, the ability of the classification performance of checking support vector machine and search globally optimal solution, the supporting vector machine model of use tabu search algorithm optimization treats optimizing function and classification samples is processed.
Beneficial effect of the present invention is: the present invention uses tabu search algorithm that the parameter of support vector machine is optimized, and introduces the structure that eight gridding methods have been expanded the neighborhood solution, and the escape machine of traditionally taboo search has been manufactured improvement.Thereby realized the disaggregated model of support vector machine is optimized, reduced blindness that parameter selects and inaccurate.Through our method improvement, Support Vector Machines Optimized all has a certain upgrade in the ability of finding the solution globally optimal solution and classifying quality; And, use tabu search algorithm to select the parameter of support vector machine, the time and effort consuming defective when having avoided traditional support vector machine Selecting All Parameters can disposablely obtain the near-optimization parameter exactly.The present invention utilizes tabu search algorithm to carry out parameter optimization to the support vector machine based on the radial basis kernel function.Tabu search algorithm to classics is expanded, and neighborhood generates and adopts eight gridding methods, and can automatically adjust.In the situation that loss of accuracy does not improve convergence of algorithm speed, and by the local optimum place to around radioactivity explore, all locally optimal solutions around finding, thus realize global optimum as much as possible.Algorithm is tested with trial function and standard data set respectively, and result shows that the algorithm of improved can find globally optimal solution effectively, makes SVM that higher classification accuracy rate be arranged.
Description of drawings
Fig. 1 is the structural drawing of tabu search algorithm neighborhood solution in the present invention.
Function Shaffer ' the s F6 that Fig. 2 adopts when being solved function optimum solution of the present invention.
Fig. 3 is that the accuracy rate of the present invention when being used for the classification of classical data set moved towards figure.
Fig. 4 is the inventive method flow chart of steps.
Embodiment
Further describe the present invention below in conjunction with embodiment.Scope of the present invention is not subjected to the restriction of these embodiment, and scope of the present invention proposes in claims.
As shown in Figure 4:
Beginning, setup parameter immediately, initialization record, es=0;
The calculating parameter frequency is established jump;
Calculate the step number that the local optimum record does not upgrade, judge whether to escape, if so, explore the escape point, add the escape Candidate Set, es=1, if not,, if es=1 makes jump=1, parameter is got the parameter in the escape Candidate Set, es=0 successively;
Generate Candidate Set, calculate accuracy;
The Candidate Set element sorts from big to small by accuracy; Judgment accuracy whether greater than local optimum, if not, needs whether to have judged all Candidate Set elements successively, if not, whether needs to rejudge accuracy greater than local optimum, and if so, this solution is as next step starting point;
Upgrade local, global optimum's record;
Add taboo list, first solution further judges whether to satisfy end condition as next step starting point, if so, introduces, and if not, gets back to beginning.
Particularly, tabu search algorithm parameter optimization design, as follows:
First step A, the given algorithm parameter produces initial parameter c, γ at random, and the initialization taboo list also is set as sky with tab, respectively globally optimal solution best_glo and locally optimal solution best_loc are carried out initialization, putting escape state es was 0 (namely having shown the non-escape state that is initially in); According to penalty factor c and the possible span of radial basis function g, choose integer c ∈ [0.1,100], g ∈ [0.1,1000].The variation range of penalty factor c and kernel functional parameter γ is respectively [0,100] and [0,1000].
First step B, whether the evaluation algorithm end condition satisfies.If finish algorithm and export optimum results; Otherwise, continue following steps.In the present invention, the termination rules of tabu search optimized algorithm is set as follows:
(1) a maximum iteration time value being set, can be for example 500 generations.After the number of run of algorithm reaches this value, no matter how current search condition all will stop the algorithm operation, return to up to now optimum solution and state.(2) set the maximum tabu frequency value of single object.For fear of convolution search, if certain state has occured in the algorithm operational process, corresponding its tabu frequency of function fitness value surpasses a certain predefined numerical value (15 times), stop algorithm and return results.
First step C calculates the number of times that parameter current is got, and sets accordingly radius of neighbourhood jump;
First step D records local optimum and records the step number that best_loc does not upgrade continuously, if this step number has surpassed predefined threshold value, can think that current solution is absorbed in local optimum.Algorithm begins to escape, and explores to obtain escaping a little, adds escape Candidate Set es_para, and putting escape state es is 1;
First step E judges whether to be in escape state (es=1), if radius of neighbourhood jump is set to 1, parameter is got the parameter in the escape Candidate Set successively, otherwise, continue following steps;
First step F produces the neighborhood solution of some, as candidate solution according to certain rule based on radius of neighbourhood jump;
First step G calculates the adaptation value of each neighborhood solution by SVM, namely should separate and produce corresponding classification accuracy rate, will sort from big to small according to accuracy to solution;
First step H makes a decision the element in Candidate Set successively: whether see accuracy corresponding to this parameter greater than the optimum record of local history, if greater than, replace the optimum record of local history, upgrade taboo list, with the starting point of this solution as next step search; Otherwise, step below continuing;
First step I judges that this parameter whether in taboo list, if do not exist, adds taboo list, with the starting point of this parameter as next step search; Otherwise judgement is next separates; If do not obtain greater than historical optimum record or the solution in taboo list not, with the starting point of optimal parameter as next step search;
First step J turns step (1.B).
Second step, the supporting vector machine model of use tabu search algorithm optimization treats optimizing function and classification samples is processed, and specifically is divided into following components and realizes:
Second step A based on the test of classical function, verifies the global optimizing ability of institute's algorithm;
Shaffer ' s F6 function is the propositions such as J.D.Shaffer, and expression formula is
Wherein, the span of independent variable is, [100,100].Fig. 2 is the figure of Shaffer ' s F6 function in [10,10] interval, and it has a unlimited local pole a little bigger, and only having one (0,0) is Global maximum, and maximal value is 1.At the maximum circle ridge that is with on weekly duty of this function, their value is 0.990283, simultaneously because this function has strong oscillation property and range of variables wide ranges, the hunting zone is large, therefore be easy to be absorbed in local pole a little bigger, and in case be absorbed in and just be difficult to jump out.
Utilize Shaffer ' s F6 function that tabu search algorithm in this paper is tested, setting and calculating step number was 100 steps, parameter c ∈ (10,10), step-length c_step=0.01, γ ∈ (10,10), step-length γ _ step=0.01 tests 10 times, and the result of test is as shown in table 4-1.
Table 1 Shaffer ' s F6 function test result
Appoint and get 1 test result as shown in Figure 3.The figure illustrates the increase along with the computing step number, the tendency of objective function.By map analysis, algorithm can be restrained rapidly, seeks better point by the escape process afterwards, shows Algorithm Performance or reasonable.
Second step B, the test of based on data collection.Do great many of experiments on the classical data set of support vector machine, and done Performance Ratio with method of the same type.
Data set information such as table 2, each data set test 10 times, experimental result is listed in table 3.
Table 2 data set information
| Data set | The training set size | The test set size | |
| Heart | |||
| 100 | 170 | 13 | |
| Australian | 200 | 490 | 14 |
| German | 600 | 400 | 24 |
| Vehicle | 400 | 445 | 18 |
Table 3 experimental result
Abovely be described with reference to the exemplary embodiment of accompanying drawing to the application.Those skilled in the art should understand that; above-mentioned embodiment is only the example of lifting for illustrative purposes; rather than be used for limiting; all in the application instruction and the claim protection domain under do any modification, be equal to replacement etc., all should be included in the claimed scope of the application.
Claims (5)
1. method based on the tabu search algorithm Support Vector Machines Optimized, raw data is that trial function Shaffer's F6 is characterized in that, comprises the following steps:
(1), use tabu search algorithm that penalty factor and the kernel functional parameter of support vector machine are optimized;
(2), utilize the near-optimization parameter training that obtains to obtain optimum supporting vector machine model, and based on this trial function is tested, the ability of the classification performance of checking support vector machine and search globally optimal solution.
2. the method based on the tabu search algorithm Support Vector Machines Optimized according to claim 1, is characterized in that, described step (1) comprises the following steps:
(1.A), the given algorithm parameter produces initial parameter c, γ at random, puts taboo list tab for empty,
Global optimum records best_glo, and local optimum records the best_loc initialization, and putting escape state es is 0, i.e. non-escape state;
(1.B), whether the evaluation algorithm end condition satisfies; If finish algorithm and export optimum results; Otherwise, continue following steps;
(1.C), calculate the number of times that parameter current is got, set accordingly radius of neighbourhood jump;
(1.D), record local optimum and record the step number that best_loc does not upgrade continuously, if after certain step number, local record does not upgrade yet, think that current solution is absorbed in local optimum, begin to escape, exploration obtains escaping a little, add escape Candidate Set es_para, putting escape state es is 1;
(1.E), judge whether the escape state, i.e. es=1, if radius of neighbourhood jump is set to 1, parameter is got the parameter in the escape Candidate Set successively, otherwise, continue following steps;
(1.F), produce its all neighborhood solutions based on radius of neighbourhood jump, as candidate solution;
(1.G), calculate classification accuracy rate corresponding to neighborhood solution by SVM, solution is sorted from big to small according to accuracy;
(1.H), successively the element in Candidate Set is made a decision: whether see accuracy corresponding to this parameter greater than the optimum record of local history, if greater than, replace the optimum record of local history, upgrade taboo list, with the starting point of this solution as next step search; Otherwise, step below continuing;
(1.I), judge that this parameter whether in taboo list, if do not exist, adds taboo list, with the starting point of this parameter as next step search; Otherwise judgement is next separates; If do not obtain greater than historical optimum record or the solution in taboo list not, with the starting point of optimal parameter as next step search;
(1.J), turn step (1.B).
3. the method based on the tabu search algorithm Support Vector Machines Optimized according to claim 1, it is characterized in that, described step (2) further uses the supporting vector machine model of tabu search algorithm optimization to treat optimizing function and classification samples is processed, and specifically is divided into following components and realizes:
(2.A), based on the test of classical function, verify the global optimizing ability of institute's algorithm;
Shaffer's F6 function is the propositions such as J.D.Shaffer, and expression formula is
Wherein, the span of independent variable is, [100,100]; Utilize Shaffer's F6 function that the tabu search algorithm that proposes is tested, setting and calculating step number was 100 steps, parameter c ∈ (10,10), and, long c_step=0.01, γ ∈ (10,10), the random footwear flag of step-length first Lu=test N time;
(2.B), the test of based on data collection: do experiment on the classical data set of support vector machine, and do Performance Ratio with method of the same type.
4. the method based on the tabu search algorithm Support Vector Machines Optimized according to claim 3, it is characterized in that, described step (2) is that Shaffer's F6 function is [10,10] in the interval, it has a unlimited local pole a little bigger, only has one (0,0) be Global maximum, maximal value is 1, and at the maximum circle ridge that is with on weekly duty of this function, their value is 0.990283.
5. the method based on the tabu search algorithm Support Vector Machines Optimized according to claim 3, is characterized in that, in described step (2), test is N time, can get N=10 time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201210575338 CN103164709A (en) | 2012-12-24 | 2012-12-24 | Method for optimizing support vector machine based on tabu search algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201210575338 CN103164709A (en) | 2012-12-24 | 2012-12-24 | Method for optimizing support vector machine based on tabu search algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103164709A true CN103164709A (en) | 2013-06-19 |
Family
ID=48587779
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201210575338 Pending CN103164709A (en) | 2012-12-24 | 2012-12-24 | Method for optimizing support vector machine based on tabu search algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103164709A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106604229A (en) * | 2016-12-27 | 2017-04-26 | 东南大学 | Indoor positioning method based on manifold learning and improved support vector machine |
| CN108073761A (en) * | 2016-11-14 | 2018-05-25 | 波音公司 | System and method for optimizing battery pack design |
| CN109857937A (en) * | 2019-01-25 | 2019-06-07 | 同济大学 | One kind being based on Sine mapping and segmented continuous TABU search data processing method parallel |
| CN110008546A (en) * | 2019-03-22 | 2019-07-12 | 西南交通大学 | Circular passageway facility method for arranging |
| CN110555989A (en) * | 2019-08-16 | 2019-12-10 | 华南理工大学 | Xgboost algorithm-based traffic prediction method |
| CN110728075A (en) * | 2019-10-25 | 2020-01-24 | 山西应用科技学院 | Method for optimizing cryogenic process parameters by MOA algorithm |
| CN111866877A (en) * | 2020-06-11 | 2020-10-30 | 南京邮电大学 | 5G physical layer security authentication method based on memory |
| CN112285761A (en) * | 2020-11-24 | 2021-01-29 | 南昌华亮光电有限责任公司 | System and method for segmenting crystal position spectrum data for plant PET |
| CN110221170B (en) * | 2019-06-05 | 2021-07-27 | 贵州电网有限责任公司 | Low-current grounding line selection method based on tabu search optimization RBF network |
-
2012
- 2012-12-24 CN CN 201210575338 patent/CN103164709A/en active Pending
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108073761A (en) * | 2016-11-14 | 2018-05-25 | 波音公司 | System and method for optimizing battery pack design |
| CN106604229A (en) * | 2016-12-27 | 2017-04-26 | 东南大学 | Indoor positioning method based on manifold learning and improved support vector machine |
| CN106604229B (en) * | 2016-12-27 | 2020-02-18 | 东南大学 | Indoor positioning method based on manifold learning and improved support vector machine |
| CN109857937A (en) * | 2019-01-25 | 2019-06-07 | 同济大学 | One kind being based on Sine mapping and segmented continuous TABU search data processing method parallel |
| CN110008546A (en) * | 2019-03-22 | 2019-07-12 | 西南交通大学 | Circular passageway facility method for arranging |
| CN110221170B (en) * | 2019-06-05 | 2021-07-27 | 贵州电网有限责任公司 | Low-current grounding line selection method based on tabu search optimization RBF network |
| CN110555989A (en) * | 2019-08-16 | 2019-12-10 | 华南理工大学 | Xgboost algorithm-based traffic prediction method |
| CN110728075A (en) * | 2019-10-25 | 2020-01-24 | 山西应用科技学院 | Method for optimizing cryogenic process parameters by MOA algorithm |
| CN111866877A (en) * | 2020-06-11 | 2020-10-30 | 南京邮电大学 | 5G physical layer security authentication method based on memory |
| CN112285761A (en) * | 2020-11-24 | 2021-01-29 | 南昌华亮光电有限责任公司 | System and method for segmenting crystal position spectrum data for plant PET |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103164709A (en) | Method for optimizing support vector machine based on tabu search algorithm | |
| Lai et al. | A new DBSCAN parameters determination method based on improved MVO | |
| CN105930856A (en) | Classification method based on improved DBSCAN-SMOTE algorithm | |
| CN106096727A (en) | A kind of network model based on machine learning building method and device | |
| CN106604229A (en) | Indoor positioning method based on manifold learning and improved support vector machine | |
| CN104766098A (en) | Construction method for classifier | |
| Wang et al. | A novel adaptive density-based spatial clustering of application with noise based on bird swarm optimization algorithm | |
| CN105005825B (en) | Method and system for predicting photovoltaic power based on dynamic neural network | |
| CN104573708A (en) | Ensemble-of-under-sampled extreme learning machine | |
| CN110991653A (en) | Method for classifying unbalanced data sets | |
| Derrac et al. | Stratified prototype selection based on a steady-state memetic algorithm: a study of scalability | |
| CN109657891A (en) | Load characteristic analysis method based on self-adaptive k-means + + algorithm | |
| von Lücken et al. | An overview on evolutionary algorithms for many‐objective optimization problems | |
| Ali et al. | A modified cultural algorithm with a balanced performance for the differential evolution frameworks | |
| CN103678681B (en) | The Multiple Kernel Learning sorting technique of the auto-adaptive parameter based on large-scale data | |
| CN105184486A (en) | Power grid business classification method based on directed acyclic graphs support vector machine | |
| CN110991494A (en) | Method for constructing prediction model based on improved moth optimization algorithm | |
| Guo et al. | Accelerating differential evolution based on a subset-to-subset survivor selection operator | |
| Ducange et al. | Multi-objective evolutionary fuzzy systems | |
| Guo et al. | Harris hawks optimization algorithm based on elite fractional mutation for data clustering | |
| Zhao et al. | A best firework updating information guided adaptive fireworks algorithm | |
| CN109948675A (en) | The method for constructing prediction model based on outpost's mechanism drosophila optimization algorithm on multiple populations | |
| CN106209614A (en) | A kind of net packet classifying method and device | |
| CN105550711A (en) | Firefly algorithm based selective ensemble learning method | |
| Gao et al. | A joint landscape metric and error image approach to unsupervised band selection for hyperspectral image classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20130619 |