CN106408031A - Super parameter optimization method of least squares support vector machine - Google Patents
Super parameter optimization method of least squares support vector machine Download PDFInfo
- Publication number
- CN106408031A CN106408031A CN201610866414.1A CN201610866414A CN106408031A CN 106408031 A CN106408031 A CN 106408031A CN 201610866414 A CN201610866414 A CN 201610866414A CN 106408031 A CN106408031 A CN 106408031A
- Authority
- CN
- China
- Prior art keywords
- itr
- algorithm
- optimum
- local
- population
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/12—Computing arrangements based on biological models using genetic models
- G06N3/126—Evolutionary algorithms, e.g. genetic algorithms or genetic programming
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Theoretical Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Biology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physiology (AREA)
- Genetics & Genomics (AREA)
- Biomedical Technology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention discloses a super parameter optimization method of least squares support vector machine (LS-SVM). On the basis of the least squares support vector machine model, aiming at the problem that the model super parameter optimization phase is likely to fall into local optimization, through combination of the particle swarm search algorithm, the genetic algorithm and the Logistic mapping, the two parameters of particle state description and local optimization threshold are introduced to provide a PSO-GA-LM algorithm so as to effectively solve the locality problem of the super parameter optimization and realize the more accurate prediction.
Description
Technical field
The present invention relates to a kind of super ginseng optimization method of new least square method supporting vector machine (LS-SVM), belong to optimization neck
Domain
Background technology
Least square method supporting vector machine (Least Squares Support Vector Machine, LSSVM) is a kind of
Follow the kernel function Learning machine of structural risk minimization, this model was proposed by Suykens first in 1999, this algorithm
From the angle of loss function, on the basis of standard SVM, direction is commented as object function using indifference, and using linear
The form of equation replaces quadratic programming as constraints.
LS-SVM model performance be subject to model hyper parameter (regularization parameter γ, core width cs) optimize, the selection of kernel function and
Support vector machine rarefaction etc. affects, and wherein model hyperparameter optimization directly affects matching and the Generalization Capability of LS-SVM.Cause
And surpass, with regard to LS-SVM, the extensive concern that ginseng optimization problem has drawn domestic and international research worker.There is scholar directly to choose satisfaction to set in advance
The parameter of fixed model prediction accuracy combines as optimal hyper parameter, to improve the efficiency of modeling with this.By grid optimizing
Rebuild network node centered on the optimal hyper parameter that algorithm obtains, realize multilamellar dynamic self-adapting hyperparameter optimization method,
Optimize LS-SVM model hyper parameter using three step search algorithm.Said method fully relies on experience to arrange the initial of super ginseng optimization
Value is although simple and clear, but is a lack of certain theoretical foundation, is easily trapped into local optimum.Meanwhile, there is scholar using change
Yardstick chaotic optimization algorithm determining the hyper parameter of model, although but chaotic motion no repeatedly can travel through in certain area
All hyper parameter, but have no ability to depart from this region, are still easily caused searching process and are absorbed in local optimum.Additionally, there being scholar to divide
Based on genetic algorithm, particle cluster algorithm do not obtain LS-SVM hyper parameter combination, although but these methods to a certain extent
Solve the problems, such as the local convergence of optimizing, but sensitive for initial value, and model initial assignment will directly influence the property of algorithm
Energy (convergence), and in searching process, also can be absorbed in local optimum.Thus, current super ginseng optimization method is not really effective
Solve the problems, such as local convergence.
Content of the invention
Present invention seek to address that set up surpassing ginseng optimization problem during LS-SVM model so that obtaining when setting up LS-SVM model
More accurately hyper parameter combination, and then improve generalization ability and the degree of accuracy of LS-SVM.Due to existing super ginseng optimized algorithm
Initial value relies primarily on empirical value, and optimizing algorithm is easily absorbed in locally optimal solution simultaneously, and the present invention proposes PSO-GA-LM algorithm, should
Algorithm is by population searching algorithm (Particle Swarm Optimization, PSO) and genetic algorithm (Genetic
Algorithm, GA), Logistics mapping (LM) combine formation new PSO-GA-LM surpass ginseng optimized algorithm.This algorithm is not only
Absorb genetic algorithm also to have chaos simultaneously under certain condition and reflect by outstanding gene genetic to follow-on excellent characteristic
Penetrate the advantage that (Logistic mapping) reinitializes, there is the preferable performance departing from local optimum, and can effectively carry
High least square method supporting vector machine precision of prediction.
The present invention employs the following technical solutions：
A kind of super ginseng optimization method of least square method supporting vector machine, including step：
Step 1：The initial population of random generation, optimum super ginseng combinesThe meter of note setting globally optimal solution
Number device T=0, iterationses itr=0；
Step 2：Calculate the fitness function of population, and try to achieve the minimum adaptive value particle combinations Best, T=of this colony
T+1；
Step 3：Judge the decision threshold N of T and local optimumlocalSize；If T≤Nlocal, then using genetic algorithm more
New particle：Fitness function according to population sorts to population, takes first n optimum particle individuality, it is carried out intersect,
Mutation operation is come individual, the itr=itr+1 that to substitute after sequence n particle；Otherwise regenerate particle using Logistic mapping
Group, itr=itr+1, reset T=0；
Step 4：The fitness function of the population that calculation procedure 3 obtains, the optimum super ginseng combination trying to achieve this colony is designated as
pBest；
Step 5：If Best=PBest, T=T+1；If Best<PBest, then keep constant；Otherwise make Best=
PBest, resets T=0 simultaneously；
Step 6：Judge iterationses whether itr ＞ exetime；If so, then stop, Best is required hyper parameter；No
Then return to step 3.
The decision threshold N of described local optimumlocalThere is the actually used situation of this algorithm related to application scenarios, by user certainly
Row sets.
The fitness function of described step 2 is error function.
The beneficial effect that the present invention reaches：1. in terms of the precision of prediction of LS-SVM, the PSO-GA-LM algorithm that proposed
The forecast model prediction accuracy set up is higher；2. super ginseng optimize initial value setting in terms of, this algorithm do not need with
Set super ginseng combination toward the same empirically value of algorithm, super ginseng combination can be generated at random；3. in super ginseng optimization algorithms SO-GA-LM
Convergence aspect, this algorithm has preferable convergence, and it can be avoided that the process of cruising is absorbed in local optimum.
Brief description
Fig. 1 is the flow chart of LS-SVM.
Fig. 2 is the flow chart of PSO-GA-LM.
Specific embodiment
In order that the objects, technical solutions and advantages of the present invention become more apparent, below in conjunction with drawings and Examples, right
The present invention is further elaborated.It should be appreciated that specific embodiment described herein is only in order to explain the present invention, and
It is not used in the restriction present invention.
As shown in figure 1, data set will be divided into training set and test set, training set is used for trying to achieve the super ginseng combination of optimum,
Using the PSO-GA-LM algorithm that we are proposed, LS-SVM parameter is optimized, and circulates and carry out, when reaching end condition
When, the super ginseng combination finally giving is exactly our required LS-SVM parameters, and the super ginseng combination tried to achieve is brought into LS-SVM model
LS-SVM model after being optimized.
The super ginseng optimization process of the present invention is used in the LS-SVM modelling phase, is tried to achieve using this algorithm using training data
The super ginseng combination of global optimum, and set up LS-SVM model using this parameter.Fig. 1 is the LS-SVM Establishing process using this algorithm
Figure；Fig. 2 is the PSO-GA-LM flow chart proposing.In super ginseng optimization process, go mean square error to be fitness function, then surpass ginseng
Optimal enforcement mode is as follows：
1) primary group, the enumerator T=0 of note setting globally optimal solution are generated；T refers to the counting of globally optimal solution
Device, optimum super ginseng combinesIterationses itr=0；
2) fitness function of calculating primary group, and try to achieve the minimum adaptive value particle combinations of this colony, it is designated as
Best, T=T+1；
3) if T≤Nlocal, then using genetic algorithm more new particle：According to fitness function, population is sorted, take front n
Optimum particle is individual, and it is intersected, and mutation operation is come individual, the itr=itr+1 that to substitute last n poor particle；By
According to ranking fitness, thus, front n of selection is individual for the less example of more outstanding error, and the specific size of n
Then to be made a concrete analysis of according to the condition that user is used and be defined；Otherwise go to 4)；
4) regenerate primary group, itr=itr+1 using Logistic mapping, reset T=0；
5) calculate 3) using genetic algorithm update after population fitness function, try to achieve this colony optimum surpass ginseng combination
It is designated as pBest；Because the error that algorithm takes is fitness function, thus error is little, illustrates that predictive value and actual value relatively connect
Closely；
6) if Best≤pBest, turn (7), otherwise turn (8)；
7) if Best=PBest, T=T+1；If Best<PBest, then keep constant；
8) reset T=0, make Best=pBest；If locally optimal solution i.e. now is less than globally optimal solution, by office
Portion's optimal solution is copied to globally optimal solution, and globally optimal solution enumerator is set to 0；
9) judge whether itr ＞ exetime；If so, then stop, Best is required hyper parameter；Otherwise turn (3)；
Wherein Exetime is maximum iteration time.
At present parameter optimization be have ignored with the problem that the global optimum tried to achieve may be absorbed in the locally optimal solution of the overall situation, this
Invention, during carrying out algorithm performs, increased the enumerator of a global optimum, when the number of globally optimal solution is more than
During threshold value set by user, reinitialize particle, re-search for global optimum, otherwise carry out cluster ion using genetic algorithm
Renewal.It is excellent that the new PSO-GA-LM of formation that population searching algorithm is combined with genetic algorithm and Logistics mapping surpasses ginseng
Change algorithm.This algorithm not only absorbs genetic algorithm can be by outstanding gene genetic to follow-on excellent characteristic, simultaneously certain
Under the conditions of also have the advantage that chaotic maps (Logistic mapping) reinitialize, there is the preferable property departing from local optimum
Can, and least square method supporting vector machine precision of prediction can be effectively improved.
Claims (3)
1. a kind of super ginseng optimization method of least square method supporting vector machine is it is characterised in that include step：
Step 1：The initial population of random generation, optimum super ginseng combinesThe enumerator of note setting globally optimal solution
T=0, iterationses itr=0；
Step 2：Calculate the fitness function of population, and try to achieve the minimum adaptive value particle combinations Best, T=T+1 of this colony；
Step 3：Judge the decision threshold N of T and local optimumlocalSize；If T≤Nlocal, then update grain using genetic algorithm
Son：Fitness function according to population sorts to population, takes first n optimum particle individual, it is carried out intersecting, makes a variation
Operate and to substitute n particle individuality, itr=itr+1 after sequence；Otherwise regenerate primary using Logistic mapping
Group, itr=itr+1, reset T=0；
Step 4：The fitness function of the population that calculation procedure 3 obtains, the optimum super ginseng combination trying to achieve this colony is designated as
pBest；
Step 5：If Best=PBest, T=T+1；If Best<PBest, then keep constant；Otherwise make Best=pBest, with
When reset T=0；
Step 6：Judge iterationses whether itr ＞ exetime；If so, then stop, Best is required hyper parameter；Otherwise return
Return step 3.
2. super ginseng optimization method according to claim 1 is it is characterised in that the decision threshold N of described local optimumlocalHave
The actually used situation of this algorithm is related to application scenarios, by user's sets itself.
3. super ginseng optimization method according to claim 1 is it is characterised in that the fitness function of described step 2 is error
Function.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610866414.1A CN106408031A (en) | 2016-09-29 | 2016-09-29 | Super parameter optimization method of least squares support vector machine |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610866414.1A CN106408031A (en) | 2016-09-29 | 2016-09-29 | Super parameter optimization method of least squares support vector machine |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106408031A true CN106408031A (en) | 2017-02-15 |
Family
ID=59228557
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610866414.1A Pending CN106408031A (en) | 2016-09-29 | 2016-09-29 | Super parameter optimization method of least squares support vector machine |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106408031A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109344968A (en) * | 2018-10-10 | 2019-02-15 | 郑州云海信息技术有限公司 | A kind of method and device of the hyper parameter processing of neural network |
| CN110252034A (en) * | 2019-05-10 | 2019-09-20 | 太原理工大学 | Biological 3D printing toilet bacterium degree control and monitoring system |
| CN110503632A (en) * | 2019-07-26 | 2019-11-26 | 南昌大学 | SVR parameter optimization method in a kind of blind image quality evaluation algorithm |
| CN111260077A (en) * | 2020-01-14 | 2020-06-09 | 支付宝(杭州)信息技术有限公司 | Method and device for determining hyper-parameters of business processing model |
| CN112199890A (en) * | 2020-10-11 | 2021-01-08 | 哈尔滨工程大学 | System-level fault diagnosis method for integrated nuclear power device |
| WO2021135025A1 (en) * | 2019-12-30 | 2021-07-08 | 上海依图网络科技有限公司 | Hyperparameter optimization apparatus and method |
| CN113466043A (en) * | 2021-07-19 | 2021-10-01 | 蒙娜丽莎集团股份有限公司 | Method for testing fracture toughness of ceramic rock plate |
| CN113536690A (en) * | 2021-07-30 | 2021-10-22 | 安徽容知日新科技股份有限公司 | Parameter adjusting method of model and computing device |
-
2016
- 2016-09-29 CN CN201610866414.1A patent/CN106408031A/en active Pending
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109344968A (en) * | 2018-10-10 | 2019-02-15 | 郑州云海信息技术有限公司 | A kind of method and device of the hyper parameter processing of neural network |
| CN110252034A (en) * | 2019-05-10 | 2019-09-20 | 太原理工大学 | Biological 3D printing toilet bacterium degree control and monitoring system |
| CN110252034B (en) * | 2019-05-10 | 2021-07-06 | 太原理工大学 | Biological 3D prints toilet's fungus degree control and monitoring system |
| CN110503632A (en) * | 2019-07-26 | 2019-11-26 | 南昌大学 | SVR parameter optimization method in a kind of blind image quality evaluation algorithm |
| CN110503632B (en) * | 2019-07-26 | 2022-08-09 | 南昌大学 | SVR parameter optimization method in blind image quality evaluation algorithm |
| WO2021135025A1 (en) * | 2019-12-30 | 2021-07-08 | 上海依图网络科技有限公司 | Hyperparameter optimization apparatus and method |
| CN111260077A (en) * | 2020-01-14 | 2020-06-09 | 支付宝(杭州)信息技术有限公司 | Method and device for determining hyper-parameters of business processing model |
| CN112199890A (en) * | 2020-10-11 | 2021-01-08 | 哈尔滨工程大学 | System-level fault diagnosis method for integrated nuclear power device |
| CN113466043A (en) * | 2021-07-19 | 2021-10-01 | 蒙娜丽莎集团股份有限公司 | Method for testing fracture toughness of ceramic rock plate |
| CN113536690A (en) * | 2021-07-30 | 2021-10-22 | 安徽容知日新科技股份有限公司 | Parameter adjusting method of model and computing device |
| CN113536690B (en) * | 2021-07-30 | 2024-02-27 | 安徽容知日新科技股份有限公司 | Parameter adjustment method of model and computing equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106408031A (en) | Super parameter optimization method of least squares support vector machine | |
| CN109451012B (en) | End cloud collaborative load balancing scheduling method, system and storage medium | |
| CN104731916A (en) | Optimizing initial center K-means clustering method based on density in data mining | |
| Yong | An improved artificial fish swarm algorithm for optimal operation of cascade reservoirs | |
| CN111240796B (en) | Load balancing scheduling method based on improved MMAS | |
| CN105373183A (en) | Method for tracking whole-situation maximum power point in photovoltaic array | |
| CN111767689A (en) | Three-dimensional integrated circuit layout method based on graphic processing | |
| CN112071122A (en) | Unmanned aerial vehicle group path planning method for information acquisition | |
| CN110119408A (en) | Mobile object continuous-query method under geographical space real-time streaming data | |
| WO2017049428A1 (en) | Spinning reserve capacity optimization method based on cost-performance ratio of reserve object | |
| Xiao | A clustering algorithm based on artificial fish school | |
| Su et al. | Research on virtual machine placement in the cloud based on improved simulated annealing algorithm | |
| CN103595652B (en) | The stage division of QoS efficiency in a kind of powerline network | |
| CN105678844B (en) | One kind is based on atural object scattered points increased profile construction method point by point | |
| CN112417761B (en) | Mooring truncation design method based on multi-objective cuckoo optimization algorithm | |
| CN106786499A (en) | Based on the short-term wind power forecast method for improving AFSA optimizations ELM | |
| Zhu et al. | Optimal schedule for agricultural machinery using an improved Immune-Tabu Search Algorithm | |
| CN107341193B (en) | Method for inquiring mobile object in road network | |
| CN109033159A (en) | A kind of diagram data layout method based on vertex influence power | |
| CN113075995A (en) | Virtual machine energy-saving integration method, system and storage medium based on mixed group intelligence | |
| CN106780747A (en) | A kind of method that Fast Segmentation CFD calculates grid | |
| CN109635473B (en) | Heuristic high-flux material simulation calculation optimization method | |
| CN115494840B (en) | Monte Carlo factor-based MC-IACO welding robot path planning method | |
| CN116431281A (en) | Virtual machine migration method based on whale optimization algorithm | |
| CN116542003A (en) | New energy charging station optimizing arrangement method based on reinforcement learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170215 |
|
| RJ01 | Rejection of invention patent application after publication |