CN103164709A - Method for optimizing support vector machine based on tabu search algorithm - Google Patents

Method for optimizing support vector machine based on tabu search algorithm Download PDF

Info

Publication number
CN103164709A
CN103164709A CN 201210575338 CN201210575338A CN103164709A CN 103164709 A CN103164709 A CN 103164709A CN 201210575338 CN201210575338 CN 201210575338 CN 201210575338 A CN201210575338 A CN 201210575338A CN 103164709 A CN103164709 A CN 103164709A
Authority
CN
China
Prior art keywords
support vector
vector machine
algorithm
parameters
tabu search
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN 201210575338
Other languages
Chinese (zh)
Inventor
苏华
孙学梅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin Polytechnic University
Original Assignee
Tianjin Polytechnic University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin Polytechnic University filed Critical Tianjin Polytechnic University
Priority to CN 201210575338 priority Critical patent/CN103164709A/en
Publication of CN103164709A publication Critical patent/CN103164709A/en
Pending legal-status Critical Current

Links

Images

Landscapes

  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明涉及在应用支持向量机解决具体问题时,首先考虑核函数的选取及参数的选择。虽然目前关于核函数在理论研究和应用上取得了一定的成果,但尚未形成指导支持向量机参数选取的理论。本发明利用禁忌搜索算法对基于径向基核函数的支持向量机进行参数优化。对经典的禁忌搜索算法进行了扩展,邻域生成采用八网格法,并且能够自动调整。在不损失精度的情况下对算法的收敛速度进行了改善,并通过在局部最优处对周围的放射性探索,找到周围所有的局部最优解,从而实现尽可能的全局最优。对算法分别用测试函数和标准数据集进行了测试,结果表明改进过的算法能够有效地找到全局最优解,使得SVM有较高的分类正确率。

Figure 201210575338

The invention relates to firstly considering the selection of kernel function and the selection of parameters when applying the support vector machine to solve specific problems. Although some achievements have been made in the theoretical research and application of the kernel function, the theory to guide the selection of support vector machine parameters has not yet been formed. The invention uses a tabu search algorithm to optimize the parameters of the support vector machine based on the radial basis kernel function. The classic tabu search algorithm is extended, and the neighborhood generation adopts the eight-grid method, which can be adjusted automatically. The convergence speed of the algorithm is improved without loss of accuracy, and all the local optimal solutions around are found by radially exploring the surrounding at the local optimum, so as to achieve the global optimum as much as possible. The algorithm is tested with the test function and the standard data set, and the results show that the improved algorithm can effectively find the global optimal solution, so that SVM has a higher classification accuracy.

Figure 201210575338

Description

一种基于禁忌搜索算法优化支持向量机的方法A Method of Optimizing Support Vector Machine Based on Tabu Search Algorithm

技术领域 technical field

本发明涉及一种支持向量机优化方法,特别地,涉及一种基于禁忌搜索算法优化支持向量机的方法。The invention relates to a method for optimizing a support vector machine, in particular to a method for optimizing a support vector machine based on a tabu search algorithm.

背景技术 Background technique

由于支持向量机在众多领域应用广泛,其重要性日益凸显,而参数选择直接影响支持向量机识别目标性能的优劣。如何确定支持向量机的参数是研究支持向量机的重要内容,也就顺其自然成了研究热点。目前,有很多智能算法被用于支持向量机参数的优化,而禁忌搜索算法以其较高的求解质量和效率以及在诸多组合优化邻域显示出的强大的寻优能力得到人们越来越多的青睐。Since the support vector machine is widely used in many fields, its importance is becoming more and more prominent, and the parameter selection directly affects the performance of the support vector machine to identify the target. How to determine the parameters of the support vector machine is an important content of the study of the support vector machine, and it has naturally become a research hotspot. At present, many intelligent algorithms are used to optimize the parameters of support vector machines, and the tabu search algorithm is getting more and more attention because of its high solution quality and efficiency and its powerful optimization ability in many combinatorial optimization neighborhoods. of favor.

本发明将禁忌搜索算法用于支持向量机参数的优化,对禁忌搜索算法做了一些改进并基于标准数据集进行了测试,与基于结构风险上界的支持向量机参数选择相比表明本发明算法有较好地性能。The present invention uses the tabu search algorithm for the optimization of support vector machine parameters, makes some improvements to the tabu search algorithm and tests it based on a standard data set, and compares it with the support vector machine parameter selection based on the upper bound of structural risk to show that the algorithm of the present invention Have better performance.

禁忌搜索算法通过引入禁忌表和禁忌准则来避免迂回搜索,并通过特赦准则来赦免一些被禁忌的优良状态,进而保证多样化的有效搜索,最终实现全局优化。在文献题目:《基于结构风险上界的SVM参数选择》,作者:宋小杉,蒋晓瑜,罗建华,汪熙,科技导报2011,29(08)中,给出了结构风险上界的算法,提出了一种基于结构风险上界的SVM参数选择方法,可以得到使SVM泛化性较强的参数。The tabu search algorithm avoids roundabout searches by introducing a tabu list and a taboo criterion, and pardons some tabooed good states through the amnesty criterion, thereby ensuring a variety of effective searches and finally achieving global optimization. In the literature title: "SVM parameter selection based on the upper bound of structural risk", authors: Song Xiaoshan, Jiang Xiaoyu, Luo Jianhua, Wang Xi, Science and Technology Herald 2011, 29 (08), gave the algorithm of the upper bound of structural risk, proposed A SVM parameter selection method based on the upper bound of structural risk can obtain parameters that make SVM more generalizable.

但是支持向量机参数的选取至今仍然没有一个统一的标准,参数选取大多依靠经验采取试凑的方法,这样不仅费时而且很难得到满意的结果。However, there is still no uniform standard for the selection of support vector machine parameters, and the selection of parameters mostly relies on experience and adopts a trial and error method, which is not only time-consuming but also difficult to obtain satisfactory results.

发明内容 Contents of the invention

在应用支持向量机解决具体问题时,首先考虑核函数的选取及参数的选择。虽然目前关于核函数在理论研究和应用上取得了一定的成果,但尚未形成指导支持向量机参数选取的理论。本发明要解决的技术问题是:针对现有支持向量机选取参数时的盲目性和低效率,提出一种基于禁忌搜索算法优化支持向量机的方法。通过该算法,能够更加有针对性和高效地选取支持向量机参数,通过禁忌搜索得到解空间内的近似最优参数,进而使得支持向量机的分类效果和全局寻优性能有一定程度上的提升。When applying support vector machine to solve specific problems, the choice of kernel function and parameters should be considered first. Although some achievements have been made in the theoretical research and application of the kernel function, the theory to guide the parameter selection of the support vector machine has not yet been formed. The technical problem to be solved by the present invention is to propose a method for optimizing the support vector machine based on a tabu search algorithm in view of the blindness and low efficiency of selecting parameters of the existing support vector machine. Through this algorithm, the parameters of the support vector machine can be selected more specifically and efficiently, and the approximate optimal parameters in the solution space can be obtained through the tabu search, thereby improving the classification effect and global optimization performance of the support vector machine to a certain extent .

本发明是通过以下技术方案实现的:The present invention is achieved through the following technical solutions:

一种基于禁忌搜索算法优化支持向量机的方法,原始数据是经典的测试函数Shaffer’s F6。A method for optimizing support vector machines based on the tabu search algorithm, the original data is the classic test function Shaffer's F6.

第一步首先使用禁忌搜索算法对支持向量机的惩罚因子和核函数参数进行优化。具体分为以下几个部分来实现:The first step is to use the tabu search algorithm to optimize the penalty factor and kernel function parameters of the support vector machine. Specifically, it is divided into the following parts to achieve:

第一步A,给定算法参数,随机产生初始参数c、γ,初始化禁忌表并将tab设定为空,分别对全局最优解best_glo和局部最优解best_loc进行初始化,置逃逸状态es为0(即表明了初始处于非逃逸状态);The first step A, given the algorithm parameters, randomly generates the initial parameters c and γ, initializes the taboo table and sets tab to be empty, respectively initializes the global optimal solution best_glo and the local optimal solution best_loc, and sets the escape state es as 0 (that is, it indicates that it is initially in a non-escape state);

第一步B,判断算法终止条件是否满足。若是,则结束算法并输出优化结果;否则,继续以下步骤;The first step B is to judge whether the termination condition of the algorithm is satisfied. If so, end the algorithm and output the optimization result; otherwise, continue with the following steps;

第一步C,计算当前参数取到的次数,据此设定邻域半径jump;The first step C is to calculate the number of times the current parameter is obtained, and set the neighborhood radius jump accordingly;

第一步D,记录局部最优记录best_loc连续未更新的步数,如果这一步数超过了预先设定的阈值,可以认为当前解陷入局部最优。算法开始逃逸,探索得到逃逸点,加入逃逸候选集es_para,置逃逸状态es为1;The first step D is to record the number of consecutive unupdated steps of the local optimal record best_loc. If the number of steps exceeds the preset threshold, it can be considered that the current solution falls into the local optimal. The algorithm starts to escape, explore and get the escape point, add the escape candidate set es_para, and set the escape state es to 1;

第一步E,判断是否处于逃逸状态(es=1),若是,邻域半径jump置为1,参数依次取逃逸候选集中的参数,否则,继续以下步骤;The first step E is to judge whether it is in the escape state (es=1), if so, the neighborhood radius jump is set to 1, and the parameters are successively taken from the parameters in the escape candidate set, otherwise, continue the following steps;

第一步F,基于邻域半径jump按照一定规则产生一定数量的邻域解,作为候选解;In the first step F, based on the neighborhood radius jump, a certain number of neighborhood solutions are generated according to certain rules as candidate solutions;

第一步G,通过SVM计算得到每个邻域解的适配值,即该解产生对应的分类正确率,将对解按照正确率从大到小排序;In the first step G, the adaptation value of each neighborhood solution is obtained through SVM calculation, that is, the corresponding classification accuracy rate is generated by the solution, and the solutions are sorted according to the accuracy rate from large to small;

第一步H,依次对候选集中的元素做判断:看该参数对应的正确率是否大于局部历史最优记录,如果大于,则替换局部历史最优记录,更新禁忌表,将该解作为下一步搜索的起始点;否则,继续下面步骤;The first step H is to make judgments on the elements in the candidate set in turn: check whether the correct rate corresponding to this parameter is greater than the local historical optimal record, if it is greater, replace the local historical optimal record, update the taboo table, and use this solution as the next step The starting point of the search; otherwise, proceed to the following steps;

第一步I,判断该参数是否在禁忌表中,如果不在,加入禁忌表,将该参数作为下一步搜索的起始点;否则,判断下一个解;如果没有得到大于历史最优记录的或者不在禁忌表中的解,将最佳参数作为下一步搜索的起始点;The first step I is to judge whether the parameter is in the taboo list, if not, add the taboo list, and use this parameter as the starting point of the next search; otherwise, judge the next solution; if it is not obtained greater than the historical optimal record or not in The solution in the tabu table, using the best parameters as the starting point for the next search;

第一步J,转步骤(1.B)。In the first step J, turn to step (1.B).

第二步,利用得到的近似最优参数训练得到最优支持向量机模型,并以此为基础对测试函数进行实验,验证支持向量机的分类性能和搜索全局最优解的能力,使用禁忌搜索算法优化过的支持向量机模型对待寻优函数和分类样本进行处理。The second step is to use the obtained approximate optimal parameters to train to obtain the optimal support vector machine model, and to conduct experiments on the test function based on this, to verify the classification performance of the support vector machine and the ability to search for the global optimal solution, using the tabu search The algorithm-optimized support vector machine model processes the optimization function and classification samples.

本发明的有益效果在于:本发明使用禁忌搜索算法对支持向量机的参数进行优化,引入八网格法扩展了邻域解的结构,并对传统禁忌搜索的逃逸机制做了改进。从而实现了对支持向量机的分类模型进行了优化,降低了参数选择的盲目性和不准确。经过我们的方法改进,优化支持向量机在求解全局最优解的能力和分类效果都有一定的提升;而且,使用禁忌搜索算法选择支持向量机的参数,避免了传统支持向量机选取参数时的耗时费力缺陷,能够一次性准确地得到近似最优参数。本发明利用禁忌搜索算法对基于径向基核函数的支持向量机进行参数优化。对经典的禁忌搜索算法进行了扩展,邻域生成采用八网格法,并且能够自动调整。在不损失精度的情况下对算法的收敛速度进行了改善,并通过在局部最优处对周围的放射性探索,找到周围所有的局部最优解,从而实现尽可能的全局最优。对算法分别用测试函数和标准数据集进行了测试,结果表明改进过的算法能够有效地找到全局最优解,使得SVM有较高的分类正确率。The beneficial effect of the present invention is that: the present invention uses the tabu search algorithm to optimize the parameters of the support vector machine, introduces the eight-grid method to expand the structure of the neighborhood solution, and improves the escape mechanism of the traditional tabu search. Therefore, the classification model of the support vector machine is optimized, and the blindness and inaccuracy of parameter selection are reduced. After the improvement of our method, the optimized support vector machine has a certain improvement in the ability to solve the global optimal solution and the classification effect; moreover, using the tabu search algorithm to select the parameters of the support vector machine avoids the traditional support vector machine when selecting parameters. Time-consuming and labor-intensive defect, can accurately obtain approximate optimal parameters at one time. The invention uses a tabu search algorithm to optimize the parameters of the support vector machine based on the radial basis kernel function. The classic tabu search algorithm is extended, and the neighborhood generation adopts the eight-grid method, which can be adjusted automatically. The convergence speed of the algorithm is improved without loss of accuracy, and all the local optimal solutions around are found by radially exploring the surrounding at the local optimum, so as to achieve the global optimum as much as possible. The algorithm is tested with the test function and the standard data set, and the results show that the improved algorithm can effectively find the global optimal solution, so that SVM has a higher classification accuracy.

附图说明 Description of drawings

图1是本发明中禁忌搜索算法邻域解的结构图。Fig. 1 is a structural diagram of the neighborhood solution of the tabu search algorithm in the present invention.

图2是本发明求解函数最优解时所采用的函数Shaffer’s F6。Fig. 2 is the function Shaffer's F6 that adopts when solving function optimal solution of the present invention.

图3是本发明用于经典数据集分类时的准确率走向图。Fig. 3 is a trend diagram of the accuracy rate when the present invention is used for classical data set classification.

图4是本发明方法步骤流程图。Fig. 4 is a flowchart of the method steps of the present invention.

具体实施方式 Detailed ways

下面结合实施例进一步描述本发明。本发明的范围不受这些实施例的限制,本发明的范围在权利要求书中提出。The present invention is further described below in conjunction with embodiment. The scope of the present invention is not limited by these examples, and the scope of the present invention is set forth in the claims.

如图4所示:As shown in Figure 4:

开始,随即设定参数,初始化记录,es=0;Start, then set the parameters, initialize the record, es=0;

计算参数频数,设jump;Calculate the parameter frequency, set jump;

计算局部最优记录未更新的步数,判断是否逃逸,如果是,则探索逃逸点,加入逃逸候选集,es=1,如果否,则,如果es=1,令jump=1,参数依次取逃逸候选集中的参数,es=0;Calculate the number of unupdated steps of the local optimal record, judge whether to escape, if yes, explore the escape point, add the escape candidate set, es=1, if not, then, if es=1, let jump=1, the parameters are taken in turn Parameters in the escape candidate set, es=0;

生成候选集,计算正确率;Generate a candidate set and calculate the correct rate;

候选集元素按正确率从大到小排序;依次判断正确率是否大于局部最优,如果否,则需要是否判断完所有候选集元素,如果否,则需要重新判断正确率是否大于局部最优,如果是,则该解作为下一步起始点;The elements of the candidate set are sorted from the largest to the smallest according to the correct rate; judge in turn whether the correct rate is greater than the local optimum, if not, it is necessary to judge whether all the candidate set elements have been judged, if not, it is necessary to re-judge whether the correct rate is greater than the local optimum, If yes, the solution is used as the starting point for the next step;

更新局部、全局最优记录;Update local and global optimal records;

加入禁忌表,第一个解作为下一步起始点,进一步判断是否满足终止条件,如果是,则介绍,如果否,则回到开始。Add the taboo table, the first solution is used as the starting point for the next step, further judge whether the termination condition is met, if yes, introduce, if not, return to the beginning.

具体地,禁忌搜索算法参数寻优设计,如下:Specifically, the parameter optimization design of the tabu search algorithm is as follows:

第一步A,给定算法参数,随机产生初始参数c、γ,初始化禁忌表并将tab设定为空,分别对全局最优解best_glo和局部最优解best_loc进行初始化,置逃逸状态es为0(即表明了初始处于非逃逸状态);根据惩罚因子c和径向基函数g可能的取值范围,选取整数c∈[0.1,100],g∈[0.1,1000]。惩罚因子c和核函数参数γ的变化范围分别是[0,100]和[0,1000]。The first step A, given the algorithm parameters, randomly generates the initial parameters c and γ, initializes the taboo table and sets tab to be empty, respectively initializes the global optimal solution best_glo and the local optimal solution best_loc, and sets the escape state es as 0 (that is, it indicates that it is initially in a non-escape state); according to the possible value range of the penalty factor c and the radial basis function g, the integers c∈[0.1, 100], g∈[0.1, 1000] are selected. The variation ranges of penalty factor c and kernel function parameter γ are [0, 100] and [0, 1000] respectively.

第一步B,判断算法终止条件是否满足。若是,则结束算法并输出优化结果;否则,继续以下步骤。本发明中禁忌搜索优化算法的终止规则设定如下:The first step B is to judge whether the termination condition of the algorithm is satisfied. If yes, end the algorithm and output the optimization result; otherwise, continue with the following steps. The termination rule setting of tabu search optimization algorithm among the present invention is as follows:

(1)设置一个最大迭代次数值,譬如可以是500代。当算法的运行次数达到该值以后,不论当前的搜索状态如何都要终止算法运行,返回迄今为止的最佳解和状态。(2)设定单个对象的最大禁忌频率值。为了避免回旋搜索,若算法运行过程中发生了某个状态、对应的函数适应度值其禁忌频率超过某一预先设定的数值(15次),则终止算法并返回结果。(1) Set a maximum number of iterations, for example, 500 generations. When the running times of the algorithm reach this value, regardless of the current search state, the algorithm will be terminated and the best solution and state so far will be returned. (2) Set the maximum taboo frequency value of a single object. In order to avoid roundabout search, if a certain state occurs during the operation of the algorithm, and the taboo frequency of the corresponding function fitness value exceeds a preset value (15 times), the algorithm is terminated and the result is returned.

第一步C,计算当前参数取到的次数,据此设定邻域半径jump;The first step C is to calculate the number of times the current parameter is obtained, and set the neighborhood radius jump accordingly;

第一步D,记录局部最优记录best_loc连续未更新的步数,如果这一步数超过了预先设定的阈值,可以认为当前解陷入局部最优。算法开始逃逸,探索得到逃逸点,加入逃逸候选集es_para,置逃逸状态es为1;The first step D is to record the number of consecutive unupdated steps of the local optimal record best_loc. If the number of steps exceeds the preset threshold, it can be considered that the current solution falls into the local optimal. The algorithm starts to escape, explore and get the escape point, add the escape candidate set es_para, and set the escape state es to 1;

第一步E,判断是否处于逃逸状态(es=1),若是,邻域半径jump置为1,参数依次取逃逸候选集中的参数,否则,继续以下步骤;The first step E is to judge whether it is in the escape state (es=1), if so, the neighborhood radius jump is set to 1, and the parameters are successively taken from the parameters in the escape candidate set, otherwise, continue the following steps;

第一步F,基于邻域半径jump按照一定规则产生一定数量的邻域解,作为候选解;In the first step F, based on the neighborhood radius jump, a certain number of neighborhood solutions are generated according to certain rules as candidate solutions;

第一步G,通过SVM计算得到每个邻域解的适配值,即该解产生对应的分类正确率,将对解按照正确率从大到小排序;In the first step G, the adaptation value of each neighborhood solution is obtained through SVM calculation, that is, the corresponding classification accuracy rate is generated by the solution, and the solutions are sorted according to the accuracy rate from large to small;

第一步H,依次对候选集中的元素做判断:看该参数对应的正确率是否大于局部历史最优记录,如果大于,则替换局部历史最优记录,更新禁忌表,将该解作为下一步搜索的起始点;否则,继续下面步骤;The first step H is to make judgments on the elements in the candidate set in turn: check whether the correct rate corresponding to this parameter is greater than the local historical optimal record, if it is greater, replace the local historical optimal record, update the taboo table, and use this solution as the next step The starting point of the search; otherwise, proceed to the following steps;

第一步I,判断该参数是否在禁忌表中,如果不在,加入禁忌表,将该参数作为下一步搜索的起始点;否则,判断下一个解;如果没有得到大于历史最优记录的或者不在禁忌表中的解,将最佳参数作为下一步搜索的起始点;The first step I is to judge whether the parameter is in the taboo list, if not, add the taboo list, and use this parameter as the starting point of the next search; otherwise, judge the next solution; if it is not obtained greater than the historical optimal record or not in The solution in the tabu table, using the best parameters as the starting point for the next search;

第一步J,转步骤(1.B)。In the first step J, turn to step (1.B).

第二步,使用禁忌搜索算法优化过的支持向量机模型对待寻优函数和分类样本进行处理,具体分为以下几个部分来实现:The second step is to use the support vector machine model optimized by the tabu search algorithm to process the optimization function and classification samples, which are divided into the following parts:

第二步A,基于经典函数的测试,验证所提出算法的全局寻优能力;The second step A is to verify the global optimization ability of the proposed algorithm based on the test of classical functions;

Shaffer’s F6函数是J.D.Shaffer等提出的,表达式为Shaffer’s F6 function was proposed by J.D.Shaffer et al., the expression is

ff (( xx ,, ythe y )) == 0.50.5 -- sinsin 22 xx 22 ++ ythe y 22 -- 0.50.5 [[ 11 ++ 0.0010.001 (( xx 22 ++ ythe y 22 )) ]] 22

其中,自变量的取值范围为,[-100,100]。图2为Shaffer’s F6函数在[-10,10]区间内的图形,它有无限个局部极大点,只有一个(0,0)为全局最大,最大值为1。在此函数最大值周围有一个圈脊,它们的取值均为0.990283,同时由于该函数具有强烈的振荡性质以及变量区间范围宽、搜索范围大的特点,因此很容易陷入局部极大点,而且一旦陷入就很难跳出。Among them, the value range of the independent variable is [-100, 100]. Figure 2 is the graph of Shaffer’s F6 function in the [-10, 10] interval, it has infinite local maximum points, only one (0, 0) is the global maximum, and the maximum value is 1. There is a circular ridge around the maximum value of this function, and their values are all 0.990283. At the same time, because this function has a strong oscillation property, a wide range of variable intervals, and a large search range, it is easy to fall into a local maximum point, and Once you're in it's hard to get out.

利用Shaffer’s F6函数对本文提出的禁忌搜索算法进行测试,设定计算步数为100步,参数c∈(-10,10),步长c_step=0.01,γ∈(-10,10),步长γ_step=0.01,测试10次,测试的结果如表4-1所示。Use the Shaffer's F6 function to test the tabu search algorithm proposed in this paper, set the number of calculation steps as 100 steps, parameter c∈(-10, 10), step size c_step=0.01, γ∈(-10, 10), step size γ_step=0.01, test 10 times, the test results are shown in Table 4-1.

表1 Shaffer’s F6函数测试结果Table 1 Shaffer’s F6 function test results

Figure BDA00002637974600062
Figure BDA00002637974600062

Figure BDA00002637974600071
Figure BDA00002637974600071

任取1个测试结果如图3所示。该图显示了随着运算步数的增加,目标函数的走势。由图分析,算法可以快速地收敛,之后通过逃逸过程寻找更好的点,表明算法的性能还是比较好的。Take any one test result as shown in Figure 3. The figure shows the trend of the objective function as the number of operation steps increases. From the graph analysis, the algorithm can converge quickly, and then find a better point through the escape process, which shows that the performance of the algorithm is still relatively good.

第二步B,基于数据集的测试。在支持向量机的经典数据集上做了大量实验,并与同类型方法做了性能比较。The second step B, the test based on the data set. A large number of experiments have been done on the classic data sets of support vector machines, and performance comparisons have been made with similar methods.

数据集信息如表2,每个数据集测试10次,实验结果列在表3中。The data set information is shown in Table 2, each data set is tested 10 times, and the experimental results are listed in Table 3.

表2 数据集信息Table 2 Dataset information

  数据集 data set   训练集大小 training set size   测试集大小 Test set size   数据维度 data dimension   Heart heart   100 100   170 170   13 13   Australian Australian   200 200   490 490   14 14   German German   600 600   400 400   24 twenty four   Vehicle Vehicle   400 400   445 445   18 18

表3 实验结果Table 3 Experimental results

Figure BDA00002637974600072
Figure BDA00002637974600072

Figure BDA00002637974600081
Figure BDA00002637974600081

以上参照附图对本申请的示例性的实施方案进行了描述。本领域技术人员应该理解,上述实施方案仅仅是为了说明的目的而所举的示例,而不是用来进行限制,凡在本申请的教导和权利要求保护范围下所作的任何修改、等同替换等,均应包含在本申请要求保护的范围内。The exemplary embodiments of the present application are described above with reference to the accompanying drawings. It should be understood by those skilled in the art that the above-mentioned embodiments are only examples for the purpose of illustration, and are not used for limitation. Any modifications, equivalent replacements, etc. All should be included in the protection scope of this application.

Claims (5)

1.一种基于禁忌搜索算法优化支持向量机的方法,原始数据是测试函数Shaffer's F6其特征在于,包括以下步骤:1. A method based on tabu search algorithm optimization support vector machine, original data is that test function Shaffer's F6 is characterized in that, comprises the following steps: (1),使用禁忌搜索算法对支持向量机的惩罚因子和核函数参数进行优化;(1), use the tabu search algorithm to optimize the penalty factor and kernel function parameters of the support vector machine; (2),利用得到的近似最优参数训练得到最优支持向量机模型,并以此为基础对测试函数进行实验,验证支持向量机的分类性能和搜索全局最优解的能力。(2) Use the obtained approximate optimal parameters to train the optimal support vector machine model, and based on this, conduct experiments on the test function to verify the classification performance of the support vector machine and the ability to search for the global optimal solution. 2.根据权利要求1所述的基于禁忌搜索算法优化支持向量机的方法,其特征在于,所述的步骤(1)包括以下步骤:2. The method for optimizing a support vector machine based on a tabu search algorithm according to claim 1, wherein said step (1) comprises the following steps: (1.A),给定算法参数,随机产生初始参数c、γ,置禁忌表tab为空,(1.A), the algorithm parameters are given, the initial parameters c and γ are randomly generated, and the taboo table tab is set to empty, 全局最优记录best_glo,局部最优记录best_loc初始化,置逃逸状态es为0,即非逃逸状态;The global optimal record best_glo, the local optimal record best_loc are initialized, and the escape state es is set to 0, that is, the non-escape state; (1.B),判断算法终止条件是否满足;若是,则结束算法并输出优化结果;否则,继续以下步骤;(1.B), determine whether the algorithm termination condition is satisfied; if so, end the algorithm and output the optimization result; otherwise, continue the following steps; (1.C),计算当前参数取到的次数,据此设定邻域半径jump;(1.C), calculate the number of times the current parameter is fetched, and set the neighborhood radius jump accordingly; (1.D),记录局部最优记录best_loc连续未更新的步数,如果一定步数之后局部记录仍未更新,认为当前解陷入局部最优,开始逃逸,探索得到逃逸点,加入逃逸候选集es_para,置逃逸状态es为1;(1.D), record the number of consecutive unupdated steps of the local optimal record best_loc, if the local record has not been updated after a certain number of steps, it is considered that the current solution is trapped in the local optimal, start to escape, explore the escape point, and join the escape candidate set es_para, set the escape state es to 1; (1.E),判断是否逃逸状态,即es=1,若是,邻域半径jump置为1,参数依次取逃逸候选集中的参数,否则,继续以下步骤;(1.E), determine whether it is in escape state, that is, es=1, if so, set the neighborhood radius jump to 1, and take the parameters in the escape candidate set in turn, otherwise, continue the following steps; (1.F),基于邻域半径jump产生其所有邻域解,作为候选解;(1.F), based on the neighborhood radius jump, generate all its neighborhood solutions as candidate solutions; (1.G),通过SVM计算得到邻域解对应的分类正确率,将解按照正确率从大到小排序;(1.G), the classification accuracy rate corresponding to the neighborhood solution is obtained through SVM calculation, and the solutions are sorted according to the accuracy rate from large to small; (1.H),依次对候选集中的元素做判断:看该参数对应的正确率是否大于局部历史最优记录,如果大于,则替换局部历史最优记录,更新禁忌表,将该解作为下一步搜索的起始点;否则,继续下面步骤;(1.H), make judgments on the elements in the candidate set in turn: check whether the correct rate corresponding to this parameter is greater than the local historical optimal record, if it is greater, replace the local historical optimal record, update the taboo table, and use the solution as the following The starting point of one-step search; otherwise, continue to the following steps; (1.I),判断该参数是否在禁忌表中,如果不在,加入禁忌表,将该参数作为下一步搜索的起始点;否则,判断下一个解;如果没有得到大于历史最优记录的或者不在禁忌表中的解,将最佳参数作为下一步搜索的起始点;(1.I), judge whether the parameter is in the taboo list, if not, add it to the taboo list, and use this parameter as the starting point for the next search; otherwise, judge the next solution; if no record greater than the historical optimal record or For solutions that are not in the tabu list, the best parameters are used as the starting point for the next search; (1.J),转步骤(1.B)。(1.J), go to step (1.B). 3.根据权利要求1所述的基于禁忌搜索算法优化支持向量机的方法,其特征在于,所述的步骤(2)进一步使用禁忌搜索算法优化过的支持向量机模型对待寻优函数和分类样本进行处理,具体分为以下几个部分来实现:3. The method for optimizing a support vector machine based on a tabu search algorithm according to claim 1, wherein the step (2) further uses the support vector machine model optimized by the tabu search algorithm to treat optimization functions and classification samples For processing, it is divided into the following parts to achieve: (2.A),基于经典函数的测试,验证所提出算法的全局寻优能力;(2.A), based on the test of classical functions, verify the global optimization ability of the proposed algorithm; Shaffer's F6函数是J.D.Shaffer等提出的,表达式为Shaffer's F6 function is proposed by J.D.Shaffer et al., the expression is ff (( xx ,, ythe y )) == 0.50.5 -- sinsin 22 xx 22 ++ ythe y 22 -- 0.50.5 [[ 11 ++ 0.0010.001 (( xx 22 ++ ythe y 22 )) ]] 22 其中,自变量的取值范围为,[-100,100];利用Shaffer's F6函数对提出的禁忌搜索算法进行测试,设定计算步数为100步,参数c∈(-10,10),,长c_step=0.01,γ∈(-10,10),步长甲吕乱鞋帜=测试N次;Among them, the value range of the independent variable is, [-100,100]; use Shaffer's F6 function to test the proposed tabu search algorithm, set the number of calculation steps to 100 steps, the parameter c∈(-10,10), and the length c_step =0.01, γ∈(-10,10), step size = test N times; (2.B),基于数据集的测试:在支持向量机的经典数据集上做实验,并与同类型方法做性能比较。(2.B), data set-based test: Experiment on the classic data set of support vector machine, and compare the performance with the same type of method. 4.根据权利要求3所述的基于禁忌搜索算法优化支持向量机的方法,其特征在于,所述的步骤(2)为Shaffer's F6函数在[-10,10]区间内,它有无限个局部极大点,只有一个(0,0)为全局最大,最大值为1,在此函数最大值周围有一个圈脊,它们的取值均为0.990283。4. the method for optimizing support vector machine based on tabu search algorithm according to claim 3, is characterized in that, described step (2) is that Shaffer's F6 function is in [-10,10] interval, and it has infinite local For the maximum point, only one (0,0) is the global maximum, and the maximum value is 1. There is a circle ridge around the maximum value of this function, and their values are all 0.990283. 5.根据权利要求3所述的基于禁忌搜索算法优化支持向量机的方法,其特征在于,所述的步骤(2)中测试N次,可以取N=10次。5. The method for optimizing a support vector machine based on a tabu search algorithm according to claim 3, characterized in that, in the step (2), the test is performed N times, and N=10 times can be taken.
CN 201210575338 2012-12-24 2012-12-24 Method for optimizing support vector machine based on tabu search algorithm Pending CN103164709A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN 201210575338 CN103164709A (en) 2012-12-24 2012-12-24 Method for optimizing support vector machine based on tabu search algorithm

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN 201210575338 CN103164709A (en) 2012-12-24 2012-12-24 Method for optimizing support vector machine based on tabu search algorithm

Publications (1)

Publication Number Publication Date
CN103164709A true CN103164709A (en) 2013-06-19

Family

ID=48587779

Family Applications (1)

Application Number Title Priority Date Filing Date
CN 201210575338 Pending CN103164709A (en) 2012-12-24 2012-12-24 Method for optimizing support vector machine based on tabu search algorithm

Country Status (1)

Country Link
CN (1) CN103164709A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106604229A (en) * 2016-12-27 2017-04-26 东南大学 Indoor positioning method based on manifold learning and improved support vector machine
CN108073761A (en) * 2016-11-14 2018-05-25 波音公司 System and method for optimizing battery pack design
CN109857937A (en) * 2019-01-25 2019-06-07 同济大学 One kind being based on Sine mapping and segmented continuous TABU search data processing method parallel
CN110008546A (en) * 2019-03-22 2019-07-12 西南交通大学 Circular aisle facility layout method
CN110555989A (en) * 2019-08-16 2019-12-10 华南理工大学 Xgboost algorithm-based traffic prediction method
CN110728075A (en) * 2019-10-25 2020-01-24 山西应用科技学院 Method for optimizing cryogenic process parameters by MOA algorithm
CN111866877A (en) * 2020-06-11 2020-10-30 南京邮电大学 A memory-based 5G physical layer security authentication method
CN112285761A (en) * 2020-11-24 2021-01-29 南昌华亮光电有限责任公司 System and method for segmenting crystal position spectrum data for plant PET
CN110221170B (en) * 2019-06-05 2021-07-27 贵州电网有限责任公司 Low-current grounding line selection method based on tabu search optimization RBF network

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108073761A (en) * 2016-11-14 2018-05-25 波音公司 System and method for optimizing battery pack design
CN106604229A (en) * 2016-12-27 2017-04-26 东南大学 Indoor positioning method based on manifold learning and improved support vector machine
CN106604229B (en) * 2016-12-27 2020-02-18 东南大学 An indoor localization method based on manifold learning and improved support vector machine
CN109857937A (en) * 2019-01-25 2019-06-07 同济大学 One kind being based on Sine mapping and segmented continuous TABU search data processing method parallel
CN110008546A (en) * 2019-03-22 2019-07-12 西南交通大学 Circular aisle facility layout method
CN110221170B (en) * 2019-06-05 2021-07-27 贵州电网有限责任公司 Low-current grounding line selection method based on tabu search optimization RBF network
CN110555989A (en) * 2019-08-16 2019-12-10 华南理工大学 Xgboost algorithm-based traffic prediction method
CN110728075A (en) * 2019-10-25 2020-01-24 山西应用科技学院 Method for optimizing cryogenic process parameters by MOA algorithm
CN111866877A (en) * 2020-06-11 2020-10-30 南京邮电大学 A memory-based 5G physical layer security authentication method
CN112285761A (en) * 2020-11-24 2021-01-29 南昌华亮光电有限责任公司 System and method for segmenting crystal position spectrum data for plant PET

Similar Documents

Publication Publication Date Title
CN103164709A (en) Method for optimizing support vector machine based on tabu search algorithm
CN108038271B (en) A kind of wear prediction method and state identification method of milling cutter
Grahn et al. CudaRF: a CUDA-based implementation of random forests
CN105844300A (en) Optimized classification method and optimized classification device based on random forest algorithm
CN111062425B (en) Unbalanced data set processing method based on C-K-SMOTE algorithm
CN111209192B (en) A Method for Automatic Test Case Generation Based on Double Chaotic Whale Optimization Algorithm
Goel et al. Evaluation of sampling methods for learning from imbalanced data
CN110334026A (en) Combination Test Case Generation Method Based on CS-SPSO Algorithm
CN103714153A (en) Density clustering method based on limited area data sampling
CN113419947B (en) Regression test case priority ordering method based on improved ant colony algorithm
CN106445821A (en) Method for automatically generating test case based on genetic algorithm
CN105095595A (en) Particle swarm optimization algorithm based on clustering degree of swarm
CN102819652A (en) Mechanical parameter optimization design method based on adaptive reverse differential evolution
CN117763428A (en) Lithium battery charging and discharging process performance test method, electronic equipment and storage medium
CN103455612A (en) Method for detecting non-overlapping network communities and overlapping network communities based on two-stage strategy
CN112015631A (en) An Imbalanced Data Generation Method for Software Defect Prediction
CN106951728B (en) Tumor key gene identification method based on particle swarm optimization and scoring criterion
CN105590039B (en) A kind of protein complex recognizing method based on BSO optimizations
CN103235862B (en) Select disassembly sequence planning method and device
CN114860297B (en) SMOTE (short message analysis) improvement-based Bayes-LightGBM software defect prediction method
CN112836735A (en) An Optimized Random Forest Approach to Handling Imbalanced Datasets
CN111858934B (en) Article heat prediction method and device
Newaz et al. ibrf: Improved balanced random forest classifier
CN103544328A (en) Parallel k mean value clustering method based on Hadoop
CN110399917A (en) An image classification method based on hyperparameter optimized CNN

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C02 Deemed withdrawal of patent application after publication (patent law 2001)
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20130619