JP2018156619A - Global search device and program for consecutive optimization problem - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a global search device and a program for a consecutive optimization problem capable of improving both of accuracy of a solution and a processing speed.SOLUTION: A global search device for a consecutive optimization problem comprises: a setting unit: and an optimum solution search unit. The setting unit creates a partial evaluation function based on a part of requests and constraints from multiple requests and constraints, and sets one or more individuals along the partial evaluation function. The optimum solution search unit applies, while searching an optimum solution of the set partial evaluation function or after searching the partial optimum solution, attraction force between individuals having the searched partial solution to search an optimum solution of a total evaluation function.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a global search apparatus and program for continuous optimization problems.

  For example, a method for obtaining an optimum value of a variable, that is, a parameter in a multidimensional search space has been proposed (see, for example, Patent Document 1). According to the technique described in Patent Document 1, a plurality of individuals whose elements are the parameters for which the optimum value is calculated are generated in the search space, the evaluation values of these individuals are calculated, and the individuals with bad evaluation values are selected from the individuals. Is selected. Then, the selected individual is brought close to an individual having a good evaluation value at a predetermined ratio (α). In addition, an individual with a bad evaluation value is moved to an arbitrary region within a certain Euclidean distance from the individual with the best evaluation value. Then, select an individual with a better evaluation value, update the individual with the best evaluation value as needed, converge the evaluation values of multiple individuals, and select the individual with the best evaluation value when it is determined that the termination determination condition is satisfied. The included parameters are output as the optimum parameter values.

JP 2007-233676 A

For example, when an evaluation function having a large number of extreme values is applied, when the optimum value of a variable is searched using the technique described in Patent Document 1, a predetermined ratio α is used to search for a large number of extreme values comprehensively. The search processing in the convergence process must be set finely by setting it small. Then, it takes a long time to complete the convergence, and it becomes difficult to achieve both solution accuracy and processing speed.
It is an object of the present invention to provide a global search apparatus and program for continuous optimization problems that can improve both the accuracy and processing speed of solutions.

  The invention described in claim 1 is a global search device for a continuous optimization problem for searching for an optimal solution of an overall evaluation function generated by a plurality of requirements and constraints. According to the first aspect of the present invention, the operation is as follows. The setting unit divides into a partial evaluation function generated by some requests and constraints from among a large number of requests and constraints, and sets one or a plurality of individuals along the partial evaluation function. Then, the optimal solution search unit searches for the optimal solution of the set partial evaluation function or, after searching for the partial optimal solution, draws an attractive force between individuals having the partial solution to be searched. Search for the optimal solution of the overall evaluation function.

  According to the first aspect of the present invention, the entire solution based on all the requests and constraints is searched while searching for the optimal solution of the partial evaluation function based on a small number of requests and constraints without handling all the requests and constraints at once. The optimum solution of the evaluation function can be searched, and the search can be performed with high speed and high accuracy.

Electrical configuration diagram of global search device for continuous optimization problem showing first embodiment Functional block diagram of a global search device for continuous optimization problems Flow chart schematically showing the flow of processing (part 1) Image of search process Acquired data and model examples Image of partial evaluation function Setting example when one individual is set for each grouped partial evaluation function Diagram showing the position of the individual when converged to the minimum value of the partial evaluation function Explanatory drawing explaining attractive interaction Flow chart (2) schematically showing the flow of processing Diagram showing a detailed image of the auxiliary function method The figure which shows initial stage distribution about 3rd Embodiment (the 1) Figure showing the initial distribution (2) Figure showing the initial distribution (part 3) Figure showing the initial distribution (Part 4) Diagram showing initial distribution (Part 5) Explanatory drawing which shows the update image in the update initial stage about 4th Embodiment Explanatory drawing showing the update image after the variable has transitioned to the vicinity of the minimum value Deep learning network configuration example Validation condition parameters Lossfunction verification results when applying the fourth embodiment Lossfunction verification results when a comparative example is applied Accuracy verification results when applying the fourth embodiment Accuracy verification results when a comparative example is applied

  Hereinafter, several embodiments of a global search apparatus and program for a continuous optimization problem of the present invention will be described with reference to the drawings. In the following embodiments, parts having the same function or similar functions are denoted by the same reference numerals in each embodiment, and a configuration having the same or similar functions, its operation, explanation of cooperative operation, etc. are necessary. Omitted according to.

(First embodiment)
FIG. 1A to FIG. 10 are explanatory diagrams of the first embodiment. The optimization device 1 is a continuous optimization problem using a microcomputer (hereinafter referred to as a microcomputer) 5 configured by connecting a CPU 2, a memory 3 such as a ROM and a RAM, and an input / output interface 4 or a general-purpose computer. It is configured as a global search device. Hereinafter, description will be made assuming that the microcomputer 5 executes the optimization process. The microcomputer 5 executes an optimization program stored in the memory 3 and executes various procedures to execute an optimization process. The memory 3 is used as a non-transitional tangible recording medium.

Here, the optimization process assumes a search space S composed of Euclidean space having one or more M (≧ 1) dimensions, and an overall evaluation generated by a plurality of requests and constraints in the search space S. A process of obtaining a variable x _i that satisfies the minimum value of the function Hopt or the overall evaluation function Hopt as an optimal value, that is, an optimal solution is shown.
In the following, a form for obtaining the minimum value of the overall evaluation function Hopt as the optimum value is shown, but it may be applied to processing for obtaining the maximum value as the optimum value.

  The overall evaluation function Hopt is a function that is generated by a plurality of requests or constraints and is derived based on one or more M variables (parameters). For example, an arbitrary polynomial, rational function, or irrational function , An exponential function, a logarithmic function, and a combination of addition, subtraction, multiplication and division. As shown in FIG. 1B, the optimization apparatus 1 includes various functions such as a setting unit 6 and an optimal solution search unit 7 as functions realized by the microcomputer 5.

FIG. 2 is a flowchart showing a schematic flow of the optimization process, and FIG. 3 is an image diagram of the search process. The microcomputer 5 divides the overall requests and constraints into a plurality of groups in S1 of FIG. 2 for the multiple requests and constraints that form the overall evaluation function H _opt . (S10 in FIG. 3: function of the setting unit 6).

Then, the microcomputer 5 sets the partial evaluation function H _i (x _i ) according to the classified requirements and constraints in S2 of FIG. 2 (S11 of FIG. 3: function of the setting unit 6). FIG. 5 schematically shows an image of the partial evaluation function H _i (x _i ). The partial evaluation function H _i (x _i ) is set according to the requirements and constraints selected in S10 described above, and the effective evaluation function H _eff is expressed as in the following equation (1).

The effective evaluation function H _eff ({x _i }) of the equation ( ₁ ) indicates an integrated value of the sum of the individual partial evaluation functions H _i (x _i ) and the function f (x _i −x _{i + 1} ). And the function f (x _i −x _{i + 1} ) is a function using attractive coefficients λ _i and ρ _i / 2 as shown in, for example, the equation (2). Here, λ _i represents a primary attractive coefficient, and ρ _i / 2 represents a secondary attractive coefficient. Here, ρ _i is a predetermined positive constant.

The microcomputer 5 sets one or a plurality of individuals K along the partial evaluation function H _i (x _i ) in S3 of FIG. 2 (function of the setting unit 6). FIG. 6 shows a setting example when one individual K is set for each of the four divided partial evaluation functions H _i (x _i ). That is, a setting example of four individuals K1 to K4 is shown.

Next, in S4 of FIG. 2, the microcomputer 5 minimizes the effective evaluation function H _eff while controlling the attractive force between the plurality of individuals K so that the plurality of individuals K converge (see FIG. 3). S12, S13: Function of optimum solution search unit 7). By setting the attractive force weakly in the initial stage of the search, the partial evaluation function H _i (x _i ) is minimized in the initial stage of the search, and the individual K is converged by the attractive force in the second half of the search. Further, if the attractive force is set strongly at the initial stage of the search, the minimization of the partial evaluation function H _i (x _i ) and the convergence of the individual K are executed in parallel. FIG. 7 shows the positions of the individuals K1 to K4 when the attractive force at the initial stage of the search is set weak and converged to the minimum value of the partial evaluation function H _i (x _i ). As described above, the microcomputer 5 searches for a partial optimum solution in S4 of FIG. 2 and applies an attractive force to the individual K having the partial solution to be searched for, thereby obtaining an effective evaluation function H _{eff. Is} searched for the optimal solution of the overall evaluation function H _opt (S13 in FIG. 3: function of the optimal solution search unit 7).

For example, as shown in FIG. 4, in machine learning, a model (for example, y = ax + b) along a large amount of acquired data is converted into an optimum parameter a for the acquired data using, for example, a fitting model based on the least square method. , B may be determined. Each of these acquired data becomes one requirement and restriction for the parameters a and b. In this embodiment, divided into a number of entire data into a plurality of groups, a fitting model for each group as a partial evaluation function H _{i (x),} the individual K while searching the optimal solution of the partial evaluation function H _{i (x)} The parameters a and b most suitable as a model for expressing all data are derived by applying an attractive force between the two to converge to one.

In order to realize this content, an effective evaluation function H _eff is set as in the expressions (1) and (2) so that the conditions of the following expressions (3-1) and (3-2) are satisfied. The primary attractive coefficient λ is updated. Here, the secondary attractive force coefficient ρ _i is set to a predetermined value.

In this equation (3), g (x) is a predetermined function and satisfies dg (x) / dx ≧ 0 and x · g (x) ≧ 0, and therefore g (x) is a simple increasing function. It is desirable that the variable x and the function g (x) have the same sign. For example, it is desirable to set so as to satisfy the following expression (4). Here, the order n is an odd number satisfying 2m−1 (where m is a natural number).

In the update process that satisfies the expressions (1) to (3-1) and (3-2) in S4 of FIG. 2, while deriving the optimal solution of the partial evaluation function H _i (x _i ), By repeating the _minimization of the effective evaluation function H _eff while increasing the attractive force, the optimum solution of the overall evaluation function H _opt is searched. If the attractive force at the initial stage of the search is weakened, the optimal solution search for the partial evaluation function H _i (x _i ) and the optimal solution search process for the overall evaluation function H _opt by convergence are separated. FIGS. 7 and 8 show an image of searching for an optimal solution of the overall evaluation function H _opt by convergence of the individual K after searching for an optimal solution of the partial evaluation function H _i (x _i ).

Next, an example of the optimization processing procedure will be described using more specific mathematical expression expansion. First microcomputer 5 sets the individual K along portions evaluation function H _i in S21 in FIG. Hereinafter, unless otherwise noted, some or all of the plurality of individuals K1 and K2 will be abbreviated as individuals K, and special individuals K will be described with a suffix after the symbol K. .

ここで変数ｘ_ｉは下記の（６−１）式のように設定され、変数λ_ｉは（６−２）式のように設定される。

Next, the microcomputer 5 changes and updates the variable x _{i of the} individual K so as to minimize the effective evaluation function H _eff in S22 of FIG. When the effective evaluation function H _eff is decomposed into N partial evaluation functions H _i having M variables x _i and searched, the mathematical expression is set as shown in the following equation (5).

Here, the variable x _i is set as in the following equation (6-1), and the variable λ _i is set as in the equation (6-2).

(5) the first term of the right side of expression evaluation value H _{i (x i)} evaluation sum value obtained by adding the partial evaluation function H _i of a plurality of individual K, i.e., partial evaluation of the plurality of functions H _i of the individual K An addition value obtained by adding an evaluation value H _i (x _i ) into which a variable x _i is substituted is shown. The second term and the third term on the right side of the equation (1) are attractive interaction terms between a plurality of individuals K. Λ _i ^ T is the primary attractive coefficient, and ρ _i / 2 is the secondary attractive coefficient.

Here, ρ _i is a predetermined positive constant. Attractive interaction terms of these second and third terms, the difference between the variable x _i between the plurality of individual K increases as large, small differences in the variable x _i between the plurality of individual K conversely Sometimes it changes to be smaller.

Next, in order to update the variable x _i defined by the equation (6-1) in the equation (5), the partial evaluation function H _i (x _i ) is converted to an extreme value using the auxiliary function method. Auxiliary function method, (1) is one method used when minimizing the partial evaluation function H _i of the first term of equation quadratic function that approximates the partial evaluation function H _i to the partial evaluation function H _i A method of converting to ff and converting to an extremum based on the replaced quadratic function ff is shown.

Hereinafter, the auxiliary function method will be described in detail. A detailed image of the auxiliary function method is shown in FIG. As shown in FIG. 10, the current solution candidate x _i ^ * is assigned to the variable x of the partial evaluation function H _i (x), and the evaluation value H _i (x ^ *) is passed. Evaluation values for all variables x _i in the search space S in the search space S that are equal to the partial differential value ∂H _i (x ^ *) / ∂x of the partial evaluation function H _i (x) and include the solution candidates x _i ^ * A quadratic function ff using a Lipschitz constant L that satisfies a condition for obtaining a value larger than H _i (x _i ) is introduced and replaced. This quadratic function ff can be expressed as a right side of the equation (7) when it is mathematically expressed.

In this equation (7), L _{i, p} represents a Lipschitz constant, and x ^ * represents the current solution candidate for x. At this time, the minimum value of the quadratic function ff is repeatedly updated as a solution candidate for the next value, and the minimum value is repeatedly updated a number of times equal to or greater than a predetermined number of updates.

この方程式を一般化すると、ｐ番目の変数ｘ_ｉの更新式は（１０）式の方程式のように展開できる。

そしてマイコン５は、図９のＳ１２において（１１）式に示すｐ番目の変数行列~ｘ_ｐを算出する。

The right side of the equation (8) is differentiated by the p-th variable x _{i, p} of the i-th individual K, and the minimum point of the variable x _{i, p} satisfying the condition that this differential value is zero is calculated. In this equation (8), the value before update is x _{i, p} ^ *, and the value after update is x _{i, p} . Then, it can be developed as the equation (9).

If this equation is generalized, the update equation for the p-th variable x _i can be expanded as shown in equation (10).

Then, the microcomputer 5 calculates the p-th variable matrix ~ x _p shown in the equation (11) in S12 of FIG.

Here, the updated p-th variable matrix ~ x _p is the expression (12-1), the Lipschitz constant matrix L _p is the expression (12-2), the secondary attractive coefficient matrix ρ is the expression (12-3), the coefficient The matrix G is expressed as (12-4).

そしてマイコン５は、図９のＳ２３において（１４）式のように一次引力係数行列~λ_ｉを更新する。

Further, the p-th variable matrix x _p ^ * before the update is expressed by equation (13-1), and the partial differential matrix _{ｐ p} H _i (x _i ^ *) of the partial evaluation function H _i (x _i ^ *) is expressed by (13 -2), the primary attractive coefficient matrix λ _p ^ * is the expression (13-3), and the coefficient matrix W is the expression (13-4).

Then, the microcomputer 5 updates the primary attractive coefficient matrix ~ λ _i as shown in equation (14) in S23 of FIG.

In this equation (14), lambda _{i ^} * denotes the update before the primary attraction coefficient matrix, variable ~ x _i, indicating the variables calculated in S22. Next, the microcomputer 5 determines whether or not the end condition is satisfied in S24 of FIG. This termination condition is based on the condition that, for example, it is determined whether or not the effective evaluation value of the effective evaluation function H _eff is equal to or less than a predetermined threshold, or whether or not the number of repetition processing exceeds a predetermined upper limit number. Yes.

Threshold of the effective evaluation value of the effective evaluation function H _eff, it is desirable to pre-set to a value in which a plurality of individual K is able to substantially converge on the position of the extreme value of the effective evaluation function H _eff is assumed. For this reason, if a plurality of individuals K has not converged to one point, the process returns to S22 and the process is repeated. As shown in FIG. 7, an attractive interaction based on the paths M <b> 1 to M <b> 4 between the valleys can be generated as shown in FIG. 8, while the plurality of individuals K move to the minimum value of each valley. .

<Conceptual summary of this embodiment>
As described above, according to the present embodiment as described, a plurality of requests, the total evaluation function H _opt generated by restriction of some overall evaluation function H _opt request, partial evaluation were prepared from the constraint function H _i ( divided into x _i), set along one or more individuals K to the partial evaluation function H _{i (x i),} while searching the partial optimal solution set partial evaluation function H _{i (x i)} Alternatively, after searching for a partial optimal solution, an attractive force is applied to the individual K having the partial solution to be searched to search for an optimal solution of the overall evaluation function H _opt . As a result, the optimum solution of the entire complex overall evaluation function H _opt can be searched based on the simplified optimization process of the partial partial evaluation function H _i (x _i ). The optimal solution of _opt can be acquired at high speed.

When the effective evaluation function H _eff is expressed by the equation (1), the term of the attractive interaction is expressed by the equation (2) and the equation (3-1) and the equation (3-2) are satisfied between the individual K so that Since the attractive force is applied, a plurality of individuals K can be converged to one point. At this time, it is desirable to search for g (x) in equation (3) as equation (4).

(Second Embodiment)
In the second embodiment, a mode in which f (x _i -x _{i + 1} ) serving as an attractive interaction term is changed as in equation (15) will be described.

F (x _i −x _{i + 1} ) in the equation (15) is obtained by setting λ _i = 0 in the equation (2). By gradually increasing the secondary attractive coefficient ρ _i / 2 from a predetermined value, the individual K Can be converged to one point.

Therefore, in the present embodiment, by applying the process of gradually increasing the secondary attractive coefficient ρ _i / 2 in the equations (2) and (15) from a predetermined value, each partial evaluation function H _i (x _i ) is set. The individual K is gradually converged to search for the optimum solution of the overall evaluation function H _opt .

According to the present embodiment, when the effective evaluation function H _eff is defined as the equation (1), the term of the attractive interaction is the equation (15), and the secondary attractive force coefficient ρ _i in the equation (15) is By increasing the value from the initial value, an attractive force is applied between the individuals K to converge to the optimal value and the optimal solution is derived. Even in such a case, the same effects can be obtained.

(Third embodiment)
11 to 15 show additional explanatory diagrams of the third embodiment. In the third embodiment, illustrating a method of setting the initial distribution of the variables x _i of the individual K. As shown in the previous embodiment, each individual K has an M-dimensional variable x _i , but an effective evaluation function depends on the form of the initial distribution of the M-dimensional variable x _i. The number of passes through the valley of H _eff , that is, the number of passes of extreme values, and the convergence method also change.

Since the effective evaluation value of the effective evaluation function H _eff cannot be grasped in advance as to what value is the extreme value or the minimum value, the microcomputer 5 initially sets the individual K so as to fill the search space S. It is desirable to set. Increasing the number of individuals K increases accuracy but increases processing time. For this reason, it is desirable to perform processing using a limited number of individuals K, and the initial distribution of the variable x _i of the limited number of individuals K occupies an important element in the optimum value search processing. For this purpose, for example, from FIG. 11, as shown in FIG. 15, it is desirable to set the initial distribution of the variables x _i of the individual K.

Figures 11-15, each individual K indicates an example of an initial distribution when and a M = 2 dimensional variables x _i. When an M = 2-dimensional search space S is assumed, for example, as shown in FIG. 11, variables x _i of a plurality of individuals K may be set in the search space S at random. In this case, a wide range of extreme values can be searched in the process in which the individual K converges to one point.

Individuals Ka of 12 Among these, as shown in Kb, may be an upper limit value or lower limit value of at least one or more individuals Ka, the Kb of the variable x _i search space S. In addition, the variable x _i of other individual K may be set to random. By setting in this way, a wider range of extreme values can be searched. In particular, it is more desirable to set the variable x _{i of the} individual Kam so that all the variables x _i in the search space S become the upper limit value or the lower limit value.

In order to be able to search a wide range of extreme values in a certain convergence process, it is desirable to distribute the individuals K over a wide range so as to cover all of the search space S as much as possible, but before starting the search for the variable x _i , for example, An estimated solution may be given inside the search space S.

For example, consider that a variable x _i changes continuously in time. For example, when the microcomputer 5 derives the solution of the variable x _i (t) at every predetermined time, in order to obtain the solution of the variable x _i (t + 1) at the next timing, as shown in FIG. By giving the solution of the current variable x _i (t) as the estimated solution xiz, the derivation processing of the solution of the variable x _i (t + 1) at the next timing may be performed quickly and accurately.

In such a case, as shown in FIG. 13, a limited search range Sa limited in the search space S is provided as an initial distribution, and the initial distribution is set so as to include the estimated solution xiz as the limited search range Sa. desirable. Particularly in this case, for example, a predetermined range from the estimated solution xiz (for example, xi _{, 1−} α1 <variable xi _{, 1} <xi _{, 1} + β1, xi _{, 2−} α2 <variable xi _{, 2} <xi _{, 2} + β2). However, it is desirable to narrow the limited search range Sa so as to satisfy α1, α2, β1, β2> 0). As a result, the probability of reaching the optimal solution of the variable x _i can be increased with a smaller number of evaluations.

Furthermore, as shown in FIG. 14, the initial distribution may be set such that the closer to the space including the estimated solution xiz, the denser the individuals K are distributed, and the further away from the estimated solution, the density of the individuals K is decreased. As a result, the probability of reaching the optimal solution of the variable x _i can be increased with a smaller number of individuals K.

Further, as shown in FIG. 15, the individuals K are densely distributed within a predetermined range Sb close to the space including the estimated solution xiz as an initial distribution, and other individuals K are randomly distributed outside the predetermined range Sb. among individual K, at least one individual Ka, Kb, may be set to the upper limit value or lower limit value of the variable x _i of the variable x _i a search range S of Kam. Thus, while intensively searching the vicinity of the estimated solution xiz variables x _i can explore a wide range, in a case where the estimated solution xiz or solutions in its periphery did not exist, the optimal variables x _i A decrease in the probability of reaching the solution can be prevented.

(Fourth embodiment)
16 to 23 show additional explanatory views of the fourth embodiment. 4th Embodiment demonstrates another form about the method of minimization.

In the fourth embodiment, with respect to the effective evaluation function H _eff of the above-described equation (1), the update width Δx _c (corresponding to the second update width) of the partial evaluation function H _i and the attractive interaction term f (x _i −x The update width Δx _q (corresponding to the first update width) of _{i + 1} ) is calculated separately, and a total update width Δx obtained by adding these update widths Δx _c and Δx _q is derived.

At this time, first, the microcomputer 5 defines the gradient g _c (t) of the partial evaluation function H _{i at} a certain time t as the equation (16), and the exponential moving average value m of the gradient g _c of the partial evaluation function H _{i c} is defined and derived as in the equation (17-1), and the exponential moving average value v _c of the square of the gradient g _c of the partial evaluation function H _i is defined and derived as in the equation (17-2). To do. In these equations, β _1c and β _2c are constants, and are set to appropriate values as gradient calculation processing in deep learning.

このようにして更新幅Δｘ_ｃを導出できる。 Then, the microcomputer 5 derives the update width Δx _c as shown in equations (18-1) and (18-2). Note that ε _c in the equation (18-1) is a constant for avoiding division by zero, and is set much smaller than other constants.

In this way, the update width Δx _c can be derived.

Further, the microcomputer 5 similarly performs such processing for the update width Δx _q of the attractive interaction term f (x _i -x _{i + 1} ). That is, the microcomputer 5 defines the gradient g _q (t) of the attractive interaction term f (x _i −x _{i + 1} ) at a certain time t as defined by the equation (19), and calculates the attractive interaction term f (x _i -X _{i + 1} ) The exponential moving average value m _q of the gradient g _q is derived as defined by the equation (20-1), and the square of the gradient g _q of the attractive interaction term f (x _i -x _{i + 1} ) define derived as an exponential moving average value _{v q} (20-2) expression. In these equations, β _1q and β _2q are constants and are set to appropriate values for the gradient calculation processing in deep learning.

このようにして更新幅Δｘ_ｑを導出できる。そしてマイコン５は更新幅Δｘ_ｃ、Δｘ_ｑを加算することで合計更新幅Δｘを（２２）式のように導出する。

Then, the microcomputer 5 derives the update width Δx _q as shown in equations (21-1) and (21-2). Note that ε _q in the equation (21-1) is a constant for avoiding division by zero, and is set to be significantly smaller than other constants.

In this way, the update width Δx _q can be derived. The microcomputer 5 adds the update widths Δx _c and Δx _q to derive the total update width Δx as shown in equation (22).

Thus can derive the total update width Δx in, it becomes possible to adaptively change the update width Δx that the optimization variables x _i used as an update width Δx of the variable x _i of the effective evaluation function H _eff. Are shown in FIGS. 16 and 17 the update image of the optimization variables x _i.

FIG. 16 shows an update image at the initial update stage. For example, if the variable of the attractive interaction term f (x _i −x _{i + 1} ) is separated from the optimization variable that satisfies the minimum value of the evaluation function, the variable is reduced toward the optimization variable that minimizes the function. As shown by 16, it is gradually updated. At this time, since the gradient g _q (t) changes gently, the gradient g _q does not change much even if the timings of the time t and the time t + 1 are compared as shown in the equation (23-1). As shown in this order (23-2) below, becomes substantially the same value as compared to the square root of the gradient g exponential moving average of _q ~ m _q and slope g exponential moving average of the square of the _q ~ v _q The update width Δx _q is substantially constant at −η as shown in the equation (23-3).

On the other hand, FIG. 17 shows an update image after the variable transitions to the vicinity of the minimum value. When the variable transitions to the vicinity of the local minimum value, the gradient g _q (t) moves so as to oscillate with the local minimum value interposed therebetween. Therefore, as shown in the equation (24-1), the timing between the time t and the time t + 1 is changed. compared the gradient g _q is the absolute value of approximately positive and negative in the same changes as reversed. Then (24-2) as shown in equation exponential moving average of the gradient g _q ~ m _q is generally closer to 0, exponential moving average of the square of the gradient exponential moving average ~ m _q gradient g _q ~ Even when compared with the square root of v _q, the square exponential moving average value to the square root of v _q are significantly larger. Therefore, the update width Δx _q is a value significantly smaller than −η as shown in the equation (24-3).

The microcomputer 5 in this way, the gradient _{g q} gravitational interaction term _{f (x i -x i + 1} ) exponential moving average of the gradient _{g q} of ~ _{m q} and attractive interaction term _{f (x i -x i + 1} ) The update width Δx _q is increased as the square exponential moving average value ~ v _q becomes closer, and the exponent moving average value ~ m _q becomes smaller than the square exponential moving average value ~ v _{q of the} gradient g _q The update width Δx _q is derived to be small. That is, it is possible to automatically change to a smaller width from a large width update width [Delta] x _q, it is possible to efficiently reach a local minimum as small as possible number of updates.

Here, the attractive interaction term f (x _i -x _{i + 1} ) has been described. However, since it can be similarly applied to the partial evaluation function H _i , the detailed description thereof is omitted. Conceptually summarized processed partial evaluation function H _i, the microcomputer 5, the square index gradient g _c partial evaluation function H exponential moving average value of the gradient g _c of _i ~ m _c and partial evaluation function H _i moved while increasing the update width [Delta] x _c according to the average value ~ v _c is closer, reducing the update width [Delta] x _c as exponential moving average ~ m _c is smaller than the exponential moving average ~ v _c of the square of the gradient Will be derived. Similarly in this case, automatically can be changed to a smaller width from a large width update width [Delta] x _c, it is possible to efficiently reach a minimum value with a small number of updates.

<Verification conditions for specific examples>
The inventor has verified the method described above using a specific example. Below, this verification condition is demonstrated. The inventors have verified the problem by identifying single-digit handwritten numerals “0” to “9” using deep learning.

  As the verification conditions, as shown in FIG. 18, 64 inputs corresponding to 8 × 8 pixels are prepared, 10 outputs “0” to “9” are prepared, and 5 intermediate layers are prepared. A neural network is used. The number of nodes in each intermediate layer is 30 nodes in the first layer, 20 nodes each in the second to fifth layers are prepared, and the network combines all the nodes in the input layer, intermediate layer, and output layer. The configured form was used.

In addition, 1000 pieces of learning data for determining the connection weight of each node are prepared and learning is attempted. Here, the number of divisions of the learning data for defining the partial evaluation function H _i is 10. That is, N in the equations (16) and (19) is set to 10, and the N partial evaluation functions _Hi are evaluated using 100 pieces of learning data (hereinafter referred to as “partial data” as necessary). did. Further, the equation (15) was applied to the attractive interaction term f (x _i -x _{i + 1} ), and the secondary attractive coefficient ρ _i of the equation (15) was verified as a fixed value 0.1. Moreover, each constant (eta), (beta) _1c , (beta) _2c , (beta) _1q , (beta) _2q , (epsilon) _c , (epsilon) _q used for the other (16)-(22) Formula was evaluated using the value shown in FIG.

Further, the inventor made a comparative example using a general evaluation function in which the attractive interaction term f (x _i -x _{i + 1} ) in the second term on the right-hand side of the equation ( ₁ ) was omitted. It is verified to identify handwritten numbers by learning. This comparative example is a method similar to an example in which a gradient method in deep learning generally called Adam (Adaptive moment estimation) is applied. Also in this comparative example, in order to perform comparison verification with the processing method of the present embodiment, 1000 pieces of the same learning data are prepared and the learning data is divided into N = 10, and each of the 100 pieces of learning data is used. evaluated. That is, evaluation is performed by performing the same processing as in the equations (16) to (18), and at this time, the values of the constants η, β _1c , β _2c , and ε _c similar to those described above are used.

<Verification results>
Hereinafter, the verification result will be described.
FIG. 20 shows the verification result of Lossfunction obtained using the attractive interaction term f (x _i −x _{i + 1} ) of this embodiment, and FIG. 21 shows the case where the attractive interaction term f (x _i −x _{i + 1} ) is not used. The verification result of Lossfunction of a comparative example is shown.

The Lossfunction characteristics of FIGS. 20 and 21 show the number of search steps on the horizontal axis, and show the learning error after determining the weight of each node by adapting the neural network shown in FIG. 18 to the learning data. . To become to seek Lossfunction corresponding to adaptability the search step number is small for each partial evaluation function H _i of N = 10 pieces of divided data, its the absolute value is also large variation for each partial data, search step number It has been confirmed that the action of the attractive interaction term f advances as the value increases, and can be identified more appropriately.

  According to this verification result, it can be seen that the loss function can be decreased as the number of search steps is increased by using any of the methods of the present embodiment and the comparative example, but in the comparative example, the minimum value of the loss function is 0. On the other hand, in this embodiment, the minimum value can be set to 0.021, and it can be understood that the minimum value of the loss function can be further reduced.

<Verification results for unknown test data>
Hereinafter, verification results for unknown test data will be described.
FIG. 22 shows the accuracy verification result of this embodiment, and FIG. 23 shows the accuracy verification result of the comparative example. Accuracy in FIGS. 22 and 23 indicates a correct answer rate using unknown test data different from the above-described learning data, and generalization indicating whether class labels and function values for new data can be accurately predicted. An index related to performance is shown, and the number of search steps is shown on the horizontal axis in the same manner as described above. In the comparative example, the correct answer rate, that is, the identification accuracy is 73.65%, whereas in this embodiment, the correct answer rate and the identification accuracy can be 91.71%. By adopting this embodiment, the comparative example It can be understood that it shows higher discrimination performance.

<Conceptual summary of this embodiment>
As described above, according to this embodiment, attractive interaction term _{f (x i -x i + 1} ) and partial evaluation function _{H i} of the gradient _g q, according to _{g c,} the variable _{x i} of each individual K update width [Delta] x _q, when deriving the [Delta] x _c, attractive interaction term _{f (x i -x i + 1} ) and partial evaluation function _{H i} of the gradient _g q, exponential moving average of _{g c ~ m q, ~ m} c Gravity interaction terms _{f (x i -x i + 1} ) and partial evaluation function _{H i} of the gradient _g q, exponential moving average of the square of _{g c ~ v q, ~ v} c each update width in accordance with the close respectively Δx _q and Δx _c are increased. Furthermore, exponential moving average ~ m _{q gradient,} ~ m _c is exponential moving average of the square of the gradient ~ v _q, update width [Delta] x q as smaller than ~ v _c, derived so as to reduce the [Delta] x _c ing. For this reason, the update widths Δx _q and Δx _c can be automatically changed from a large width to a small width, and the individual K can efficiently reach the minimum value with as few updates as possible.

<Modification of Fourth Embodiment>
If the process shown in the fourth embodiment is used, the influence of the attractive interaction term f (x _i -x _{i + 1} ) is adaptively changed according to the change in the gradient g _q described above. For this reason, even if the secondary attractive coefficient ρ _i in the equation (15) is set to a fixed value (for example, 0.1), the convergence is achieved without any problem. That is, for example, in the second embodiment, the secondary attractive force coefficient ρ _i is increased from the initial value to cause the attractive force to act on the individual K so as to converge to the optimum value. There is no need to do. Further, as described in the second embodiment, the secondary attraction coefficient ρ _i may be increased.

このような（２５）式に示される引力相互作用項ｆ（ｘ_ｉ−ｘ_ｉ＋１）を用いたとしても変数ｘ_ｉの更新処理が複雑な処理に変化することはなく極力迅速に最適解を探索できる。 Further, when the method of the fourth embodiment is adopted, when the variable x _i is updated, information according to the gradients g _q and g _c is used, so that the attractive interaction term f (x _i −x _{i + 1} ) is It is not limited to a quadratic function as in equation (15), but using an even-order function of the fourth or higher order as shown in the attractive interaction term f (x _i -x _{i + 1} ) in equation (25) below. Also good.

Even if the gravitational interaction term f (x _i -x _{i + 1} ) shown in the equation (25) is used, the update process of the variable x _i does not change to a complicated process, and the optimum solution is searched as quickly as possible. it can.

(Other embodiments)
The present invention is not limited to the above-described embodiment, and can be modified or expanded as follows.
It is desirable that the individual K has the same number of variables x _i among the partial evaluation functions H _i (x _i ). In the above embodiment has been described a form of solution output variables x _i of the individual K, evaluation value corresponding to the variable x _i, i.e., the variable x _i to be construed output to the effective evaluation function H _eff The assigned evaluation value may be output as a solution. In the third embodiment, the secondary attractive force coefficient ρ _i / 2 is gradually increased from the initial value = 0, but the initial value is not limited to zero.

  Further, the reference numerals in parentheses described in the claims indicate the correspondence with the specific means described in the embodiment described above as one aspect of the present invention, and the technical scope of the present invention is It is not limited. An aspect in which a part of the above-described embodiment is omitted as long as the problem can be solved can be regarded as the embodiment. In addition, any conceivable aspect can be regarded as an embodiment as long as it does not depart from the essence of the invention specified by the words described in the claims.

  Moreover, although this invention was described based on embodiment mentioned above, it understands that this invention is not limited to the said embodiment and structure. The present invention includes various modifications and modifications within an equivalent range. In addition, various combinations and forms, as well as other combinations and forms including one element, more or less, are within the scope and spirit of the present disclosure.

In the drawings, 1 is an optimization device (global search device for continuous optimization problems), 6 is a setting unit, 7 is an optimal solution search unit, 8 is a derivation unit, K is an individual, S is a search space, and x _i is a variable. , g / 2 is attraction coefficient, whole Hopt evaluation function, Heff is an effective evaluation value, Xiz estimated _solution, g q, _{g c} is the slope, ~ _{m q,} ~ _{m c} is exponential moving average value of the gradient, ~ v _q and ˜v _c represent exponential moving average values of the squares of gradients, and Δx _q and Δx _c represent update widths.

Claims (26)

The overall evaluation function generated by a plurality of requests and constraints is divided into a partial evaluation function ( _Hi (x)) generated by some requests and constraints, and one or a plurality of individuals are set along the partial evaluation function. A setting unit (6) to perform,
While searching for a partial optimal solution of the set partial evaluation function or after searching for a partial optimal solution, an attractive force is applied between individuals having the partial solution to be searched for. An optimal solution search unit (7) for searching for an optimal solution of the overall evaluation function;
A global search device for continuous optimization problems. The optimal solution search unit includes:
When an effective evaluation function (H _eff ) composed of the partial evaluation function and the attractive interaction term (f (x _i −x _{i + 1} )) is expressed as (1),
A first update width (Δx _q ) of the variable of the individual is derived according to the gradient of the attractive interaction term, and a second update width (Δx _c ) of the variable of the individual is determined according to the gradient of the partial evaluation function. Deriving and calculating and deriving a total update width (Δx) obtained by adding the second update width (Δx _c ) to the first update width (Δx _q ),
When deriving the first update width (Δx _q ) of the individual variable according to the gradient of the attractive interaction term, the moving average value of the gradient of the attractive interaction term and the square of the gradient of the attractive interaction term The first update width is increased as the moving average value of the gradient becomes closer, and the first update width is decreased as the moving average value of the gradient becomes smaller than the moving average value of the square of the gradient. The global search device for continuous optimization problems according to claim 1.

The gradient of the attractive interaction term is represented by Equation (19), the moving average value of the gradient of the attractive interaction term is represented by Equation (20-1), and the moving average value of the square of the gradient of the attractive interaction term is represented by (20). The global search device for a continuous optimization problem according to claim 2, wherein the first update width is derived as in formulas (21-1) and (21-2) when formula (-2) is used.

The optimal solution search unit includes:
When deriving the second update width, the second update width is increased as the moving average value of the gradient of the partial evaluation function and the moving average value of the square of the gradient of the partial evaluation function become closer, and 4. The global search device for a continuous optimization problem according to claim 2, wherein the second update width is derived so as to decrease as the moving average value of the gradient becomes smaller than the moving average value of the square of the gradient. The gradient of the partial evaluation function is set as equation (16), the moving average value of the gradient of the partial evaluation function is set as equation (17-1), and the moving average value of the square of the gradient of the partial evaluation function is set as (17-2). 5. The global search device for continuous optimization problems according to claim 4, wherein the second update width is derived as shown in equations (18-1) and (18-2) when the equations are used.

The optimal solution search unit defines an attractive interaction term as an equation (25) when the effective evaluation function is defined as an equation (1), and sets ρ _i in the equation (25) as a predetermined value. The global search device for continuous optimization problems according to any one of claims 2 to 5, wherein an attractive force is applied between individuals by being increased from an initial value so as to converge to an optimal value.

The optimal solution search unit has an attractive force when an effective evaluation function (H _eff ) composed of the partial evaluation function and an attractive interaction term (f (x _i −x _{i + 1} )) is expressed by equation (1). 2. The global search device for continuous optimization problems according to claim 1, wherein an attractive force is applied between individuals so that the interaction term is expressed by equation (2) and equation (3) is satisfied.

The global search device for a continuous optimization problem according to claim 7, wherein g (x) in the equation (3) is searched as an equation (4).

The optimum solution search unit defines the attractive interaction term as equation (15) when the effective evaluation function is defined as equation (1), and increases ρ _i in the equation (15) from the initial value. The global search device of the continuous optimization problem according to claim 1, wherein an attractive force acts between individuals to converge to an optimum value.

When the optimal solution search unit minimizes the effective evaluation function,
After obtaining a solution candidate for the partial evaluation function (H _i ), the differential value when the solution candidate is substituted is equal to the differential value of the partial evaluation function, and all of the search spaces including the solution candidate 8. An auxiliary function method is used in which a quadratic function that satisfies a condition that obtains a value larger than the evaluation value in the variable is replaced with an extrinsic function method that repeatedly updates the extreme value of the quadratic function as a solution candidate for the next value. The global search apparatus of the continuous optimization problem as described in any one of 1-9.   The global search device for a continuous optimization problem according to any one of claims 1 to 10, wherein the setting unit sets an initial distribution so that variables of the plurality of individuals are randomly distributed in the search space. If a variable estimation solution (xiz) is given in advance before the search for the variable is started,
The said setting part is the limited search range (Sa) limited to the inside of the said search space as an initial distribution, The variable of the said individual is set to the limited search range containing the said estimated solution. A global search apparatus for continuous optimization problems according to one item. The individual global search device of a continuous optimization problem claimed in any one of 12 with the same number of variables (x _i) between the partial cost function. In global search device (1),
The overall evaluation function generated by a plurality of requests and constraints is divided into partial evaluation functions (H _i (x _i )) generated by a part of the requests and constraints, and one or more individuals are arranged along the partial evaluation function. Steps to set,
A procedure for searching for a partial optimal solution by the set partial evaluation function;
While searching for the partial optimal solution, or after searching for the partial optimal solution, an attractive force is applied between the individuals having the partial solution to be searched for to determine the optimal solution of the overall evaluation function. Steps to explore,
A program that executes In the procedure for searching for the optimal solution of the overall evaluation function,
When an effective evaluation function (H _eff ) composed of the partial evaluation function and the attractive interaction term (f (x _i −x _{i + 1} )) is expressed as (1),
A first update width (Δx _q ) of the variable of the individual is derived according to the gradient of the attractive interaction term, and a second update width (Δx _c ) of the variable of the individual is determined according to the gradient of the partial evaluation function. And calculating and deriving a total update width (Δx) obtained by adding the second update width (Δx _c ) to the first update width (Δx _q ),
When deriving the first update width (Δx _q ) of the individual variable according to the gradient of the attractive interaction term, the moving average value of the gradient of the attractive interaction term and the square of the gradient of the attractive interaction term The first update width is increased as the moving average value of the gradient becomes closer, and the first update width is decreased as the moving average value of the gradient becomes smaller than the moving average value of the square of the gradient. The program according to claim 14.

In the procedure for searching for the optimal solution of the overall evaluation function,
The gradient of the attractive interaction term is represented by Equation (19), the moving average value of the gradient of the attractive interaction term is represented by Equation (20-1), and the moving average value of the square of the gradient of the attractive interaction term is represented by (20). The program according to claim 15, wherein the first update width is derived as in the formulas (21-1) and (21-2) when the formula (-2) is used.

In the procedure for searching for the optimal solution of the overall evaluation function,
A total update width (Δx) obtained by deriving a second update width (Δx _c ) of the variable of the individual according to the gradient of the partial evaluation function and adding the second update width to the first update width (Δx _q ). Is configured to calculate and derive
When deriving the second update width, the second update width is increased as the moving average value of the gradient of the partial evaluation function and the moving average value of the square of the gradient of the partial evaluation function become closer, and The program according to claim 15 or 16, wherein the second update width is derived so as to decrease as the moving average value of the gradient becomes smaller than the moving average value of the square of the gradient. The gradient of the partial evaluation function is set as equation (16), the moving average value of the gradient of the partial evaluation function is set as equation (17-1), and the moving average value of the square of the gradient of the partial evaluation function is set as (17-2). 18. The program according to claim 17, wherein the second update width is derived as in the formulas (18-1) and (18-2) when the formulas are used.

When the effective evaluation function is defined as the equation (1), the attractive interaction term is the equation (25), and ρ _i in the equation (25) is set to a predetermined value or increased from the initial value. The program according to any one of claims 15 to 18, wherein an attractive force is applied between individuals to converge to an optimum value.

In the procedure for searching for the optimal solution of the overall evaluation function,
When the effective evaluation function (H _eff ) composed of the partial evaluation function and the attractive interaction term (f (x _i −x _{i + 1} )) is expressed by equation (1), the attractive interaction term is expressed as (2 The program according to claim 14, wherein an attractive force is applied between the individuals so as to satisfy the expression (3) as an expression.

In the procedure for searching for the optimal solution of the overall evaluation function,
The program according to claim 20, wherein g (x) in the equation (3) is searched as an equation (4).

In the procedure for searching for the optimal solution of the overall evaluation function,
When the effective evaluation function is defined as equation (1), the term of attractive interaction is defined as equation (15), and ρ _i in the equation (15) is increased from the initial value, so The program according to claim 14, wherein an attractive force is applied to converge to an optimum value.

In the procedure for minimizing the effective evaluation function,
After obtaining a solution candidate for the partial evaluation function (H _i ), the differential value when the solution candidate is substituted is equal to the differential value of the partial evaluation function, and all of the search spaces including the solution candidate 21. An auxiliary function method is used in which a quadratic function satisfying a condition for obtaining a value larger than an evaluation value in the variable is used, and an extreme value of the quadratic function is repeatedly updated as a solution candidate for the next value. The program according to any one of 1 to 22. In the procedure for setting the variables of the plurality of individuals,
The program according to any one of claims 14 to 23, wherein the initial distribution is set so that the variables of the plurality of individuals are randomly distributed in the search space. If an estimated variable solution (xiz) is given in advance before starting the search for the variable,
In the procedure for setting the variables of the plurality of individuals,
24. The variable of the individual is set in a limited search range (Sa) limited to the inside of the search space as the initial distribution and including the estimated solution. Program. It said individual program according to any one of claims 14 to 25 comprising the same number of variables (x _i) between the partial cost function.

Images