DE202022106313U1 - A direct comparison based system for the analysis of evolutionary optimization algorithms - Google Patents

Abstract

A head-to-head comparison based system for evolutionary optimization algorithms, the system comprising:a comparison processing unit for performing a head-to-head comparison of two evolutionary optimization algorithms, one algorithm representing a primary algorithm and the other algorithm representing an alternative algorithm;a matrix processing unit for constructing a Prasatul matrix performing a head-to-head comparison of the best solution obtained with the primary algorithm and the alternative algorithm for a specified number of trials; anda ranking processing unit for ranking the algorithms based on their evaluated performance, performing a problem-related ranking and an overall ranking for ranking the algorithms based on a head-to-head comparison approach.

Description

FIELD OF THE INVENTION

The present disclosure relates to the field of analyzing the performance of evolutionary optimization algorithms. In particular, the present disclosure relates to a head-to-head comparison based system for analyzing evolutionary optimization algorithms.

BACKGROUND OF THE INVENTION

In the last decade, the application of Evolutionary Optimization Algorithms (EOAs) has increased exponentially. However, there is still a great need for better methods of evaluating the performance of algorithms. EOA is evaluated from two points of view, namely solution quality and convergence, where solution quality is the measurement of performance based on the solutions obtained in different trials of the algorithm. On the other hand, the convergence analysis is performed based on the solutions obtained in different interactions for a single algorithm trial. It can be seen that most of the assessment methods are based on the solution quality approach.

Despite the existence of different approaches, parametric and non-parametric approaches are commonly used for analyzing the solution quality of EOAs. The performance of the EOAs is compared by estimating basic statistical measures such as standard deviation, mean, median, maximum and minimum. However, this type of comparison is problematic because it is an indirect approach that does not directly compare the solutions obtained with two algorithms. It is important to note that equal statistics values do not mean that the solutions of the two algorithms are exactly the same. Another problem with comparing EOAs' performance is looking at multiple statistics to draw conclusions.

Therefore, a new direct approach to comparing the performance of EOAs is required.

In view of the previous discussion, it becomes clear that a head-to-head comparison based system for the analysis of evolutionary optimization algorithms is needed.

SUMMARY OF THE INVENTION

The present disclosure relates to a head-to-head comparison based system for analyzing evolutionary optimization algorithms. In the present disclosure, a head-to-head comparison approach for analyzing the performance of evolutionary optimization algorithms is proposed, developing a head-to-head comparison matrix, called the Prasatul matrix, that considers the result of the head-to-head comparison of the best solutions obtained with two algorithms for a certain number of attempts. To perform the comparison, five different performance measures based on the Prasatul matrix are developed to assess the performance of the algorithms in terms of optimality and comparability of the solutions. These values are used to develop a points-based approach for comparing the performance of multiple algorithms and additionally for ranking in terms of both solution quality and convergence analysis. The proposed approach system with the direct comparison approach is analyzed using six evolutionary optimization algorithms on 25 benchmark functions. Furthermore, in the present paper, a non-parametric statistical analysis, the Wilcoxon test for paired sums, is performed to verify the results of the proposed direct comparison approach.

The present disclosure aims to provide a head-to-head comparison based system for analysis of evolutionary optimization algorithms. The system comprises: a comparison processing unit for performing a head-to-head comparison of two evolutionary optimization algorithms, one algorithm representing a primary algorithm and the other algorithm representing an alternative algorithm; a matrix processing unit for preparing a Prasatul matrix by performing a direct comparison of the best solution obtained with the primary algorithm and the alternative algorithm for a certain number of trials; and a ranking processing unit for ranking the algorithms based on their evaluated performance, performing a problem-related ranking and an overall ranking for ranking the algorithms based on the direct comparison approach.

An aim of the present disclosure is to provide a head-to-head comparison based system for analyzing evolutionary optimization algorithms.

Another objective of the present disclosure is to provide an approach that directly compares both algorithms and dataset in prasatul matrix, rather than comparing statistics such as mean and standard deviation.

Another aim of the present disclosure is to provide five measures designed on the basis of the Prasatul matrix that are equally able to compare evolutionary optimization algorithms on the basis of both solution quality and convergence.

Another aim of the present disclosure is to provide a point-driven ranking scheme developed on the basis of the Prasatul matrix and able to rank the evolutionary optimization algorithm based on solution quality and convergence.

Another goal of the present disclosure is to provide a novel, score-driven comparison and ranking approach for analyzing the convergence of evolutionary optimization algorithms.

In order to further clarify the advantages and features of the present disclosure, a more detailed description of the invention is provided by reference to specific embodiments that are illustrated in the accompanying figures. It is understood that these figures represent only typical embodiments of the invention and therefore should not be considered as limiting the scope of the invention. The invention will be described and illustrated with additional specificity and detail with the accompanying figures.

character list

• 1 ein Blockdiagramm eines auf direktem Vergleich basierenden Systems zur Analyse von evolutionären Optimierungsalgorithmen gemäß einer Ausführungsform der vorliegenden Offenbarung zeigt; und
• 2 eine Prasatul-Matrix (L) mit verschiedenen Abstraktionsstufen gemäß einer Ausführungsform der vorliegenden Offenbarung zeigt.

These and other features, aspects, and advantages of the present disclosure will be better understood when the following detailed description is read with reference to the accompanying figures, in which like characters represent like parts throughout the figures, wherein:

• 1 Figure 12 shows a block diagram of a head-to-head comparison based system for analysis of evolutionary optimization algorithms according to an embodiment of the present disclosure; and
• 2 12 shows a Prasatul matrix (L) with different levels of abstraction according to an embodiment of the present disclosure.

Those skilled in the art will understand that the elements in the figures are presented for simplicity and are not necessarily drawn to scale. For example, the flow charts illustrate the method of key steps to enhance understanding of aspects of the present disclosure. In addition, one or more components of the device may be represented in the figures by conventional symbols, and the figures only show the specific details relevant to an understanding of the embodiments of the present disclosure in order not to over-detail the figures overloads that are readily apparent to those skilled in the art who have the descriptions contained herein.

DETAILED DESCRIPTION

For the purposes of promoting an understanding of the invention, reference will now be made to the embodiment illustrated in the figures and specific language will be used to describe the same. It should be understood, however, that no limitation on the scope of the invention is intended, and such alterations and further modifications to the illustrated system and such further applications of the principles of the invention set forth therein are contemplated as would occur to those skilled in the art invention would normally come to mind.

Those skilled in the art will understand that the foregoing general description and the following detailed description are exemplary and explanatory of the invention and are not intended to be limiting.

When this specification refers to "an aspect," "another aspect," or the like, it means that a particular feature, structure, or characteristic described in connection with the embodiment is present in at least one embodiment included in the present disclosure. Therefore, the phrases "in one embodiment," "in another embodiment," and similar phrases throughout this specification may or may not all refer to the same embodiment.

The terms "comprises,""including," or other variations thereof are intended to cover non-exclusive inclusion, such that a method or method that includes a list of steps includes not only those steps, but may also include other steps that are not expressly stated or pertaining to any such process or method. Likewise, any device or subsystem or element or structure or component preceded by "comprises...a" does not, without further limitation, exclude the existence of other devices or other subsystem or other element or other structure or other component or additional device or additional subsystems or additional elements or additional structures or additional components.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one skilled in the art to which this invention pertains. The system, methods, and examples provided herein are for purposes of illustration only and are not intended to be limiting.

Embodiments of the present disclosure are described in detail below with reference to the attached figures.

The functional units described in this specification are referred to as devices. A device may be implemented in programmable hardware devices such as processors, digital signal processors, central processing units, field programmable gate arrays, programmable logic arrays, programmable logic devices, cloud processing systems, or the like. The devices can also be implemented in software for execution by various types of processors. An identified device may include executable code and may include, for example, one or more physical or logical blocks of computer instructions, which may be organized as, for example, an object, procedure, function, or other construct. However, the executable file of an identified device does not have to reside physically in one location, but may consist of various instructions stored in different locations which, when logically assembled, form the device and fulfill the stated purpose of the device.

The executable code of a device or module can be a single instruction or a multitude of instructions, and can even be distributed across several different code segments, different applications, and multiple storage devices. Similarly, operational data may be identified and represented within the device and may be in any suitable form and organized in any suitable type of data structure. The operational data may be collected as a single set of data or distributed across various locations, including various storage devices, and may exist, at least in part, as electronic signals in a system or network.

When this specification refers to "a selected embodiment," "an embodiment," or "an embodiment," it means that a particular feature, structure, or characteristic described in connection with the embodiment is at least one embodiment of the disclosed subject matter. As such, the phrases "a selected embodiment," "in one embodiment," or "in one embodiment" throughout this specification are not necessarily all referring to the same embodiment.

Furthermore, the features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided in order to provide a thorough understanding of the embodiments of the disclosed subject matter. However, one skilled in the art will recognize that the disclosed subject matter may be practiced without one or more of the specific details, or using other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosed subject matter.

According to the example embodiments, the disclosed computer programs or modules may be implemented in many example ways, such as as an application residing in the memory of a device or as a hosted application running on a server and connected to the device application or browser via a Communicates series of standard protocols like TCP/IP, HTTP, XML, SOAP, REST, JSON and other sufficient protocols. The disclosed computer programs may be written in example programming languages executed from memory on the device or from a hosted server, such as BASIC, COBOL, C, C++, Java, Pascal, or scripting languages such as JavaScript, Python, Ruby, PHP, Perl, or others sufficient programming languages.

Some of the disclosed embodiments include or involve data transmission over a network, e.g. B. the transmission of various inputs or files over the network. The network can be, for example, the Internet, wide area networks (WANs), local area networks (LANs), analog or digital wired and wireless telephone networks (e.g. PSTN, Integrated Services Digital Network (ISDN), a cellular network and digital subscriber line (xDSL)), radio, television, cable, satellite and/or other transmission or tunneling mechanisms for the transmission of data. The network may comprise several networks or sub-networks, each of which e.g. B. a wired or wireless data path may contain. The network may comprise a circuit switched voice network, a packet switched data network or any other network for the transmission of electronic communications. The network may for example comprise Internet Protocol (IP) or Asynchronous Transfer Mode (ATM) based networks and may support voice e.g. B. VoIP, Voice-over-ATM or other comparable protocols for voice data communication. In one implementation, the network includes a cellular telephone network configured to allow the exchange of text or SMS messages.

Examples of a network include a personal area network (PAN), a storage area network (SAN), a home area network (HAN), a campus area network (CAN), a local area network (LAN), a wide area Network (WAN), a Metropolitan Area Network (MAN), a Virtual Private Network (VPN), an Enterprise Private Network (EPN), the Internet, a Global Area Network (GAN), and so on.

1 10 shows a block diagram of a head-to-head based system for evolutionary optimization algorithms, in accordance with an embodiment of the present disclosure. The system 100 comprises a comparison processing unit (102) for performing a head-to-head comparison of two evolutionary optimization algorithms, one algorithm representing a primary algorithm (102a) and the other algorithm representing an alternative algorithm (102b).

In one embodiment, a matrix processing unit (104) is used to construct a Prasatul matrix (104a) by directly comparing the best solution obtained with the primary algorithm (102a) and the alternative algorithm (102b) for a specified number of trials was carried out.

In one embodiment, a ranking processing unit (106) is used to rank the algorithms based on their assessed performance, performing a problem-related ranking (106a) and an overall ranking (106b) for ranking the algorithms based on a head-to-head comparison approach .

In one embodiment, a primary algorithm (102a) is compared to one or more alternative algorithms (102b), during the comparison each algorithm is given a chance to act as the primary algorithm (102a) while other algorithms act as alternative algorithms (102b). .

In one embodiment, the Prasatul matrix (104a) has three levels of abstraction in both rows and columns, these abstractions being defined on the basis of comparability and optimality with respect to the alternatives and the optima, respectively.

In one embodiment, the performance of the algorithms is interpreted in terms of the optimality of the solutions and the comparability of the solutions, using the head-to-head comparison and the overall comparison to determine the quality of the solutions, which contributes to a ranking of the algorithms.

In one embodiment, the head-to-head comparison and overall comparison are referred to as D-scores and K-scores, respectively, where D-scores include direct optimality and direct comparability and K-scores include overall optimality, overall comparability, and ensemble.

In one embodiment, an empirical analysis is performed using 25 benchmark functions, using several proven algorithms for head-to-head comparison and then verifying the results obtained.

In one embodiment, the comparison processing unit (102), the matrix processing unit (104) and the ranking processing unit (106) can be implemented in programmable hardware devices such as processors, digital signal processors, central processing units, field programmable gate arrays, programmable array logic, programmable logic devices, cloud processing systems or the like to be implemented.

• Primäre Algorithmen (Ap): Dies ist der Algorithmus, dessen Leistung im Vergleich zu anderen Algorithmen bewertet wird.
• Alternativer Algorithmus (Aq): Dies ist der Algorithmus, mit dem der primäre Algorithmus (Ap) verglichen wird.
• Bestes Universum (Ub): Dies bezieht sich auf die beste Lösung des Universums, wobei das beste Universum für Minimierungs- und Maximierungsprobleme als Gleichung (1) bzw. (2) definiert ist:

$$U_b=\text{arg}\underset{i}{\text{min}}{u}_i$$

und $$U_b=\text{arg}\underset{i}{\text{max}}{u}_i$$

In one embodiment, the roles of the participating optimization algorithms in the direct comparison approach are defined as follows:

• Primary Algorithms (Ap): This is the algorithm whose performance is evaluated against other algorithms.
• Alternate Algorithm (Aq): This is the algorithm against which the primary algorithm (Ap) is compared.
• Best Universe (Ub): This refers to the best universe solution, where the best universe for minimization and maximization problems is defined as Equations (1) and (2), respectively:

$$u_b=\text{bad}\underset{i}{\text{at least}}{\mathrm{and}}_i$$

and $$u_b=\text{bad}\underset{i}{\text{Max}}{\mathrm{and}}_i$$

Here ui stands for the i-th best solution from U in the residence equations.

Mean of the universe (Uσ): This is defined as the mean or average of all the best solutions in the universe and is defined by equation (3) below. $$u_{\sigma }\frac{{\displaystyle {\sum }_{\forall {\mathrm{and}}_i\in u}{\mathrm{and}}_i}}{\left|u\right|}$$

$$U_w=\text{arg}\underset{i}{\text{min}}{u}_i$$

Worst Universe (Uw): This is defined as the worst universe solution, where for minimization and maximization problems the worst universe is defined as given in Equations (4) and (5), respectively. $$u_w=\text{bad}\underset{i}{\text{Max}}{\mathrm{and}}_i$$

$$u_w=\text{bad}\underset{i}{\text{at least}}{\mathrm{and}}_i$$

Here ui stands for the i-th best solution from U in the residence equations.

$$O=\text{arg}\underset{x}{\text{max}}ƒ\left(x\right)$$

Optimum (O): It is defined as the optimal value of the search or solution space of the objective function f(x) considered for both algorithms. For minimization and maximization problems, the optima are defined as Equations (6) and (7), respectively. $$O=\text{bad}\underset{x}{\text{at least}}ƒ\left(x\right)$$

$$O=\text{bad}\underset{x}{\text{Max}}ƒ\left(x\right)$$

In one embodiment, a solution is simply compared to another solution to determine the degree of comparability of a solution, with three levels of comparability defined and given in equations (8), (9), and (10), as noted below.

Comparability level 1: If a solution pi obtained with an algorithm Ap is better than the solution q _i obtained with an algorithm Aq, then the solution p _i is considered as comparability level 1. Therefore, for the algorithms Ap and Aq in a minimization problem, the set of comparability level 1 (L° ₁ ) can be defined as follows: $$L_1^c=\left\{p_i|p_i<q_i,\forall \left(p_i,q_i\right)\right\}$$

Comparability level 2: In this case, if a solution pi is the same as or equal to the solution q _i , then the solution p _i is defined as comparability level 2. Therefore, for the algorithms Ap and Aq, the set of comparability level 2 (L° ₂ ) can be defined as follows:

Comparability level 3: If in this case a solution pi is worse than the solution q _i , then the solution p _i is defined as comparability level 3. Therefore, for the algorithms Ap and Aq, the set of comparability level 3 (L° ₃ ) can be defined as follows: $$L_3^c=\left\{p_i|p_i>q_i,\forall \left(p_i,q_i\right)\right\}$$

In one embodiment, the optimality level of a solution can be defined by the distance between that solution and the actual optima of the given problem (maximization or minimization). There are three levels of optimality levels, which are defined below.

Optimality level 1: If a solution pi obtained with an algorithm Ap lies between the optimum (O) and the best universe (Ub), which also includes O and Ub, then the solution p _i is considered optimality level 1. Therefore, for algorithms Ap in a minimization problem, the set of optimality level 1 (L ⁰ ₁ ) can be defined as follows: $$L_1^O=\left\{p_i|p_i\in \left[u,u_b\right]\right\}$$

Optimality level 2: If a solution pi obtained with an algorithm Ap lies between the best value in the universe (Ub) and the universal mean (Uσ), which also includes the universal mean (Uσ), then the solution pi is considered to be Optimality level 2 considered. Therefore, for algorithms Ap in a minimization problem, the set of optimality level 2 (Lo2) can be defined as follows: $$L_2^O=\left\{p_i|p_i\in \left[u_b,u_{\sigma }\right]\right\}$$

Optimality level 3: In this case, if a solution pi obtained with an algorithm Ap lies between the best (Ub) and the worst (Uw) universe, including the worst universe (Uw), then the solution pi is considered optimality level 3 considered. Therefore, for algorithms Ap for a minimization problem, the set of optimality level 3 (L° ₃ ) can be defined as follows: $$L_3^O=\left\{p_i|p_i\in \left[u_{\sigma },u_w\right]\right\}$$

2 Figure 12 shows a Prasatul matrix (L) with different levels of abstraction in accordance with an embodiment of the present Epiphany. This matrix is constructed by directly comparing the best solutions obtained with the primary algorithm and the best solutions obtained with the alternative algorithm for a number of n trials. As already mentioned, the Prasatul matrix has three abstractions, which can also be seen in the figure, these abstractions are in rows and columns defined on the basis of comparability and optimality in terms of alternatives and optima, respectively.

The abstraction levels victory, draw and defeat refer to the comparability levels 1, 2 and 3 of the solution of the main algorithm with the solution of the alternative algorithm. These are represented in the rows of the Prasatul matrix and these abstractions indicate how good or bad the solution is compared to the alternative solutions. Winning means the solution is better, draw means the solution is equal or equal, and losing means the solution is worse, with the corresponding row indicating the number of such solutions.

The best, average and worst abstractions presented in the columns represent the comparison of the best solutions of the primary algorithm with the optima enumerated in optimality levels 1, 2 and 3. These abstractions in the columns indicate how good or bad the solution is in terms of the optima. Best means the solution is at most the universal best value (Ub) from the optimum (O), average means the solution is at most the universal mean (Uσ) and more than the universal best value (Ub) from the optimum , and worst means that the solution is more than the universal mean (Uσ) away from the optimum, in which case in all these cases the corresponding columns show the number of such solutions. If both comparability and optimality levels are compared, and if the abstraction levels win and best are taken together, then this is expressed as the number of solutions in both the comparability level 1 set (L° ₁ ) and the optimality level 1 set (L° ₁ ) is considered, ie (L° ₁ ∩ L° ₁ ). Similarly, when the levels of abstraction profit and average are taken together, ie (L° ₁ ∩ L° ₂ ). Therefore, the elements of the Prasatul matrix (L) can be defined by equation (14) below, $$L_{i,j}=\left|L_i^c\cap L_j^O\right|,\forall i,j$$

$$O_2=\frac{L_{12}}{1+{\displaystyle {\sum }_{i=1}^3{L}_{i2}}}$$

$$O_3=\frac{L_{13}}{1+{\displaystyle {\sum }_{i=1}^3{L}_{i3}}}$$

In one embodiment, the optimality and comparability of the solutions are considered to interpret the performance of the algorithms, where the quality of the solution is assessed based on the head-to-head and overall comparisons, referred to as D-scores and K-scores, respectively . Two D-scores are defined as Direct Optimality (DO) and Direct Comparability (DC). Three K-scores are defined as overall optimality (KO), overall comparability (KC), and overall context (KT). The best, the average and the worst abstraction, ie the optimality of levels 1, 2 and 3 are defined with equations (15), (16) and (17) below. $$O_1=\frac{L_{11}}{1+{\displaystyle {\sum }_{i=1}^3{L}_{i1}}}$$

$${\mathrm{<figure-callout\; id="2"\; label="O"\; filenames="DE202022106313U1\_0040.png"\; state="\{\{state\}\}">O</figure-callout>}}_2=\frac{L_{12}}{1+{\displaystyle {\sum }_{i=1}^3{\mathrm{<figure-callout\; id="2"\; label="L\; i"\; filenames="DE202022106313U1\_0040.png"\; state="\{\{state\}\}">L</figure-callout>}}_i}}$$

$$O_3=\frac{L_{13}}{1+{\displaystyle {\sum }_{i=1}^3{L}_{i3}}}$$

In one embodiment, the optimality and comparability of the solutions are considered to interpret the performance of the algorithms, where the quality of the solution is assessed based on the head-to-head and overall comparisons, referred to as D-scores and K-scores, respectively . Two D-scores are defined as Direct Optimality (DO) and Direct Comparability (DC). Three K-scores are defined as overall optimality (KO), overall comparability (KC), and overall context (KT). The best, the average and the worst abstraction, i.e. the optimality of levels 1, 2 and 3 are defined with equations (15), (16) and (17) below.

In equation (15) above, a higher value of O1 means that the primary algorithm's best solution is better than the alternative algorithm's best solution. Similarly, a high value of O2 and O3 is interpreted as meaning that the average solution of the primary algorithm is better than the average solution of the alternative algorithm and the worst solution of the primary algorithm is better than the worst solution of the alternative algorithm. The direct optimality (DO) can be measured by combining the weighted comparative optimality of both algorithms at all three levels of abstraction, where the direct optimality (DO) of the primary algorithm compared to the alternative algorithm can be defined as follows: $$\text{DO}={\text{O}}_1+0.5*{\text{<figure-callout id="2" label="O" filenames="DE202022106313U1\_0040.png" state="{{state}}">O</figure-callout>}}_2-{\text{O}}_3$$

The DO value indicates that the primary algorithm is better than the alternative algorithm.

$$C_2=\frac{L_{21}}{1+{\displaystyle {\sum }_{i=1}^3{L}_{2i}}}$$

$$C_3=\frac{L_{31}}{1+{\displaystyle {\sum }_{i=1}^3{L}_{3i}}}$$

Similarly, direct comparability (DC) can be measured for three rights, namely win, tie and loss, with the expression for these rights given below. $$C_1=\frac{L_{11}}{1+{\displaystyle {\sum }_{i=1}^3{L}_{1i}}}$$

$${\mathrm{<figure-callout\; id="2"\; label="C"\; filenames="DE202022106313U1\_0040.png"\; state="\{\{state\}\}">C</figure-callout>}}_2=\frac{L_{21}}{1+{\displaystyle {\sum }_{i=1}^3{L}_{2i}}}$$

$$C_3=\frac{L_{31}}{1+{\displaystyle {\sum }_{i=1}^3{L}_{3i}}}$$

In the equations above, higher values of C1, C2, and C3 indicate that the primary algorithm's solutions are better, equal, or worse than the alternative algorithm's solutions. Direct Comparability (DC) is measured by combining the weighted comparability of both algorithms at all three levels of abstraction, with the expression for the DC of the primary algorithm versus the alternative algorithm given below as Equation (22). $$DC=C_1+0.5\ast {\mathrm{<figure-callout\; id="2"\; label="C"\; filenames="DE202022106313U1\_0040.png"\; state="\{\{state\}\}">C</figure-callout>}}_2-C_3$$

A higher value of DC means that the primary algorithm is better than the alternative algorithm in terms of optimality.

In one embodiment, the K-scores take into account all comparison results. Overall performance is considered in K-Scores both in the context of optimality and comparability. However, the results of accounting comparability are not considered in overall optimality, and likewise the results of accounting optimality are not considered in overall compatibility. The overall optimality for the best level 1 is defined by equation (23). $$K_1^O=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{i1}}}{{\displaystyle \sum L_{ii}}}$$

$$K_3^o=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{i2}}}{{\displaystyle \sum L_{ii}}}$$

A high value of Ko1 means that the comparison algorithm's overall solutions are best in terms of optimality, regardless of whether it is losing or winning. Similarly, the overall optimality in stages 2 and 3 can be defined by equations (24) and (25). $$K_2^O=\frac{{\displaystyle {\sum }_{i=1}^3{\mathrm{<figure-callout\; id="2"\; label="L\; i"\; filenames="DE202022106313U1\_0040.png"\; state="\{\{state\}\}">L</figure-callout>}}_i}}{{\displaystyle \sum L_{ii}}}$$

$$K_3^O=\frac{{\displaystyle {\sum }_{i=1}^3{\mathrm{<figure-callout\; id="2"\; label="L\; i"\; filenames="DE202022106313U1\_0040.png"\; state="\{\{state\}\}">L</figure-callout>}}_i}}{{\displaystyle \sum L_{ii}}}$$

Where in Equations (24) and (25) a high value of Ko2 and Ko3 indicates that the overall solutions are average and bad in the optimal sense, respectively, regardless of whether they are losing or winning.

$$KO=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{i1}+0.5}\ast {\displaystyle {\sum }_{i=1}^3{L}_{i2}}-{\displaystyle {\sum }_{i=1}^3{L}_{i3}}}{n}$$

The overall optimality of the comparison algorithms is computed by combining the overall optimality at all three levels of abstraction (best, average, and worst). The overall optimality (KO) is defined by Equations (26) and (27). $$KO=K_1^O+0.5\ast K_2^O-K_3^O$$

$$KO=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{i1}+0.5}\ast {\displaystyle {\sum }_{i=1}^3{\mathrm{<figure-callout\; id="2"\; label="L\; i"\; filenames="DE202022106313U1\_0040.png"\; state="\{\{state\}\}">L</figure-callout>}}_i}-{\displaystyle {\sum }_{i=1}^3{L}_{i3}}}{n}$$

A higher value of KO means that the solutions of the primary algorithms are very optimal.

$$K_2^c=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{2i}}}{{\displaystyle \sum L_{ii}}}$$

$$K_3^c=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{3i}}}{{\displaystyle \sum L_{ii}}}$$

Similarly, overall comparability can be measured for the win, draw, and loss abstraction. The overall comparability at levels 1, 2 and 3 can be defined using the equations below. $$K_1^c=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{1i}}}{{\displaystyle \sum L_{ii}}}$$

$$K_2^c=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{2i}}}{{\displaystyle \sum L_{ii}}}$$

$$K_3^c=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{3i}}}{{\displaystyle \sum L_{ii}}}$$

$$KC=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{1i}+0.5}\ast {\displaystyle {\sum }_{i=1}^3{L}_{2i}}-{\displaystyle {\sum }_{i=1}^3{L}_{3i}}}{n}$$

In the equations above, high values of K ^c ₁ , K ^c ₂ , and K ^c ₃ indicate that the overall solutions are better, equal, and worse than those of other algorithms, whether they are best, average, or worst. The overall comparability of the comparison algorithms can be measured by combining the overall comparability at all three levels of abstraction, where the overall comparability (KC) of the main algorithm compared to the alternative algorithms is expressed in equations (31) and (32). $$KC=K_1^c+0.5\ast K_2^c-K_3^c$$

$$KC=\frac{{\displaystyle {\sum }_{i=1}^3{L}_{1i}+0.5}\ast {\displaystyle {\sum }_{i=1}^3{L}_{2i}}-{\displaystyle {\sum }_{i=1}^3{L}_{3i}}}{n}$$

A higher value of KC indicates that the solutions of the primary algorithms are better than those of the alternative algorithms. Since both the best and the average level of abstraction are of great importance for the overall comparability and optimality, a combination of elements of the Prasatul matrix is performed where the abstraction levels best and average and win and tie overlap, and this is called overall together (KT) which is defined as equation (33). $$KT=\frac{a+b+\mathrm{i.e}+e}{n}$$

A higher value of KT means that the primary algorithm is better overall than the alternative algorithms, whether optimal or comparable.

In one embodiment, the D-scores and K-scores have the ranges [-1, +1.5] and [-1, +1], respectively, but the KT has the range [0,1].

In one embodiment, the Prasatul matrix provides a one-to-one comparison between two algorithms, namely the primary and the alternate algorithm. As D-Scores and K-Scores provide the performance of the primary algorithm compared to the alternative algorithm. However, to evaluate the performance of a primary algorithm compared to several alternative algorithms, i. H. for a one-to-many comparison, multiple pairs of primary and alternate algorithms must be considered. If a primary algorithm has to be compared to k alternatives, there are a number of k Prasatul matrices for k pairs of primary and alternative algorithms. Therefore, for each of the k pairs, there is a separate set of D-scores and K-scores that indicate the performance of the primary algorithm compared to k alternative algorithms. To get the overall performance of the primary algorithm compared to k alternatives for a given problem, all D-scores and K-scores are considered simply by averaging, as given in the following equations.

D values: $$\begin{array}{l}ADO=\frac{{\displaystyle {\sum }_{i=1}^{k}D{O}_i}}{k}\hfill \\ ADC=\frac{{\displaystyle {\sum }_{i=1}^{k}D{C}_i}}{k}\hfill \end{array}$$

K values: $$\begin{array}{l}AKO=\frac{{\displaystyle {\sum }_{i=1}^{i}K{O}_i}}{k}\hfill \\ AKC=\frac{{\displaystyle {\sum }_{i=1}^{i}K{C}_i}}{k}\hfill \\ AKT=\frac{{\displaystyle {\sum }_{i=1}^{i}K{T}_i}}{k}\hfill \end{array}$$

Where Equation (34) represents the average DO and DC value and Equation (35) represents the average KO, KC and KT value.

In one embodiment, the ranking of the algorithms is performed with the direct comparison approach using a problem-related ranking and an overall ranking, both of which ranking approaches are explained below.

$$\begin{array}{l}PKO=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{d}K{O}_{ij}}}}{k\times d}\hfill \\ PKC=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{d}K{C}_{ij}}}}{k\times d}\hfill \\ PKT=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{d}K{T}_{ij}}}}{k\times d}\hfill \end{array}$$

Problem-related ranking: The problem-related ranking of the evolutionary optimization algorithms is performed for k algorithms and d dimensions in terms of D-scores and K-scores as given in the following equations, where Equation (36) contains the D-scores (DO and DC) and Equation (37) represents the K-scores (KO, KC and KT). $$\begin{array}{l}PDO=\frac{{\displaystyle {\sum }_{i=1}^i{\displaystyle {\sum }_{j=1}^{\mathrm{i.e}}D{O}_{ij}}}}{k\times \mathrm{i.e}}\\ PDC=\frac{{\displaystyle {\sum }_{i=1}^i{\displaystyle {\sum }_{j=1}^{\mathrm{i.e}}D{C}_{ij}}}}{k\times \mathrm{i.e}}\end{array}$$

$$\begin{array}{l}PKO=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{\mathrm{i.e}}K{O}_{ij}}}}{k\times \mathrm{i.e}}\hfill \\ PKC=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{\mathrm{i.e}}K{C}_{ij}}}}{k\times \mathrm{i.e}}\hfill \\ PKT=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{\mathrm{i.e}}K{T}_{ij}}}}{k\times \mathrm{i.e}}\hfill \end{array}$$

$$\begin{array}{l}OKO=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^d{\displaystyle {\sum }_{l=1}^{p}K{O}_{ijl}}}}}{k\times d\times p}\hfill \\ OKC=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^d{\displaystyle {\sum }_{l=1}^{p}K{C}_{ijl}}}}}{k\times d\times p}\hfill \\ OKT=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^d{\displaystyle {\sum }_{l=1}^{p}K{T}_{ijl}}}}}{k\times d\times p}\hfill \end{array}$$

Overall scoring: The overall scoring of the EOAs for k algorithms, d dimensions and p problems is performed in terms of D-scores and K-scores as given in the equation below, where Equation (38) contains the D-scores (DO and DC) and Equation (39) represents the K-scores (KO, KC and KT). $$\begin{array}{l}ODO=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{\mathrm{i.e}}{\displaystyle {\sum }_{l=1}^{p}D{O}_{ijl}}}}}{k\times \mathrm{i.e}\times p}\hfill \\ ODC=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{\mathrm{i.e}}{\displaystyle {\sum }_{l=1}^{p}D{C}_{ijl}}}}}{k\times \mathrm{i.e}\times p}\hfill \end{array}$$

$$\begin{array}{l}OKO=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{\mathrm{i.e}}{\displaystyle {\sum }_{l=1}^{p}K{O}_{ijl}}}}}{k\times \mathrm{i.e}\times p}\hfill \\ OKC=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{\mathrm{i.e}}{\displaystyle {\sum }_{l=1}^{p}K{C}_{ijl}}}}}{k\times \mathrm{i.e}\times p}\hfill \\ OKT=\frac{{\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^{\mathrm{i.e}}{\displaystyle {\sum }_{l=1}^{p}K{T}_{ijl}}}}}{k\times \mathrm{i.e}\times p}\hfill \end{array}$$

In one embodiment, an experimental setup is developed to conduct the experiment to verify the proposed head-to-head comparison approach, performing an empirical analysis with 25 benchmark functions, evaluating well-established algorithms with the proposed methodology and verifying them with the existing studies will. The experiments carried out are explained in detail below.

To carry out the experiments, a set of 25 benchmark test problems (F1-F25) is selected, these test problems being minimization problems consisting of 5 unimodal and 20 multimodal functions, with 7 of the 20 multimodal functions being basic functions, 2 advanced features and 11 hybrid features are. In order to ensure that the solution never lies in the center of the solution, the optima of all functions are shifted from the origin or from the previous position, this shift has caused difficulties for the algorithms, which have a central tendency to converge towards the optima . Hybridization has further complicated the problem, leaving the algorithms unable to follow specific patterns to reach the optima.

The effectiveness of the proposed Prasatul matrix and approach, six commonly applied optimization algorithms are selected to conduct the experiments, these algorithms include Ant Lion Optimizer (ALO), Gray Wolf Optimizer (GWO), Moth Flame Optimizer (MFO), Self -Adaptive Differential Evolution (SaDE), Sine Cosine Algorithm (SCA), and Whale Optimization Algorithm (WOA).

Default parameter settings are considered for all algorithms, with the exception of SaDE, an adaptive version of differential evolution (DE), which does not require parameter settings. MATLAB codes available online have been considered for all algorithms. The algorithm for generating the Prasatul matrix as well as the D-scores and K-scores are implemented in MATLAB R2015a and are available online. All experiments are performed on a computer with the specifications Intel (R) Core (TM) i7-8565U CPU @ 1.80 GHz with 8 cores, 4.6 GHz speed, NVIDIA GeForce MX130 graphics card, 16 GB RAM, 1TB HDD and 64-bit (AMD) Windows 10 operating system performed.

All experiments are performed with a population size of 100. The dimensions 10, 20 and 30 are considered for the functions F1-F14 and 2, 5 and 10 for the hybrid functions F15-F25 for both the quality and the convergence comparison. All algorithms are run over 50 trials up to 1000 generations so the best values can be observed. For the convergence comparison, each algorithm is run up to 1000 generations and the best value achieved in each generation is noted. For the quality comparison, the functions F1-F14 are considered, which include 5 unimodal functions, while the remaining functions are multimodal functions. The functions F15-F25 are used for the convergence comparison, all are hybrid multimodal functions. However, all 25 functions, i. H. F1-F25, considered.

In one embodiment, the performance of evolutionary optimization algorithms is compared based on two factors. One is solution quality, which determines how optimal the solutions are, and the other factor is convergence, which determines how fast the algorithm moves towards the optimum. With the proposed Prasatul matrix, the algorithms are measured and ranked based on both factors. In addition, the results of the Prasatul matrix are checked against the results of the Wilcoxon rank sum test.

In one embodiment, the best scores obtained over 50 trials with each of the six algorithms are considered. The algorithms are paired to generate the Prasatul matrix and thereby obtain D-scores and K-scores. The obtained D-scores for all pairs of SaDE and the other five algorithms with dimensions 10, 20, and 30 show that SaDE performs better in most functions across all three dimensions, indicated by high +ve scores. However, in some cases, SaDE performs poorly on the F7, F8, F11, F13, and F14 test issues, as indicated by low and negative scores. For problem F13, although the DO score is -ve, the DC scores are mostly +ve, suggesting that most solutions are better than others, even though they are worst in terms of optimality. A similar pattern can be seen in the ADO and ADC ratings. Similarly, the K-scores show that SaDE compared to the other five algorithms in has a better solution quality in all three dimensions, ie better in terms of both optimality and comparability.

In one embodiment, the best values obtained in each of the 1000 iterations over 50 trials of each of the six selected optimization algorithms are averaged iteration-by-iteration to perform a convergence comparison with the proposed direct comparison approach. Then the Prasatul matrices and the corresponding D-scores and K-scores are determined. D-scores and K-scores are obtained with all pairs of the ALO algorithm and the other four 10-dimensional algorithms, with the SaDE algorithm not being run on F23 as it is not considered for a convergence comparison. The obtained results show that the ALO algorithm mostly performs better than the other algorithms when it comes to convergence. Nevertheless, it can be observed that for some functions like F19, F22 and F25, the convergence of ALO is poor compared to other algorithms. In addition, it is found that the ALO algorithm performs worst compared to the MFO algorithm in the majority of cases, which can be overestimated by -ve values for both the D-scores and the K-scores. Although the ADO, ADO, AKO, and AKC scores indicate that ALO's convergence is better than the other algorithms, the scores are quite low.

In one embodiment, the ranking of the algorithms with a direction comparison approach is performed using two approaches. The first approach is problem-based ranking, which ranks the algorithms together based on specific problems and considering the different dimensions. The second approach is overall ranking, which considers different problems and their dimensions together to determine the overall performance of an algorithm compared to other alternatives. Both types of ranking are performed based on solution quality and algorithm convergence. The observations from both results are explained below.

Problem-related ranking: The problem-related ranking of problems (F1-F14) in relation to the solution quality of the algorithms shows that SaDE performs best on most functions and produces good-quality solutions. It was also noted that the ALO and MFO algorithms are strong competitors, but ALO mostly ranks second to SaDE. SCA is the worst-performing algorithm and therefore ranks last for almost all functions in terms of solution quality. It can be observed that the PKT score is more than sufficient for determining the problem-related ranking of the algorithms, since it includes both the optimality and the comparability, since it includes the KT scores. Therefore, only PKT is considered for convergence. The problem-related ranking of functions F15 to F25 in terms of algorithm convergence shows that the MFO algorithm performs best in terms of convergence for most functions. On the other hand, the performance of the WOA algorithm in terms of convergence is the worst for most functions.

Score: The results of the score on the solution quality of the algorithms show that the SaDE algorithm, similar to the problem-based score, is the best-performing algorithm in terms of the solution quality for F1-F14 functions. The SaDE algorithm is the best algorithm in all five rankings. The SCA algorithm is the worst algorithm while the ALO algorithm is the second best. However, for the functions F15-F25, the ALO algorithm is the best and the WOA algorithm is the worst algorithm in terms of solution quality.

The overall convergence score shows that the SaDE algorithm performs best, but only for F1-F14 functions. For F15-F25 functions, MFO algorithm is the best. Also, when comparing the one-to-one and one-to-many convergence, it was found that the ALO and MFO algorithms are strong competitors. Similar to the problem-based ranking, WOA is the algorithm with the worst convergence in the overall ranking.

In one embodiment, a statistical Wilconxon rank sum test is performed to analyze the significant difference of the solutions obtained with the SaDE algorithm compared to other algorithms, using an accuracy of 5%. The results show that there is a statistically significant difference in each function. However, no significant difference is found for the F15-F25 functions, except for the WOA algorithm, which performs the worst in terms of convergence. This test also confirms that the SaDE algorithm performs best in terms of quality and the WOA algorithm performs worst in terms of convergence. The results of the direct comparison agree with those of the Wilcoxon test.

The figures and the preceding description give examples of embodiments. Those skilled in the art will understand that one or more of the elements described may well be combined into a single functional element. Alternatively, certain elements can be broken down into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, the order of the processes described herein may be changed and is not limited to the manner described herein. Additionally, the actions of a flowchart need not be performed in the order shown; Also, not all actions have to be carried out. Also, the actions that are not dependent on other actions can be performed in parallel with the other actions. The scope of the embodiments is in no way limited by these specific examples. Numerous variations are possible, regardless of whether they are explicitly mentioned in the description or not, e.g. B. Differences in structure, dimensions and use of materials. The scope of the embodiments is at least as broad as indicated in the following claims.

Advantages, other benefits, and solutions to problems have been described above with respect to particular embodiments. However, the benefits, advantages, problem solutions, and components that can cause an advantage, benefit, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature or component of any or all claims.

reference list

100

A direct comparison based system for evolutionary optimization algorithms

102

A comparison processing unit

104

A matrix processing unit

106

A ranking processing unit

Claims (6)

A head-to-head comparison based system for evolutionary optimization algorithms, the system includes: a comparison processing unit for performing a head-to-head comparison of two evolutionary optimization algorithms, one algorithm being a primary algorithm and the other algorithm being an alternative algorithm; a matrix processing unit for constructing a Prasatul matrix by performing a head-to-head comparison of the best solution obtained with the primary algorithm and the alternative algorithm for a specified number of trials; and a ranking processing unit for ranking the algorithms based on their evaluated performance, performing a problem-related ranking and an overall ranking for ranking the algorithms based on a head-to-head comparison approach. system after claim 1 , where a primary algorithm is compared to one or more alternative algorithms, during which comparison each algorithm is given a chance to act as the primary algorithm while other algorithms act as alternative algorithms. system after claim 1 , where the Prasatul matrix has three levels of abstraction both in the rows and in the columns, these abstractions being defined on the basis of comparability and optimality with respect to the alternatives and optima, respectively. system after claim 1 , where the performance of the algorithms is interpreted as the optimality of the solutions and as the comparability of the solutions, using direct comparison and overall comparison to determine the quality of the solutions, thereby ranking the algorithms. system after claim 3 , where direct comparison and overall comparison are referred to as D-scores and K-scores, respectively, where D-scores encompass direct optimality and direct comparability, and K-scores encompass overall optimality, overall comparability, and ensemble. system after claim 1 , performing an empirical analysis using 25 benchmark functions, using several well-established algorithms to perform the head-to-head comparison, and then verifying the results obtained.

Images