CN113377651A - Class integration test sequence generation method based on reinforcement learning - Google Patents

Abstract

The invention discloses a method for generating a class integration test sequence based on reinforcement learning, which belongs to the technical field of software testing. It includes the following steps: 1) define reinforcement learning tasks; 2) program static analysis; 3) measure test pile complexity; 4) design reward function; 5) design value function; 6) generate class integration test sequence. The invention solves the problem that the existing method for generating a class integration test sequence based on reinforcement learning is not accurate enough to evaluate and determine the overall cost of the class integration test sequence, and provides more accurate testing work for testers in actual production and life. The measurement method improves the efficiency of integration testing and can better control the quality of the product.

Description

A Reinforcement Learning-Based Method for Generating Class Integration Test Sequences

technical field

The invention belongs to the technical field of software testing, and in particular relates to a method for generating a class integration test sequence based on reinforcement learning.

Background technique

The software testing phase mainly includes unit testing, integration testing, system testing, verification and validation, and regression testing. Among them, integration testing is to assemble all software units into modules, subsystems or systems on the basis of unit testing, and detect whether each part of the work has reached or achieved the corresponding technical indicators and requirements, so as to ensure that each unit can be combined according to the established intention. Work cooperatively to ensure that increments behave correctly. However, the object-oriented program has no obvious hierarchical division, and the calling relationship between them is intricate network structure, and the traditional integration testing strategy cannot be applied well. Therefore, it is necessary to propose a new integration testing strategy that conforms to the characteristics of object-oriented programs. This new strategy takes the class as the object and aims to generate the optimal class integration testing sequence, and then determines the testing method.

According to the inter-class dependencies of object-oriented programs, researchers in the field of software engineering have proposed integration strategies based on class integration test sequences. In the testing process, these strategies often need to construct the required test stubs for some classes in the object-oriented program to complete some functions instead of them. The cost of this task is high, and generally there is no way to avoid it, so how to reduce the cost has become a key issue in integration testing. In the research process, scholars measure the cost of test piles by calculating the complexity of test piles. Different types of integration test sequences have different test pile complexities and test costs. Sorting the classes in the test program reasonably to obtain a feasible class integration test sequence can greatly reduce the overall complexity of the test piles that need to be constructed, thereby reducing the test cost as much as possible.

Existing reinforcement learning-based class integration test sequence generation methods ignore the evaluation index of test stub complexity. These methods assume that each class has the same degree of dependency, that is, each test stub has the same complexity. However, different test stubs have different complexity. Fewer test stubs does not mean that the cost of determining a class integration test sequence is lower. Therefore, the existing reinforcement learning-based generation methods for class integration test sequences use the number of class test piles as a measure to determine the overall cost of class integration test sequences, which is not accurate enough. Therefore, it is very important for integration testing to propose a reasonable integration test sequence generation technique and to make the evaluation index precise.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method for generating a class integration test sequence based on reinforcement learning, which solves the problem that the existing method for generating a class integration test sequence based on reinforcement learning is not accurate enough to evaluate and determine the overall cost of the class integration test sequence. This can provide a more accurate measurement method for testers to carry out testing work in actual production and life, thereby improving the efficiency of integration testing.

The present invention is realized according to the following technical solutions:

A method for generating class integration test sequence based on reinforcement learning, the specific process is as follows:

Step 1. Define the reinforcement learning task: The task of reinforcement learning is to make the agent continue to try in the environment, constantly adjust the strategy according to the reward value obtained, and finally generate a better strategy, and the agent can know where What action should be performed in the state;

Step 2. Program static analysis: perform static analysis on the source program, use the obtained information to calculate the properties and method complexity between classes, calculate the attribute coupling between classes through the attribute complexity, and calculate the method between classes through the method complexity coupling;

Step 3, measure the complexity of the test pile: calculate the complexity of the test pile according to the properties and method complexity obtained in the front, and provide information for the design of the reward function later;

Step 4. Design the reward function: integrate the calculation of the complexity of the test pile into the design of the reward function, and guide the agent to learn in the direction of lower complexity of the test pile;

Step 5. Design the value function: feedback the value function through the reward function, and ensure that the accumulated reward is maximized through the setting of the value function;

Step 6. Generate a class integration test sequence: When the agent completes the set training times, select the action path with the largest overall reward value, which is the class integration test sequence learned this time.

For a specific scheme, the specific steps of step 1 are as follows:

1.1. Consider the software system to be analyzed as a collection of classes that need to be integrated during testing;

1.2. Retain the action sequence performed by the agent in the path, that is, the action history, as a candidate solution for the integration test class sequence;

1.3. Find an action history with the largest overall reward from the candidate solutions, which is the class integration test sequence required for this learning process.

For a specific scheme, the specific steps of step 2 are as follows:

2.1. Analyze the relationship between classes, calculate the attribute coupling between classes through the attribute complexity, use A(i, j) to represent, i, j respectively represent the class in the program, the attribute complexity is equal to the number of member variables of i calling j , the sum of the method parameter type and the number of return values of the method;

2.2. Calculate the method coupling between classes through method complexity, represented by M(i, j). The number of method complexity is equal to the number of methods in j called by i.

2.3. Standardize properties and method complexity.

For a specific scheme, the specific steps of step 3 are as follows:

3.1. Calculate the weight of attributes and method complexity between classes by entropy weight method;

3.2. Combining attributes and method complexity to calculate the test pile complexity;

3.3. When the class integration test sequence is obtained, the complexity of the test piles generated in the process is accumulated to obtain the overall test pile complexity.

Further Scenario: Test Stub Complexity

，

，A(i, j)代表类i和j之间的属性复杂度，M(i, j)代表类i和j之间的方法复杂度，熵权法最先进行的就是对连个指标进行标准化，这样得到的结果均在0到1之间，标准化之后的类间的属性复杂度是

，方法复杂度是

。in,

,

, A(i, j) represents the attribute complexity between classes i and j, M(i, j) represents the method complexity between classes i and j, and the entropy weight method is first performed on two indicators. Standardization, the results obtained in this way are all between 0 and 1, and the attribute complexity between classes after standardization is

, the method complexity is

.

For a specific scheme, the specific steps of step 4 are as follows:

4.1. Design a reward function with a higher reward value when the agent explores and integrates into a better class;

4.2. When any action class appears twice in the process, give this path a minimum value of -∞, so as to avoid it when continuing to explore;

4.3. Design the reward function by combining the above two points with the complexity of the test pile.

Further options:

Among them, the agent reaches σ _i after i-1 state transitions, σ _i represents the state path, r(σ _i ) represents the reward value that the state path will get, and Max represents the maximum reward value, where 1000 is taken here, and c is a positive Integer value, here is 100, a _σi represents the action history corresponding to the state path, and SCplx() represents the test pile complexity. When any non-compliance occurs in the process, the environment will give the agent a penalty value.

For a specific scheme, the specific steps of step 5 are as follows:

5.1. According to the state generated by the interaction of the environment and the selected action, a Q value of the immediate reward is obtained, which is represented by Q(s, a), where s represents the state and a represents the action;

5.2. Select the largest Q(s', a') according to the next state s' and multiply it by a discount factor γ;

5.3, plus the reward value r obtained by the agent performing action a in state s;

5.4. Multiply the whole by a learning rate α;

5.5, plus the Q value of the immediate return just now, the current Q value.

Further options:

Among them, α represents the learning rate, r represents the reward value obtained by the agent performing action a in state s, and γ represents the discount factor.

For a specific scheme, the specific steps of step 6 are as follows:

6.1. The agent selects actions according to the action selection mechanism;

6.2. When the agent completes the training times, the system selects the action sequence with the largest overall reward value to return, which is the desired optimal class integration test sequence.

Compared with the prior art, the present invention has the following beneficial effects:

The invention solves the problem that the existing method for generating a class integration test sequence based on reinforcement learning is not accurate enough to evaluate and determine the overall cost of the class integration test sequence, and provides more accurate testing work for testers in actual production and life. The measurement method improves the efficiency of integration testing, thereby better controlling the quality of the product.

Description of drawings

The accompanying drawings, as a part of the present invention, are used to provide further understanding of the present invention, and the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but do not constitute an improper limitation of the present invention. Obviously, the drawings in the following description are only some embodiments, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

In the attached image:

1 is a flowchart of a method for generating a class integration test sequence based on reinforcement learning according to an example of the present invention;

Figure 2 is a flowchart for defining reinforcement learning tasks;

Fig. 3 is the flow chart of program static analysis;

Fig. 4 is the flow chart of measuring test pile complexity;

Fig. 5 is the flow chart of designing reward function;

Fig. 6 is the flow chart of design value function;

Figure 7 is a flow chart of generating a class integration test sequence.

It should be noted that these drawings and written descriptions are not intended to limit the scope of the present invention in any way, but to illustrate the concept of the present invention to those skilled in the art by referring to specific embodiments.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and the following embodiments are used to illustrate the present invention , but are not intended to limit the scope of the present invention.

FIG. 1 is a flow chart of a method for generating a class integration test sequence based on reinforcement learning according to an example of the present invention.

S1 defines the reinforcement learning task. Reinforcement learning is theoretically based on the Markov decision process, that is, the conditional probability that the agent chooses an action to occur is only related to the current state of the office, but at the same time, the current action will affect the transition of the next state with a certain probability. The agent gets reward value based on state and action. Suppose there are n actions in the path action history. Under the given policy conditions, the goal of defining a reinforcement learning task is to find a set of optimal sequences that maximize the overall reward obtained.

S2 program static analysis. Perform static analysis on the source program to obtain a series of information, such as: inter-class dependency type, attribute dependency information (such as member variable dependency information, parameter transfer information and return value information, etc.) and method invocation information, etc., and use this information for Calculate property and method complexity between classes. The dependencies between classes can be divided into static dependencies between classes and dynamic dependencies between classes according to whether the program runs or not. When the program can be analyzed without running, it is called static dependency between classes, and the relationship formed when the program is running is called dynamic dependency between classes. The present invention is primarily concerned with static dependencies. The process of building test stubs needs to consider the strength of dependencies between classes, and measuring such dependencies requires analyzing the coupling information between classes, and then calculating the cost of building test stubs. After analysis, the tightness of the coupling degree is positively correlated with the strength of the dependency, that is, the coupling degree is positively correlated with the test cost.

S3 measures test pile complexity. A test stub is not a real class in a system, but a component module or a piece of target code that serves the object under test. When the dependency between the two classes is strong, the test pile needs to simulate more functions, the construction is more complicated, and the complexity of the test pile is relatively high; when the relationship is weak, the difficulty of constructing the test pile is reduced, and the cost Smaller, the test pile complexity is lower, so the test pile complexity can be calculated by the strength of the relationship between classes, and then the test cost can be obtained.

S4 designs the reward function. When the agent takes action a in state s, the environment will comprehensively give the agent a reward r. When the r obtained by the agent is positive, the agent will strengthen the choice of actions in the relevant direction, and will also affect the next state. The reward function is a function that calculates the reward value r.

S5 Design Value Function. The reward function is used to maximize the reward and punishment value obtained by the agent when it moves from one state to the next state. In order to ensure that the cumulative reward value is the largest when reaching the target state, the value function is designed to represent the cumulative reward value, and the Q table is used to store such the Q value. Assuming that the average reward value obtained after the first t actions and the reward value of the t+1st time are known, then the cumulative reward Q value after the t+1th selection action can be predicted and updated. The value function shows The process of predicting and updating feedback in reinforcement learning.

S6 generates class integration test sequences. Through the action selection and reward feedback of the agent in the learning process, a suitable action is selected to be added to the action sequence. In this process, the reward function is designed by measuring the complexity of building the test pile, and the value function is designed to ensure the cumulative reward of the action sequence. The value is also the largest, and the final selection of the appropriate action sequence is the optimal class integration test sequence obtained by this method.

Figure 2 is a flowchart for defining reinforcement learning tasks. By understanding the structure of reinforcement learning, combined with the relevant research on the class ensemble test sequence, a reinforcement learning strategy is formulated with the goal of reducing the complexity of the test pile as much as possible, and the optimal cumulative reward value of the action sequence selected by the agent is the learning goal. The specific steps are as follows: first, the software system to be analyzed is regarded as a set of classes that need to be integrated during testing; then, the action sequence performed by the agent in the path, that is, the action history, is retained as the integration test class sequence. Candidate solutions; finally, find an action history with the largest overall reward from the candidate solutions, which is the class integration test sequence required for this learning process.

Figure 3 is a flow chart of program static analysis. By analyzing the dependencies between classes in the program, the coupling degree of attributes and methods is obtained, which is prepared for the next step to calculate the complexity of test piles. The specific steps are as follows: first, analyze the relationship between classes by analyzing the specific statements between the programs; then, calculate the attribute coupling between classes through the attribute complexity, and calculate the method coupling between the classes through the method complexity; finally, in order to facilitate the next calculation, Normalize properties and method complexity separately.

FIG. 4 is a flow chart of measuring the complexity of test piles. The test pile complexity is an important indicator to measure the test cost, which is mainly calculated by the attributes and method complexity between classes. The specific steps are as follows: first, determine the weights of the attribute and method complexity, and use the entropy weight method to calculate the weight between the two classes; then, combine the standardized attributes and method complexity in the previous step to calculate the test pile complexity; finally, When the class integration test sequence is obtained, the complexity of the test piles generated in the process is accumulated to obtain the overall test pile complexity, so as to evaluate the method.

Embodiment: In order to obtain the test pile complexity more accurately, the entropy weight method is used to calculate the weights W _A and W _M of the attribute and method complexity.

The steps of calculating the complexity of the test pile by the entropy weight method are as follows:

(1) Standardized indicators

，方法复杂度是

。计算公式如下：There are two indicators for calculating the complexity of test piles, namely attribute complexity and method complexity, A(i, j) represents the attribute complexity between classes i and j, and M(i, j) represents the difference between classes i and j. The first step of the entropy weight method is to standardize two indicators, so that the obtained results are between 0 and 1. The attribute complexity between classes after standardization is

, the method complexity is

. Calculated as follows:

；

;

。

.

(2) Establish an evaluation matrix

Assuming that the system to be tested contains m classes, an m*2 matrix can be constructed to represent the two relationships between classes, in which the first column represents the evaluation value under the attribute complexity index, and the second column represents the method complexity. The evaluation value under the index of degree. The two columns together form an evaluation matrix R, and the calculation formula is as follows:

Calculate information entropy

Before calculating the information entropy, it is necessary to first calculate the proportion of the evaluation value of each class to each index, and use P _ij to represent the proportion of the jth index, and then calculate the information entropy e _j of the jth index according to the obtained proportion, where K is a constant, the formula is as follows:

Calculate weights

Denote the weight of the jth indicator as W _j , where j is 1, representing the attribute complexity weight, and when j is 2, representing the method complexity weight, the formula is as follows:

Finally, the weight of attribute complexity and method complexity is obtained, and then the test pile complexity SCplx(i, j) is obtained.

。

.

Figure 5 is a flowchart of designing a reward function. The reward function is an important indicator to guide the agent to explore. The agent tends to explore the action path with a larger reward value. Therefore, in order to obtain a class integration test sequence with the lowest possible test cost, this The invention improves the reward function in combination with the complexity of the test pile. The specific steps are as follows: First, design a reward function with a higher reward value when the agent explores and integrates into a better class; then, when any action class appears twice in the process, give this path a minimum value -∞, so that this situation can be avoided when continuing to explore; finally, by combining the above two points with the complexity of the test pile, the reward function is designed, and the training agent tends to explore the path with lower complexity of the test pile.

Implementation: Assuming that the agent has experienced n+1 states in total from the first state to the f-th state, and selected n actions, the f-th state s _f is considered to be the final state, from s ₁ to The formula of the state change function between s _f is shown below, where s' is the next state of state s.

表示从最初状态到最终状态s_f的状态路径，其中σ₀表示初始状态s₁，

。智能体按照该路径执行的n个动作序列即可以表示为

，即与状态路径对应的动作历史。如果该路径不包含重复动作，则可以认为

即为一条可供选择的类集成测试序列。use

represents the state path from the initial state to the final state s _f , where σ ₀ represents the initial state s ₁ ,

. The n action sequences performed by the agent according to the path can be expressed as

, that is, the action history corresponding to the state path. If the path does not contain repeated actions, it can be considered that

That is, a sequence of optional class integration tests.

The reward and punishment mechanism in reinforcement learning is the core of controlling the agent to explore the best path. The better the class that the agent explores and integrates into, the higher the reward value. In order to further reduce the overall test cost of the class integration test sequence, the present invention combines the complexity of the test pile to design a strengthening function, and the definition formula is as follows:

The agent reaches σ _i after i-1 state transitions, σ _i represents the state path, r(σ _i ) represents the reward value that the state path will get, and Max represents the maximum reward value, where 1000 is taken here, and c is a positive integer value , here is 100, a _σi represents the action history corresponding to the state path, and SCplx() represents the test pile complexity. When the agent does not meet the requirements in the process of exploration, the environment will give the agent a penalty value. For example, if any action class appears twice in the process, this path is given a minimum value of -∞, which can be avoided when continuing to explore later. When there is no duplicate class in the path, the path can be considered feasible, and the environment will give the agent a reward value, where c is a positive integer. When the final state is reached and no duplicate class appears, it can be considered that an alternative path has been found and its overall test cost is calculated. If the test cost of the currently obtained class integration test sequence is less than the test cost of all the previous sequences obtained, it can be given a higher reward value. Integrating according to the sequence of the path action history can minimize the overall test pile complexity of the generated class integration test sequence and save the test cost to the greatest extent.

；之后，再加上智能体在状态s下执行动作a得到的奖励值

并整体乘以一个学习率

；最后，加上刚才即时回报的Q值得到现在的Q值。Figure 6 is a flowchart of designing a value function. The value function pays more attention to the cumulative reward value obtained during the agent's exploration process, and can complement the reward function to jointly promote the agent to explore in a direction with lower testing costs. The specific steps are as follows: first, according to the state generated in the interaction of the environment and the selected action, obtain a Q value of immediate reward, which is represented by Q(s, a); then, select the largest Q(s according to the next state s'',a') and multiply it by a discount factor

; After that, add the reward value obtained by the agent performing action a in state s

and multiply by a learning rate as a whole

; Finally, add the Q value of the immediate return just now to the current Q value.

Calculated as follows:

表示学习率，

表示智能体在状态

下执行动作

得到的奖励值，

表示折扣因子。in,

represents the learning rate,

Indicates that the agent is in the state

perform action

the reward value received,

represents the discount factor.

7 is a flow chart of generating a class integration test sequence. The present invention adopts the reinforcement learning method to train the agent to learn in a direction with lower test cost, and the action sequence obtained by selecting actions in the process is the desired class integration test sequence. The specific steps are as follows: First, the agent selects actions according to the action selection mechanism; then, when the agent completes the training times, the system selects the action sequence with the largest overall reward value and returns, which is the optimal class integration test sequence.

Implementation: Reinforcement learning is a process of exploring and utilizing selection actions. In order to avoid falling into a local optimum during the learning process and further increase the proportion of agent exploration, based on the ε-greedy method, two selection mechanisms are adopted:

The traditional ε-greedy algorithm: select the action corresponding to the maximum Q value in the current state with the probability of 1-ε; randomly select the action with the probability of ε.

Dynamic adjustment ε algorithm: first dynamically adjust the value of ε, the formula is as follows, where times represents the number of training times.

。

.

Through the above Q-learning algorithm, the path σ of the agent from the initial to the final state is obtained. If all actions in all states are accessed, the Q value obtained by the agent reaches the optimal value. The action history associated with the state path is finally obtained, and its corresponding action sequence is the best class integration test sequence obtained by this method.

The experimental data of the above-mentioned reinforcement learning-based class ensemble test sequence generation method are given below:

The experimental objects are nine programs in the SIR website, including elevator, SPM, ATM, daisy, ANT, BCEL, DNS, email_spl and notepad_spl. The average of 30 experimental results of this method shows that for five of the nine programs, elevator, daisy, ANT, email_spl, and notepad_spl, the overall test cost is the lowest when using particle swarm algorithm and random algorithm to generate class integration test sequences. Compared with the stub complexity, the overall test stub complexity spent by this method to generate the optimal integration-like test sequence is the lowest, which is reduced by 39.4%, 33.3%, 7.6%, 37.9% and 17.8%, respectively.

To sum up, the present invention solves the problem that the existing reinforcement learning-based method for generating class integration test sequences is not accurate enough to evaluate and determine the overall cost of class integration test sequences, and not only can further improve the research in the field of class integration test sequence generation. , it can further reduce the test cost, provide more accurate measurement methods for testers to carry out testing work in actual production and life, improve the efficiency of software testing, and ensure the quality of software products.

In the description provided herein, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Furthermore, it will be understood by those skilled in the art that although some of the embodiments described herein include certain features contained in other embodiments and not others, that combinations of features of different embodiments are also intended to be within the scope of the present invention. within the scope of protection and form different embodiments. For example, in the above embodiments, those skilled in the art can use in a combined manner according to the known technical solutions and the technical problems to be solved by the present application.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention in any form. Although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Within the scope of the technical solution of the present invention, personnel can make some changes or modifications to equivalent examples of equivalent changes by using the above-mentioned technical content, but any content that does not depart from the technical solution of the present invention, according to the present invention. Any simple modifications, equivalent changes and modifications made to the above embodiments in the technical essence still fall within the scope of the solution of the present invention.

Claims (10)

1. A class integration test sequence generation method based on reinforcement learning is characterized in that:

step 1, defining a reinforcement learning task: the task of reinforcement learning is to make the intelligent agent continuously try in the environment, continuously adjust the strategy according to the obtained reward value, and finally generate a better strategy, and the intelligent agent can know what action should be executed under what state according to the strategy;

step 2, static analysis of the program: performing static analysis on a source program, using the acquired information for calculating the attribute and method complexity among classes, calculating the attribute coupling among the classes through the attribute complexity, and calculating the method coupling among the classes through the method complexity;

step 3, measuring the complexity of the test pile: calculating the complexity of the test pile according to the obtained attributes and method complexity, and providing information for the design of a reward function at the back;

step 4, designing a reward function: integrating the calculation of the complexity of the test pile into the design of a reward function, and guiding an intelligent agent to learn towards a direction with lower complexity of the test pile;

step 5, designing a value function: the value function is fed back through the reward function, and the maximization of the accumulated reward is ensured through the setting of the value function;

step 6, generating a class integration test sequence: and when the intelligent agent finishes the set training times, selecting the action path with the maximum integral reward value, namely the class integrated test sequence obtained by the study.

2. The class integration test sequence generation method based on reinforcement learning of claim 1, wherein:

the specific steps of step 1 are as follows:

1.1, regarding a software system to be analyzed as a set of classes to be integrated during testing;

1.2, an action sequence, namely an action history, executed by the agent in the path is reserved and used as a candidate solution of the integrated test class sequence;

1.3, finding an action history with the maximum overall reward from the candidate solutions, namely, finding a class integration test sequence required by the learning process.

3. The class integration test sequence generation method based on reinforcement learning of claim 1, wherein:

the specific steps of step 2 are as follows:

2.1, analyzing the relationship among classes, calculating the attribute coupling among the classes through the attribute complexity, using A (i, j) to represent, wherein i, j respectively represent the classes in the program, and the attribute complexity is numerically equal to the sum of the number of member variables of i call j, the type of the method parameter and the number of return values of the method;

2.2, calculating the method coupling between classes through the method complexity, using M (i, j) to represent, wherein the method complexity is equal to the number of methods in i call j in number, and then standardizing the attribute and the method complexity.

4. The class integration test sequence generation method based on reinforcement learning of claim 1, wherein:

the specific steps of step 3 are as follows:

3.1, calculating the weight of the attribute and the method complexity between classes by an entropy weight method;

3.2, calculating to obtain the complexity of the test pile by combining the attribute and the method complexity;

and 3.3, accumulating the complexity of the test pile generated in the process when the class integration test sequence is obtained, and obtaining the complexity of the total test pile.

5. The class integration test sequence generation method based on reinforcement learning of claim 4, wherein:

test pile complexity

Wherein,

，

a (i, j) represents the attribute complexity between classes i and j, M (i, j) represents the method complexity between classes i and j, the entropy weight method firstly standardizes the continuous indexes, the obtained results are all between 0 and 1, and the attribute complexity between the classes after standardization is

The complexity of the method is

。

6. The class integration test sequence generation method based on reinforcement learning of claim 1, wherein:

the specific steps of step 4 are as follows:

4.1, designing a reward function with a higher reward value when the class explored and integrated by the agent is more optimal;

4.2 when any action class in the process is repeated twice, giving a minimum value to the path

So as to avoid when continuing to explore;

4.3, by combining the complexity of the test pile and designing a reward function, the training agent tends to explore a path with lower complexity of the test pile.

7. The class integration test sequence generation method based on reinforcement learning of claim 6, wherein:

wherein the agent reaches sigma through i-1 state transitions_i，σ_iRepresents a state path, r (σ)_i) Denotes the prize value that the status path will receive, Max denotes the maximum prize value, here 1000, c is a positive integer value, here 100, a_σiRepresenting the action history corresponding to the state path, SCplx () represents the test stub complexity.

8. The class integration test sequence generation method based on reinforcement learning of claim 1, wherein:

the specific steps of step 5 are as follows:

5.1, obtaining a Q value which is reported immediately according to the state and the selected action generated by the interaction of the environment, wherein the Q value is represented by Q (s, a), s represents the state, and a represents the action;

5.2, selecting the largest Q (s ', a ') according to the next state s ' and multiplying it by a discount factor gamma;

5.3, adding the reward value r obtained by the agent executing the action a in the state s;

5.4, multiplying the whole by a learning rate alpha;

5.5, plus the value of Q just reported immediately, the current value of Q is obtained.

9. The class integration test sequence generation method based on reinforcement learning of claim 8, wherein:

where α represents the learning rate, r represents the reward value resulting from the agent performing action a in state s, and γ represents the discount factor.

10. The class integration test sequence generation method based on reinforcement learning of claim 1, wherein:

the specific steps of step 6 are as follows:

6.1, the intelligent agent selects the action according to an action selection mechanism;

6.2, when the intelligent agent finishes training times, the system selects the action sequence with the maximum integral reward value to return, and the action sequence is the required optimal class integration test sequence.

Images