CN113672508B - Simulink testing method based on risk strategy and diversity strategy - Google Patents

Abstract

The application discloses a Simulink testing method based on a risk strategy and a diversity strategy, which comprises the following specific steps: collecting test cases; processing the generated test cases: counting the names and the times of occurrence of the Simulink modules in each test case; modeling keyword vectors: forming a keyword dictionary by using the Simulink module names appearing in all the test cases, recording the occurrence times of each keyword, and deleting the keywords with occurrence times lower than a set threshold value; constructing a keyword matrix KV, a risk vector RV and a distance matrix DM; the method comprises the steps of prioritizing Simulink test cases by using a risk strategy Dan based on a risk vector RV, a diversity strategy Var based on a distance matrix DM and a diversity risk mixing strategy VarDan combining the two strategies.

Description

A Simulink testing method based on risk strategy and diversity strategy

Technical field

The invention relates to the field of test case selection and sorting, and in particular to a Simulink testing method based on risk strategy and diversity strategy.

Background technique

Chips are called "industrial food" and are the core technology of the manufacturing industry. As the country continues to attach importance to the chip industry, the chip design industry is booming. EDA is a necessary and most important software tool for chip design, so how to achieve efficient and stable EDA is crucial. Simulink is a widely used circuit design EDA, and people have an increasingly urgent need for it to execute normally and achieve expected results.

In recent years, several methods have been proposed to facilitate automated Simulink testing. These techniques rely on some test case generation tools (such as SLforge) to generate a large number of test programs, and detect Simulink errors by running these generated test programs. Common methods include using LSTM to learn test cases generated by SLforge based on rules, and generating new test cases in batches. These test cases generated based on learning are more random than test cases generated based on rules, and are more likely to find Simulink errors. There are also some studies that perform SLEMI mutation on existing Simulink models and generate new test cases by processing zombie blocks in test cases. The specific approach is to first obtain the coverage information of each block through preprocessing and identify nested dynamic zombie blocks. There are three ways to deal with different situations: (1) For nested zombie blocks, delete the block and connect the front and back of the block. (2) Replace the top zombie blocks and ensure that the replaced blocks have the same sampling time. (3) Otherwise, perform mutation on the activity level. Finally, Simulink errors are detected by differential testing on the test cases before mutation and the newly generated test cases. This Simulink testing technology has serious efficiency problems, because they usually need to generate a large number of Simulink models and test them continuously, so it takes a long time to find Simulink errors. Although it only needs to perform EMI mutation on existing test cases to find Simulink errors, the mutation method is relatively simple and cannot find more diverse error types.

Contents of the invention

According to the problems existing in the existing technology, the present invention discloses a Simulink testing method based on risk strategy and diversity strategy, which specifically includes the following steps:

Collect test cases, use the Simulink program random generation tool Slforge to generate test cases, and use the error detection program to determine whether Simulink crashes. If a Simulink error is found, it will be marked as fault;

Process the generated test cases: count the Simulink module names that appear in each test case and the number of times they appear;

Model keyword vectors: Construct a keyword dictionary of Simulink module names that appear in all test cases, record the number of occurrences of each keyword, and delete keywords whose occurrences are lower than the set threshold;

Construct keyword matrix KV, risk vector RV and distance matrix DM;

Prioritize test cases: Use risk strategy Dan based on risk vector RV, diversity strategy Var based on distance matrix DM, and diversity risk hybrid strategy VarDan that combines the above two strategies to prioritize Simulink test cases .

When constructing the keyword matrix KV as described:

Construct a keyword vector tr _i =(e _i,1 ,e _i,2 ,...,e _i,m ) for each test case, where m is the number of keywords in the keyword dictionary; if If the i test case contains the jth keyword in the keyword dictionary, then e _i,j =1, otherwise e _i,j =0, KV is an n×m matrix, n means there are n test cases, m means There are m keywords in the keyword dictionary.

When constructing the risk vector RV as described above: count the number of 1's in the keyword vector tr _i , and use it as the risk value in the test case, using Indicates that by calculating the risk value of each test case, a risk vector RV formed by n error reports is constructed.

When constructing the distance matrix DM: Calculate the distance of each pair of test cases based on the keyword matrix KV. For the two keyword vectors tr _i and tr _k , the corresponding values of e _{i, j} and e _{k, j} in the corresponding position j are The number is counted as distance D(tr _i , tr _k ), and the distance matrix DM is constructed by calculating the distance of each pair of test cases.

When using the risk strategy Dan to prioritize Simulink test cases: select the test case tr _i with the highest risk value RV(i), put it into the ordered queue QTR, delete the test case from tr, and then delete it from tr Select the test case tr _i with the highest risk value RV(i) among the remaining test cases in tr, and repeat the above operation until the length of tr is 0.

When using the diversity strategy Var to prioritize Simulink test cases: select the test case tr _i with the highest risk value and put it into QTR, delete it from tr, and then select the test case tr i that is the furthest from tr from QTR, that is, D( tr, QTR) The largest test case tr _j is put into QTR, and it is also deleted from tr. Repeat the above operation until the TR length reaches 0.

When using the diversity risk mixture strategy VarDan to prioritize Simulink test cases: select the test case with the largest risk value for inspection, put it into QTR and delete it from tr, by selecting D(tr, QTR in tr and QTR ) the largest n _c untested cases to build the candidate set CTR, select the test case tr _j with the highest risk in CTR for inspection, add it to the QTR queue, and delete it from tr if the test case is marked as fault, and δ>0, the keyword vector KV will be updated, the risk value RV will also be updated, and then the above operation will be repeated until the length of tr is 0.

Due to the adoption of the above technical solution, the present invention provides a Simulink testing method based on risk strategy and diversity strategy. This method utilizes two key strategies: diversity strategy Var to help check various test cases and avoid Test duplication error classification; risk strategy Dan helps identify test cases that are more likely to reveal errors. By combining both strategies, you can catch as many errors as possible as early as possible.

Description of the drawings

In order to explain the embodiments of the present application or the technical solutions in the prior art more clearly, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are only These are some embodiments recorded in this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting creative efforts.

Figure 1 is a flow chart of the method in the present invention

Figure 2 is a schematic diagram of the Simulink model file in the present invention

Figure 3 is a schematic diagram of the Simulink model in the present invention

Figure 4 is a schematic diagram of common Simulink modules in the present invention.

Figure 5 is a schematic diagram of an example keyword dictionary in the present invention.

Figure 6 is a schematic diagram of an example keyword matrix in the present invention.

Figure 7 is a schematic diagram of a distance matrix in an example of the present invention.

Detailed ways

In order to make the technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described below in conjunction with the drawings in the embodiments of the present invention:

As shown in Figure 1, a Simulink testing method based on risk strategy and diversity strategy specifically includes the following steps:

Collect test cases and use the Simulink program random generation tool Slforge to generate test cases. Figure 2 shows the generated Simulink model file, and Figure 3 shows the corresponding Simulink model. Use the error detection program to determine whether Simulink crashes. If a Simulink error is found, it will be marked as "fault";

Process the generated test cases: Simulink test cases record the modules that appear in the corresponding Simulink model and the connection relationships between modules, and count the Simulink module names that appear in each test case and the number of times they appear; as shown in the figure Shown in 4 is a common Simulink module.

Model keyword vectors: Construct a keyword dictionary of Simulink module names that appear in all test cases, record the number of occurrences of each keyword, and remove some keywords whose occurrences are lower than the threshold ε by setting a threshold ε , as shown in Figure 5;

Construct keyword matrix KV: Construct a keyword vector tr _i = (e _i,1 , e _i,2 ,..., e _i,m ) for each test case, where m is the keyword in the keyword dictionary number. If the i-th test case contains the j-th keyword in the keyword dictionary, then e _i,j = 1, otherwise e _i,j = 0; KV is an n×m matrix, n means there are n test cases, m indicates that there are m keywords in the keyword dictionary, as shown in Figure 6;

Construct the risk vector RV: count the number of "1" in the keyword vector tr _i , and use it as the risk value in the test case, using Indicates that by calculating the risk value of each test case, a risk vector RV formed by n error reports is constructed;

Construct the distance matrix DM: Calculate the distance of each pair of test cases based on the keyword matrix KV. For the two keyword vectors tr _i and tr _k , count the number of different values of e _i,j and e _k,j in the corresponding position j is the distance D(tr _i ,tr _k ). By calculating the distance of each pair of test cases, the distance matrix DM is constructed, as shown in Figure 7;

Prioritize test cases: Use risk strategy Dan based on risk vector RV, diversity strategy Var based on distance matrix DM, and diversity risk hybrid strategy VarDan that combines the above two strategies to prioritize Simulink test cases .

Further, the risk strategy Dan method is: each time, the test case tr _i with the highest risk value RV(i) will be selected, put into the ordered queue QTR, and the test case will be deleted from tr. Then select the test case tr _i with the highest risk value RV(i) from the remaining test cases in tr, and repeat the above operation until the length of tr is 0;

Adopt dynamic prioritization strategies based on risk vector RV and inspection results. That is, if the test case tr _k is determined to be an error, then the weights of all keywords of tr _k in KV will increase by δ, and the risk vector RV will also change accordingly;

The diversity strategy Var method is: first select the test case tr _i with the highest risk value and put it into QTR, delete it from tr, and then select the test from tr that is farthest from QTR, that is, the test with the largest D(tr, QTR) Put the use case tr _j into QTR, delete it from tr, and repeat the above operation until the TR length reaches 0;

Further, the diversity risk mixed strategy VarDan method is to combine the risk strategy and the diversity strategy into a mixed strategy. Select the test case with the largest risk value for inspection, put it into QTR and delete it from tr, build the candidate set CTR by selecting the top n _c untested cases with the largest D(tr, QTR) in tr and QTR, select The test case tr _j with the highest risk in CTR is checked, added to the QTR queue, and deleted from tr. If the test case is marked as "fault" and δ>0, the keyword vector KV will be updated, and the risk The value RV will also be updated, and then the above operation will be repeated until the length of tr is 0;

The above are only preferred specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person familiar with the technical field can, within the technical scope disclosed in the present invention, implement the technical solutions of the present invention. Equivalent substitutions or changes of the inventive concept thereof shall be included in the protection scope of the present invention.

Claims (7)

1. A Simulink testing method based on a risk strategy and a diversity strategy is characterized by comprising the following steps:

collecting test cases, adopting a Simulink program random generation tool Slforce to generate test cases, judging whether Simulink crashes or not through an error detection program, and marking the Simulink crashes as fault if the Simulink errors are found;

processing the generated test cases: counting the names and the times of occurrence of the Simulink modules in each test case;

modeling keyword vectors: forming a keyword dictionary by using the Simulink module names appearing in all the test cases, recording the occurrence times of each keyword, and deleting the keywords with occurrence times lower than a set threshold value;

constructing a keyword matrix KV, a risk vector RV and a distance matrix DM;

prioritizing the test cases: the method comprises the steps of prioritizing Simulink test cases by using a risk strategy Dan based on a risk vector RV, a diversity strategy Var based on a distance matrix DM and a diversity risk mixing strategy VarDan combining the two strategies.

2. The method according to claim 1, characterized in that: when the key word matrix KV is constructed: building a keyword vector tr for each test case _i ＝(e _i,1 ,e _i,2 ,...,e _i,m ) Wherein m is the number of keywords in the keyword dictionary; e if the ith test case contains the jth keyword in the keyword dictionary _i,j =1, otherwise e _i,j =0, kv is an n×m matrix, n represents that there are n test cases, and m represents that there are m keywords in the keyword dictionary.

3. The method according to claim 1, characterized in that: the risk vector RV is constructed by: statistical keyword vector tr _i 1 in the test case and using the number of 1 as a risk value in the test caseThe risk value of each test case is calculated, so that a risk vector RV formed by n error reports is constructed.

4. The method according to claim 2, characterized in that: the distance matrix DM is constructed by: calculating the distance between each pair of test cases based on the keyword matrix KV, and for two keyword vectors tr _i And tr _k Will correspond to e in position j _i,j And e _k,j The number of different values is the distance D (tr _i ,tr _k ) Through meterThe distance of each pair of test cases is calculated to construct a distance matrix DM.

5. The method according to claim 1, characterized in that: when the risk policy Dan is adopted to prioritize and sort the Simulink test cases: selecting the test case tr with the highest risk value RV (i) _i Putting the test cases into an ordered queue QTR, deleting the test cases from tr, and selecting the test case tr with the highest risk value RV (i) from the rest test cases of tr _i The above operation is repeated until tr is 0.

6. The method according to claim 1, characterized in that: when the diversified strategy Var is adopted to prioritize and sort the Simulink test cases: selecting the test case tr with the highest risk value _i Put into QTR, delete it from tr, and select the test case tr with the farthest distance from QTR, i.e. the largest D (tr, QTR), from tr _j Put into QTR, it is also deleted from TR, and the above operation is repeated until TR length is 0.

7. The method according to claim 1, characterized in that: when the diversified risk mixing strategy VarDan is adopted to prioritize and sort the Simulink test cases: selecting test case with maximum risk value, checking, putting it into QTR, deleting it from tr, selecting tr and the first n with maximum D (tr, QTR) in QTR _c Constructing a candidate set CTR by using the non-tested cases, and selecting the test case tr with the largest risk in the CTR _j Checking, adding it to the QTR queue and deleting it from tr if the test case is marked as fault and delta>0, the keyword vector KV will be updated, the risk value RV will be updated, and then the above operations are repeated until the tr length is 0.