CN111367822A - Regression testing method and device based on finite state machine - Google Patents

Abstract

The invention relates to the field of system test, in particular to a regression test method and a device based on a finite-state machine, which comprises the following steps: inputting an old model, an original test suite and a new model of a system to be tested, wherein the old model, the new model M' and the test suite comprise a coverage set and a state recognizer; comparing the old model with the new model, and analyzing the change of the model to obtain a model change result; analyzing the change of the test suite according to the model change result to obtain a test suite change result; combining the changed test kits according to the test kit change result to generate a new test kit; a regression test was performed. The invention respectively considers the changes of the coverage set and the state recognizer in the test suite, thereby generating a new test suite, avoiding complete regeneration, selectively executing test cases, reducing test execution cost and greatly improving the efficiency of the regression test of the finite state machine.

Description

Regression testing method and device based on finite state machine

Technical Field

The invention relates to the field of system testing, in particular to a regression testing method and device based on a finite-state machine.

Background

Software systems need to evolve continuously to accommodate the changing demands. Both the specification and implementation of the system are changing to repair defects, improve functions, add new functions, or delete existing functions. Each time the software system is modified, regression testing is required to be performed to ensure that the original, changed and newly added components have the correct behavior.

Model-based testing is an important branch of test automation methods, and is used to model system specifications in a formal expression language, and then automatically generate test cases and their expected outputs from the system specifications to achieve the purpose of reducing test cost and error risk, and a Finite State Machine (FSM) is one important model. FSM-based test methods have been widely used in the fields of sequential circuits, lexical analysis, software design, communication protocols, object-oriented systems, Web services, and the like.

The model of the software system will also evolve continuously to reflect the change in the specification, and although the test cases can be automatically generated from the model, it is not advisable to regenerate and execute all the test cases every time the system changes, and an efficient method is still needed to reduce the overhead of the regression test. However, many existing methods are white-box, and the regression testing problem is considered from the code coverage level, and the model-based regression testing method is still limited, and particularly, there is no method suitable for FSM.

In the conventional regression testing method, the relation between the test case and each part of the system is tracked, and most of the test cases are considered to correspond to the test case only by looking at which part of the system the test case is executed to. However, the test case of the FSM contains not only the overlay set that covers the various parts of the model itself, but also a state identifier to verify that these parts are correct. When the model changes, the light coverage set does not change, and the state recognizer also can change, so that the existing regression testing method cannot be applied to the FSM and deviation of testing accuracy can occur.

Disclosure of Invention

The technical problem to be solved by the invention is to provide a regression testing method and device based on a finite-state machine, and the technical scheme solves the problem that the existing regression testing method only considers the change of a coverage set and does not consider the change of a state recognizer and cannot be suitable for the finite-state machine. The method and the device respectively consider the changes of the coverage set and the state recognizer in the test suite, so that a new test suite is generated, complete regeneration is avoided, test cases can be selectively executed, test execution cost is reduced, and the efficiency of the regression test of the finite state machine is greatly improved.

In order to solve the technical problems, the invention provides the following technical scheme:

the regression testing method based on the finite-state machine comprises the following steps:

inputting an old model, an original test suite and a new model of a system to be tested, wherein the old model M = (S, S)₀X, Y, δ, λ), new model M '= (S', S)₀', X ', Y ', delta ', lambda '), the test kit includes an overlay set including a state overlay set Q and a migration overlay set P, and a state recognizer including a feature set W and an identity set W for each state_i；

Comparing the old model with the new model, and analyzing the change of the model to obtain a model change result;

analyzing the change of the test suite according to the model change result to obtain a test suite change result;

combining the changed test kits according to the test kit change result to generate a new test kit;

a regression test was performed.

Optionally, analyzing the change of the test suite according to the model change result to obtain a test suite change result, which specifically includes:

analyzing the change of the coverage set according to the model change result, and segmenting the state coverage set Q ' and the migration coverage set P ' of the M ' to obtain a coverage set change result;

analyzing the change of the recognition state machine according to the model change result to generate a feature set W 'of M' and a recognition set W of each state_i', obtaining a state identifier change result.

Optionally, analyzing the change of the coverage set according to the model change result, and segmenting the state coverage set Q ' and the migration coverage set P ' of M ' to obtain a coverage set change result, which specifically includes:

generating a state coverage set Q ' and a migration coverage set P ' of M ';

classifying the transition sequence corresponding to the left state in Q' into the unchanged part Q of the state coverage set Q'^UTransition sequences corresponding to newly added states are attributed to different portions Q 'of the state overlay set Q'^D；

Attributing a transfer sequence corresponding to the unchanged migration in the P ' to an unchanged part P ' of the migration overlay set P '^UThe transition sequences corresponding to the newly added transitions and the changed transitions are attributed to different portions P 'of the migration overlay set P'^D；

Calculating a part R '= P' -Q 'not included in the state coverage set Q' in the migration coverage set P ', wherein R'^U=P'^U-Q'^U，R'^D=P'^D-Q'^D；

Analyzing the change of the recognition state machine according to the model change result to generate a feature set W 'of M' and a recognition set W of each state_i' obtaining a state identifier change result, specifically comprising:

for each newly added state S ∈ S ' -S and any other state t ∈ S ', a separate sequence SS ' (S, t) is generated and attributed to a different portion W ' of the corresponding state identifier '^D, W_s'^D, W_t'^D；

For each pair of left-behind states S, t ∈ S ∩ S ', it is determined whether the separation sequence SS (S, t) is still valid in M', and if so, the SS (S, t) is included in the unchanged portion W 'of the corresponding state identifier'^U, W_s'^U, W_t'^UIf not, a new separation sequence SS ' (s, t) is generated and included in a different part W ' of the corresponding state identifier '^D, W_s'^D, W_t'^D；

Different parts W 'of the combined status identifier'^D, W_i'^DAnd an unchanged portion W 'of the state identifier'^U, W_i'^UGenerating a feature set W ' = W ' of M '^U∪W'^DAnd an identification set W for each state_i'= W_i'^U∪W_i'^D。

Optionally, according to a test suite change result, combining the changed test suites to generate a new test suite, which specifically includes:

obtaining reusable test cases TS^R= r⋅Q'^U⋅W'^U⋃ r⋅R'^U⋅W_i'^U；

Obtaining a complementary test case TS^C= r⋅Q'^U⋅W'^D⋃ r⋅R'^U⋅W_i'^D；

Obtaining newly added test case TS^N= r⋅Q'^D⋅W'⋃ r⋅R'^D⋅W_i'；

Combining the reusable test cases, the supplementary test cases and the newly added test cases to form a new test suite TS' = TS^R⋃ TS^C⋃ TS^N。

Optionally, performing a regression test specifically includes:

executing the test of the newly added test case;

a test of the supplemental test case is performed.

There is also provided a regression test apparatus based on a finite state machine, comprising:

an input module for inputting an old model, an original test suite and a new model of the system to be tested, wherein the old model M = (S, S)₀X, Y, δ, λ), new model M '= (S', S)₀', X ', Y ', delta ', lambda '), the test suite includes, but is not limited to, an overlay set including a state overlay set Q and a migration overlay set P, and a state recognizer including a feature set W and an identity set W for each state_i；

The model analysis module is used for comparing the old model with the new model and analyzing the change of the model to obtain a model change result;

the test suite analysis module is used for analyzing the change of the test suite according to the model change result to obtain a test suite change result;

the kit generating module is used for combining the changed test kits according to the test kit change result to generate a new test kit;

and the test module is used for performing regression test.

Optionally, the test kit analysis module specifically includes:

the coverage set analysis module is used for analyzing the change of the coverage set according to the model change result, and segmenting the state coverage set Q ' and the migration coverage set P ' of the M ' to obtain a coverage set change result;

the state recognizer analysis module is used for analyzing and recognizing the change of the state recognizer according to the model change result to generate a feature set W 'of M' and an identification set W of each state_i', obtaining a state identifier change result.

Optionally, the overlay set analysis module specifically includes:

the first generation module is used for generating a state coverage set Q ' and a migration coverage set P ' of the M ';

a first dividing module for classifying the transition sequence corresponding to the left state in Q ' into the unchanged part Q ' of the state coverage set Q '^UTransition sequences corresponding to newly added states are attributed to different portions Q 'of the state overlay set Q'^D；

A second dividing module, configured to attribute a transition sequence corresponding to the unchanged transition in P ' to an unchanged portion P ' of the transition overlay set P '^UThe transition sequences corresponding to the newly added transitions and the changed transitions are attributed to different portions P 'of the migration overlay set P'^D；

A calculation module for calculating a portion R '= P' -Q 'in the migration overlay set P' that is not comprised by the state overlay set Q ', wherein R'^U=P'^U-Q'^U，R'^D=P'^D-Q'^D。

The state recognizer analysis module specifically comprises:

a placement module for generating a separate sequence SS ' (S, t) for each newly added state S ∈ S ' -S, any other state t ∈ S ', and attributing to a different portion W ' of the corresponding state identifier '^D, W_s'^D, W_t'^D；

A judging module for judging whether the separation sequence SS (S, t) is still valid in M 'for each pair of the left states S, t ∈ S ∩ S', and if so, attributing the SS (S, t) to the unchanged part W 'of the corresponding state identifier'^U, W_s'^U, W_t'^UIf not, generating a new separation sequence SS' (s, t) and including it in the corresponding state identifierDifferent fractions of W'^D, W_s'^D, W_t'^D；

A second generating module for combining different portions W 'of the status identifier'^D, W_i'^DAnd an unchanged portion W 'of the state identifier'^U, W_i'^UGenerating a feature set W ' = W ' of M '^U∪W'^DAnd an identification set W for each state_i'= W_i'^U∪W_i'^D。

Optionally, the kit generating module specifically includes:

a first obtaining module for obtaining reusable test case TS^R= r⋅Q'^U⋅W'^U⋃ r⋅R'^U⋅W_i'^U；

A second obtaining module for obtaining the supplemented test case TS^C= r⋅Q'^U⋅W'^D⋃ r⋅R'^U⋅W_i'^D；

A third obtaining module for obtaining the newly added test case TS^N= r⋅Q'^D⋅W'⋃ r⋅R'^D⋅W_i'；

A combination module for combining the reusable test cases, the supplemented test cases and the newly added test cases to form a new test suite TS' = TS^R⋃ TS^C⋃ TS^N。

Optionally, performing a regression test specifically includes:

the new case testing module is used for executing the test of the new test case;

and the supplementary case testing module is used for executing the test of the supplementary test case.

Compared with the prior art, the invention has the beneficial effects that:

the changes of the coverage set and the state recognizer in the test suite are respectively considered, then the changes of the coverage set and the state recognizer are combined to obtain the changes of the test suite, a new test suite is generated on the basis of the original test suite, complete regeneration is avoided, extra information is acquired, which test cases in the test suite are newly increased and reused are known, and then the test cases can be selectively executed, the test execution cost is reduced by avoiding executing the reused test cases, and the efficiency of FSM regression test is greatly improved on the whole.

Drawings

FIG. 1 is a schematic diagram of the process of the present invention;

FIG. 2 is a schematic diagram of the embodiment of S300) of the method of the present invention;

FIG. 3 is a diagram illustrating the details of S310) in the method of the present invention;

FIG. 4 is a schematic diagram of a step S320) of the method of the present invention;

FIG. 5 is a schematic diagram of S400) in the method of the present invention;

FIG. 6 is a diagram showing details of S500) in the method of the present invention

FIG. 7 is a block diagram of the apparatus of the present invention;

FIG. 8 is a block diagram of a specific structure of an analysis module of a test kit in the apparatus of the present invention;

FIG. 9 is a block diagram of a detailed structure of an overlay set analysis module in the apparatus of the present invention;

FIG. 10 is a block diagram of a specific structure of a status identifier analysis module in the apparatus according to the present invention;

FIG. 11 is a block diagram of a specific structure of a component generation module in the apparatus of the present invention;

FIG. 12 is a block diagram of a test module of the apparatus of the present invention;

FIG. 13 is a schematic flow chart of the present invention;

FIG. 14 is a model schematic of a gate that the control section used in the present invention can be modeled as a FSM;

FIG. 15 is a schematic representation of the model of FIG. 14 after modification by the present invention;

FIG. 16 is a diagram illustrating the ratio of the test cases reused after the test of the present invention to the number of all the original test cases;

FIG. 17 is a schematic diagram showing the length ratios between the test cases to be executed after the test of the present invention and all the test cases;

FIG. 18 is a schematic diagram showing the ratio of time required to regenerate the test kit after the test of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "front", "rear", "vertical", "horizontal", "inner", "outer", and the like indicate orientations or positional relationships based on those shown in the drawings, and are only for convenience of description and simplicity of description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.

As a preferred embodiment of the present invention, the present invention provides a regression testing method based on a finite state machine, comprising the steps of:

inputting an old model, an original test suite and a new model of a system to be tested, wherein the old model M = (S, S)₀X, Y, δ, λ), new model M '= (S', S)₀', X ', Y ', δ ', λ '), and the test kit includes a coverage set: state coverage set Q, migration coverage set P, state identifier: feature set W, recognition set W for each state_i；

Comparing the old model with the new model, and analyzing the change of the model to obtain a model change result;

analyzing the change of the test suite according to the model change result to obtain a test suite change result;

combining the changed test kits according to the test kit change result to generate a new test kit;

a regression test was performed.

The invention also provides a regression testing device based on the finite-state machine, which comprises:

an input module for inputting an old model, an original test suite and a new model of the system to be tested, wherein the old model M = (S, S)₀X, Y, δ, λ), new model M '= (S', S)₀', X ', Y ', delta ', lambda '), the test kit includes an overlay set including a state overlay set Q and a migration overlay set P, and a state recognizer including a feature set W and an identity set W for each state_i；

The model analysis module is used for comparing the old model with the new model and analyzing the change of the model to obtain a model change result;

the test suite analysis module is used for analyzing the change of the test suite according to the model change result to obtain a test suite change result;

the kit generating module is used for combining the changed test kits according to the test kit change result to generate a new test kit;

and the test module is used for performing regression test.

Through the method and the device design, the changes of the coverage set and the state recognizer in the test suite are respectively considered, then the changes of the test suite are combined to obtain the changes of the test suite, a new test suite is generated on the basis of the original test suite, the test suite is prevented from being completely regenerated, extra information is obtained, which test cases are newly added and reused are known, and then the test cases can be selectively executed, the test execution cost is reduced by avoiding executing the reused test cases, and the efficiency of the FSM regression test is greatly improved on the whole.

The preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

Referring to fig. 1 and 13, the method includes the following steps:

s100) inputting an old model, an original test suite and a new model of the system to be tested. In the present embodiment, the old model M = (S, S)₀X, Y, δ, λ); new model M '= (S', S)₀', X', Y', δ', λ')。The test kit comprises an overlay set and a state recognizer, wherein the overlay set comprises a state overlay set Q and a migration overlay set P, and the state recognizer comprises a feature set W and an identification set W of each state_i。

S200) comparing the old model with the new model, and analyzing the change of the model to obtain a model change result. In the present embodiment, the mentioned model changes include changes in state and transitions.

The state S is identified by name, and its change can be directly found by the set operation, where the state in S-S ' is removed, the state in S ' -S is added, and the state in S ∩ S ' is left.

The transitions S-X/y → t are identified by the start state S and the input X, and can be found by the collective operation considering the change of the output y and the end state t. the input sign can be found as well, i.e. the removed input X-X ', the newly added input X ' -X, the left-over input X ∩ X '. the removed transitions include the transitions corresponding to all inputs X ∈ 1X ' at the removed state S ∈ S-S ', the transitions corresponding to the removed inputs X ∈ X-X ' at all states S ∈ S ', the newly added transitions include the transitions corresponding to all inputs X ∈ X ' at the newly added state S ∈ S ' -S, the transitions corresponding to all inputs X ∈ X ' -X at all states S ∈ S ', the left-over transitions refer to the transitions corresponding to the left-over state S ∈ S ∈ 0S ', the transitions corresponding to the left-over inputs X ∈ X ∈ 2X ' at all states S ∈ S ', and can be further divided into the transitions of changes, the output or end state is different from the old model, i.e. the old model (S, S δ S) = (S, X ≠ S, S ≠ S ' (i.e. the new model), and the new model (S, i.e. the old model is not identical to the new model).

S300) analyzing the change of the test suite according to the model change result to obtain a test suite change result.

It can be understood that, in this embodiment, please refer to fig. 2, S300) analyzing the variation of the test kit may specifically include:

s310) analyzing the change of the state coverage set and the migration coverage set according to the model change result, and segmenting the state coverage set Q ' and the migration coverage set P ' of the M ' to obtain a coverage set change result. The coverage set of the new model is divided into two parts: the unchanged part, denoted by superscript U; the different parts, denoted by the superscript D, see fig. 3, may specifically include:

s311) generates a state coverage set Q ' and a migration coverage set P ' of M '.

S312) classifying transition sequences corresponding to the states left in Q ' into an unchanged part Q ' of a state coverage set Q '^UTransition sequences corresponding to newly added states are attributed to different portions Q 'of the state overlay set Q'^D。

S313) assigning the transition sequence corresponding to the unchanged transition in P ' to the unchanged part P ' of the transition overlay set P '^UThe transition sequences corresponding to the newly added transitions and the changed transitions are attributed to different portions P 'of the migration overlay set P'^D。

S314) calculating a portion R '= P' -Q 'not included in the state coverage set Q' in the migration coverage set P ', where R ='^U=P'^U-Q'^U，R'^D=P'^D-Q'^D。

S320) analyzing the change of the state recognizer according to the model change result to generate a feature set W 'of M' and an identification set W of each state_i', obtaining a state identifier change result. Likewise, the state identifier of the new model is also divided into an unchanged portion and a different portion. Since the feature set and the identification set are composed of separate sequences between different states, that is, different input sequences are output thereon, which are denoted as SS (s, t), and a change in the separate sequences between the states is found, a change in the state identifier is found, referring to fig. 4, which may specifically include:

s321) for each newly added state S ∈ S ' -S, whose separation sequence from any other state t ∈ S ' does not yet exist, a separation sequence SS ' (S, t) is generated and included in a different portion W ' of the corresponding state identifier '^D, W_s'^D, W_t'^D。

S322) for each pair of left-behind states S, t ∈ S ∩ S ', the original feature set W contains their separate sequence SS (S, t), and it is determined whether the sequence is still valid in M'Divide W'^U, W_s'^U, W_t'^U(ii) a Otherwise, a new separation sequence SS ' (s, t) is generated and included in a different portion W ' of the corresponding state identifier '^D, W_s'^D, W_t'^D。

S323) different parts W 'of the combined status identifier'^D, W_i'^DAnd an unchanged portion W 'of the state identifier'^U, W_i'^UGenerating a feature set W ' = W ' of M '^U∪W'^DAnd an identification set W for each state_i'= W_i'^U∪W_i'^D。

S400) combining the changed test kits according to the test kit change result to generate a new test kit, please refer to fig. 5, which specifically includes:

s401) obtaining reusable test case TS^R= r⋅Q'^U⋅W'^U⋃ r⋅R'^U⋅W_i'^UWhere r represents a reset operation of the FSM, returning the FSM to the initial state.

S402) obtaining a supplemented test case TS^C= r⋅Q'^U⋅W'^D⋃ r⋅R'^U⋅W_i'^D。

S403) obtaining a newly added test case TS^N= r⋅Q'^D⋅W'⋃ r⋅R'^D⋅W_i'。

S404) combining the reusable test cases, the supplemented test cases and the newly added test cases to form a new test suite TS' = TS^R⋃ TS^C⋃ TS^N。

In this embodiment, test cases can be classified into three categories according to the change of the model. Reusable test case TS^R= r⋅Q'^U⋅W'^U⋃ r⋅R'^U⋅W_i'^UFrom an existing test suite for testing the unaltered part of the model; supplementary test case TS^C= r⋅Q'^U⋅W'^D⋃ r⋅R'^U⋅W_i'^DThe same is used for testing the unchanged part of the model, but is not included in the original test kitThe influence of the modified part of the test system on the original part can be considered; newly added test case TS^N= r⋅Q'^D⋅W'⋃ r⋅R'^D⋅W_i' for the modified part of the test system, is naturally not included in the original test kit. Wherein the "signature" symbol refers to the concatenation of the set covering each sequence in the set concatenated with each sequence in the state recognizer corresponding to the state it has reached. The test cases of the several categories are put together to form a complete new test suite TS' = TS^R⋃ TS^C⋃ TS^N。

S500) performing a regression test, i.e. performing the regression test, please refer to fig. 6, which specifically includes:

s501) testing of the newly added test cases is carried out, and the newly added test cases and the changed parts are used for testing the newly added and changed parts of the system, are most directly influenced by modification, are not tested by the previous test cases, and are most likely to make errors.

S502) executing the test of the supplementary test case to confirm that the modification does not introduce some errors which cannot be found by the previous test case to the unchanged part.

In this embodiment, reusable test cases do not have to be executed again because they have been executed before and are not relevant to the modification of the system. When the scale of the system is large, a lot of overhead can be saved by avoiding executing these test cases.

In the method, the state machine is composed of a state register and a combinational logic circuit, can carry out state transition according to a preset state according to a control signal, and is a control center which coordinates the action of related signals and completes specific operation. Finite State machines, abbreviated as fsms (finite State machines), are mainly classified into 2 major categories: the first category, which refers to the Moore state machine if the output is state only and not input; in the second category, outputs are not only related to state but also to inputs, and are called Mealy state machines. Which is in a certain state of the finite set of states at any time. When it gets an input character, it will either transition from the current state to another state or remain in the current state. Any FSM can be described by a state transition graph, where nodes in the graph represent one state in the FSM and directed weighted edges represent changes in state as characters are input.

In the present embodiment, FSM is defined as a six-tuple M = (S, S)₀X, Y, δ, λ), where S is the set of states, S₀∈ S is its initial state, X is the set of input symbols, Y is the set of output symbols, δ: S × X → S is the state transfer function, and λ: S × X → Y is the output function when machine M is in state S and receives an input X, it will transfer to the next state t = δ (S, X) ∈ S and produce an output Y = λ (S, X) ∈ Y, referred to as S-X/Y → t as a transition.

The overlay set is used to overlay each part of the FSM and can be divided into a state overlay set and a migration overlay set. A sequence containing some input symbols, called a transition sequence, enables the FSM to transition from an initial state to each state or transition.

And the state recognizer is used for judging whether the current state of the FSM is in accordance with the expectation. Because the system to be tested is a black box FSM, a sequence is input into the FSM, only output can be observed, but the reached state cannot be observed, after a transfer sequence in a coverage set is input, if a user wants to know whether the correct state is reached, some specific sequences need to be continuously input, so that whether the current state of the system to be tested is in accordance with expectation can be judged according to the output of the specific sequences, the set of the sequences is called a state identifier, and the principle is to generate different outputs for different states.

Referring to fig. 14, in the present embodiment, the method uses a gate, whose control part can be modeled as a FSM, as an original system, and includes two states: l (for lock) and U (for unlock); two inputs: c (representing coin plunge) and p (representing push bar); and three outputs: n (indicating no response), L (indicating the crossbar is locked) and F (indicating the crossbar can be pushed freely). When the floodgate machine is locked, the horizontal pole can't promote, drops into the coin and can the unblock. After the gate is unlocked, the effect of throwing the coins again can not be generated; at this point the push rail can enter and cause the gate to be locked again.

State coverage of this modelSet Q = { epsilon, c }, migration coverage set P = { epsilon, c, P, cc, cp }. Feature set and identity set W_L=W_U=W={p}。

In daily life, it is inconvenient to carry coins with them, so an unlocking mode of swiping a card is added to the gate besides coin inserting, and the action of the unlocking mode is similar to that of coin inserting. In addition, it would be desirable to have a gate with a free mode in which access is free without having to be re-unlocked each time access is made. Such as selling a group ticket, should allow all access to the entire team without requiring each person to individually coin in. For simplicity, we choose to modify the behavior of the gate itself to enter this mode: when the system is unlocked, the coin is put in again, so that the gate can enter the mode; and a second coin in the free mode may cause the gate to exit this mode and relock.

Referring to fig. 15, the model mentioned in fig. 14 is modified according to the method to obtain a new model, and the method is verified by the generated new model.

Changes in the model are taken into account. Including a new state, a changed migration, and five new migrations. The details have been described above and are not listed here.

A state coverage set Q ' = { epsilon, c, cc } and a migration coverage set P ' = { epsilon, c, P, s, cc, cp, cs, ccc, ccp, ccs } of M ' are generated. According to the difference between M and M ', the states L and U are left-over, and their transition sequences are Q'^U= = { epsilon, c }; the state F is newly added, and the transition sequence thereof is Q'^D= cc. In addition to these, the transition sequence corresponding to the unaltered transition is R'^U= { p, cp }, and the transition sequence for different transitions is R'^D={s, cs, ccc, ccp, ccs}。

The feature set W in the original test kit contains only one isolated sequence SS (L, U) = p, which is not difficult to verify as still valid in M ', so it is included in the feature set and the invariant portion W of the identification set'_L ^U=W'_U ^U=W'^U= p. The separation sequence between the new state and the remaining states does not yet exist, generating them: SS (L, F) = p, SS (U, F) = pp, and then classified into the respectiveW 'different parts of feature set and identification set'_L ^D={p}，W'_U ^D={pp}，W'_F ^D={pp}，W'^D={pp}。

Combining these fractions gives:

reusable test case TS^R={rpp, rcpp}；

Newly added test case TS^N={rccppp, rspp, rcspp, rcccp, rccspp}；

Supplementary test case TS^C={rpp, rcpp}；

Complete test suite TS' = { rpp, rcpp, rccpp, rspp, rcspp, rcccpp, rccspp }.

Referring to fig. 7, the apparatus includes:

and the input module is used for inputting the old model, the original test suite and the new model of the system to be tested. In the present embodiment, the old model M = (S, S)₀X, Y, δ, λ); new model M '= (S', S)₀', X ', Y ', δ ', λ '). The test suite includes, but is not limited to, an overlay set including a state overlay set Q and a migration overlay set P, and a state recognizer including a feature set W and an identity set W for each state_i；

And the model analysis module is used for comparing the old model with the new model and analyzing the change of the model to obtain a model change result. In the present embodiment, the mentioned model changes include changes in state and transitions.

The state S is identified by name, and its change can be directly found by the set operation, where the state in S-S ' is removed, the state in S ' -S is added, and the state in S ∩ S ' is left.

The transitions S-X/y → t are identified by the start state S and the input X, and can be found by the collective operation considering the change of the output y and the end state t. the input sign can be found as well, i.e. the removed input X-X ', the newly added input X ' -X, the left-over input X ∩ X '. the removed transitions include the transitions corresponding to all inputs X ∈ 1X ' at the removed state S ∈ S-S ', the transitions corresponding to the removed inputs X ∈ X-X ' at all states S ∈ S ', the newly added transitions include the transitions corresponding to all inputs X ∈ X ' at the newly added state S ∈ S ' -S, the transitions corresponding to all inputs X ∈ X ' -X at all states S ∈ S ', the left-over transitions refer to the transitions corresponding to the left-over state S ∈ S ∈ 0S ', the transitions corresponding to the left-over inputs X ∈ X ∈ 2X ' at all states S ∈ S ', and can be further divided into the transitions of changes, the output or end state is different from the old model, i.e. the old model (S, S δ S) = (S, X ≠ S, S ≠ S ' (i.e. the new model), and the new model (S, i.e. the old model is not identical to the new model).

And the test suite analysis module is used for analyzing the change of the test suite according to the model change result to obtain a test suite change result.

It can be understood that, in this embodiment, referring to fig. 8, the test kit analysis module may specifically include:

and the coverage set analysis module is used for analyzing the change of the state coverage set and the migration coverage set according to the model change result, and segmenting the state coverage set Q ' and the migration coverage set P ' of the M ' to obtain a coverage set change result. The coverage set of the new model is divided into two parts: the unchanged part, denoted by superscript U; the different parts, denoted by the superscript D, as shown in fig. 9, may specifically include:

and the first generation module is used for generating the state coverage set Q ' and the migration coverage set P ' of the M '.

A first dividing module for classifying the transition sequence corresponding to the left state in Q ' into the unchanged part Q ' of the state coverage set Q '^UTransition sequences corresponding to newly added states are attributed to different portions Q 'of the state overlay set Q'^D。

A second dividing module, configured to attribute a transition sequence corresponding to the unchanged transition in P ' to an unchanged portion P ' of the transition overlay set P '^UThe transition sequences corresponding to the newly added transitions and the changed transitions are attributed to different portions P 'of the migration overlay set P'^D。

A calculation module for calculating a portion R '= P' -Q 'in the migration overlay set P' that is not comprised by the state overlay set Q ', wherein R'^U=P'^U-Q'^U，R'^D=P'^D-Q'^D。

State recognizer analysisA module for analyzing the change of the recognition state device according to the model change result to generate a feature set W 'of M' and a recognition set W of each state_i', obtaining a state identifier change result. Similarly, the state recognizer of the new model is also divided into an unchanged portion and a different portion, and since the feature set and the recognition set are composed of separate sequences between different states, that is, different input sequences are output thereon, which are denoted as SS (s, t), and a change in the separate sequences between the states is found, the change in the state recognizer is found, referring to fig. 10, which may specifically include:

a placement module for generating a separate sequence SS ' (S, t) for each newly added state S ∈ S ' -S, any other state t ∈ S ', and attributing to a different portion W ' of the corresponding state identifier '^D, W_s'^D, W_t'^D；

A judging module for judging whether the separation sequence SS (S, t) is still valid in M 'for each pair of the left states S, t ∈ S ∩ S', and if so, attributing the SS (S, t) to the unchanged part W 'of the corresponding state identifier'^U, W_s'^U, W_t'^UIf not, a new separation sequence SS ' (s, t) is generated and included in a different part W ' of the corresponding state identifier '^D, W_s'^D, W_t'^D；

A second generating module for combining different portions W 'of the status identifier'^D, W_i'^DAnd an unchanged portion W 'of the state identifier'^U, W_i'^UGenerating a feature set W ' = W ' of M '^U∪W'^DAnd an identification set W for each state_i'= W_i'^U∪W_i'^D。

The test suite generation module is configured to combine the changed test suites according to the test suite change result to generate a new test suite, please refer to fig. 11, which may specifically include:

a first obtaining module for obtaining reusable test case TS^R= r⋅Q'^U⋅W'^U⋃ r⋅R'^U⋅W_i'^UWherein r represents the reset operation of FSM to return FSM to initial stateThe initial state.

A second obtaining module for obtaining the supplemented test case TS^C= r⋅Q'^U⋅W'^D⋃ r⋅R'^U⋅W_i'^D。

A third obtaining module for obtaining the newly added test case TS^N= r⋅Q'^D⋅W'⋃ r⋅R'^D⋅W_i'。

A combination module for combining the reusable test cases, the supplemented test cases and the newly added test cases to form a new test suite TS' = TS^R⋃ TS^C⋃ TS^N。

Test cases can be classified into three categories according to the change of the model. Reusable test case TS^R= r⋅Q'^U⋅W'^U⋃ r⋅R'^U⋅W_i'^UFrom an existing test suite for testing the unaltered part of the model; supplementary test case TS^C= r⋅Q'^U⋅W'^D⋃ r⋅R'^U⋅W_i'^DThe test model is also used for testing the unchanged part of the model, but is not contained in the original test kit, and can be regarded as the influence of the modified part of the test system on the original part; newly added test case TS^N= r⋅Q'^D⋅W'⋃ r⋅R'^D⋅W_i' for the modified part of the test system, is naturally not included in the original test kit. Wherein the "signature" symbol refers to the concatenation of the set covering each sequence in the set concatenated with each sequence in the state recognizer corresponding to the state it has reached. The test cases of the several categories are put together to form a complete new test suite TS' = TS^R⋃ TS^C⋃ TS^N。

The test module is configured to perform a regression test, that is, to execute the regression test, referring to fig. 12, and specifically includes:

the new case test module is used for executing the test of the new test case, and the new case test module is used for testing the new and changed parts of the system, wherein the parts are most directly influenced by modification, are not tested by the previous test case and are most likely to make mistakes.

And the supplementary case testing module is used for executing the test of the supplementary test case to confirm that the modification does not introduce some errors which cannot be found by the previous test case to the unchanged part.

In this embodiment, reusable test cases do not have to be executed again because they have been executed before and are not relevant to the modification of the system. When the scale of the system is large, a lot of overhead can be saved by avoiding executing these test cases.

In this device, the state machine is composed of a state register and a combinational logic circuit, can perform state transition according to a preset state according to a control signal, and is a control center which coordinates the operation of related signals and completes a specific operation. Finite State machines, abbreviated as fsms (finite State machines), are mainly classified into 2 major categories: the first category, which refers to the Moore state machine if the output is state only and not input; in the second category, outputs are not only related to state but also to inputs, and are called Mealy state machines. Which is in a certain state of the finite set of states at any time. When it gets an input character, it will either transition from the current state to another state or remain in the current state. Any FSM can be described by a state transition graph, where nodes in the graph represent one state in the FSM and directed weighted edges represent changes in state as characters are input.

In the present embodiment, FSM is defined as a six-tuple M = (S, S)₀X, Y, δ, λ), where S is the set of states, S₀∈ S is its initial state, X is the set of input symbols, Y is the set of output symbols, δ: S × X → S is the state transfer function, and λ: S × X → Y is the output function when machine M is in state S and receives an input X, it will transfer to the next state t = δ (S, X) ∈ S and produce an output Y = λ (S, X) ∈ Y, referred to as S-X/Y → t as a transition.

The overlay set is used to overlay each part of the FSM and can be divided into a state overlay set and a migration overlay set. A sequence containing some input symbols, called a transition sequence, enables the FSM to transition from an initial state to each state or transition.

And the state recognizer is used for judging whether the current state of the FSM is in accordance with the expectation. Because the system to be tested is a black box FSM, a sequence is input into the FSM, only output can be observed, but the reached state cannot be observed, after a transfer sequence in a coverage set is input, if a user wants to know whether the correct state is reached, some specific sequences need to be continuously input, so that whether the current state of the system to be tested is in accordance with expectation can be judged according to the output of the specific sequences, the set of the sequences is called a state identifier, and the principle is to generate different outputs for different states.

Referring to fig. 14, in the present embodiment, the apparatus uses a gate, whose control part can be modeled as an FSM, as an original system, and includes two states: l (for lock) and U (for unlock); two inputs: c (representing coin plunge) and p (representing push bar); and three outputs: n (indicating no response), L (indicating the crossbar is locked) and F (indicating the crossbar can be pushed freely). When the floodgate machine is locked, the horizontal pole can't promote, drops into the coin and can the unblock. After the gate is unlocked, the effect of throwing the coins again can not be generated; at this point the push rail can enter and cause the gate to be locked again.

The state coverage set Q of this model = { epsilon, c }, and the migration coverage set P = { epsilon, c, P, cc, cp }. Feature set and identity set W_L=W_U=W={p}。

In daily life, it is inconvenient to carry coins with them, so an unlocking mode of swiping a card is added to the gate besides coin inserting, and the action of the unlocking mode is similar to that of coin inserting. In addition, it would be desirable to have a gate with a free mode in which access is free without having to be re-unlocked each time access is made. Such as selling a group ticket, should allow all access to the entire team without requiring each person to individually coin in. For simplicity, we choose to modify the behavior of the gate itself to enter this mode: when the system is unlocked, the coin is put in again, so that the gate can enter the mode; and a second coin in the free mode may cause the gate to exit this mode and relock.

Referring to fig. 15, the model mentioned in fig. 14 is modified according to the apparatus to obtain a new model, and the method is verified by the generated new model.

Changes in the model are taken into account. Including a new state, a changed migration, and five new migrations. The details have been set forth above and are not necessarily listed in text herein.

A state coverage set Q ' = { epsilon, c, cc } and a migration coverage set P ' = { epsilon, c, P, s, cc, cp, cs, ccc, ccp, ccs } of M ' are generated. According to the difference between M and M ', the states L and U are left-over, and their transition sequences are Q'^U= = { epsilon, c }; the state F is newly added, and the transition sequence thereof is Q'^D= cc. In addition to these, the transition sequence corresponding to the unaltered transition is R'^U= { p, cp }, and the transition sequence for different transitions is R'^D={s, cs, ccc, ccp, ccs}。

The feature set W in the original test kit only contains one separation sequence SS (L, U) = p, which is not difficult to verify as still valid in M', so it is classified into the feature set and the unchanged part W of the recognition set_L ^U=W'_U ^U=W'^U= p. The separation sequence between the new state and the remaining states does not yet exist, generating them: SS (L, F) = p, SS (U, F) = pp, then respectively subsume the corresponding feature set and different portions W 'of the identification set'_L ^D={p}，W'_U ^D={pp}，W'_F ^D={pp}，W'^D={pp}。

Combining these fractions gives:

reusable test case TS^R={rpp, rcpp}；

Newly added test case TS^N={rccppp, rspp, rcspp, rcccp, rccspp}；

Supplementary test case TS^C={rpp, rcpp}；

Complete test suite TS' = { rpp, rcpp, rccpp, rspp, rcspp, rcccpp, rccspp }.

Referring to fig. 16, 17 and 18, the comparison between the regression test using the method and apparatus and the conventional method without using the method and apparatus is shown, in which all test cases are regenerated and executed. Experiments are carried out on a group of large-scale random generation FSM models, and results show that most of original test cases can be reused, a large amount of test expenses are reduced, and the time spent on realizing the effects is shorter.

Specifically, the number of states {100, 110, …, 990, 1000}, the number of inputs 10, the number of outputs 5, are set, and 100 different FSMs are generated for each setting to increase the confidence of the results. Taking each generated FSM as the original model, 4 different modifications are made above: adding inputs, adding outputs, deleting inputs, deleting states, covering all possible situations as much as possible. The proposed regression testing method was applied to the original model and the modified model, with the following results.

With respect to reusing test cases, the proportion of reused test cases relative to the number of all original test cases can be seen from fig. 16. It can be seen that the reuse proportion is quite high, and at least exceeds 85%

In terms of test case length, as can be seen from fig. 17, the length ratio between the test case to be executed and all test cases reflects the overhead of test execution. It can be seen that this ratio is quite low, up to around 35%. That is, more than 65% of the test execution overhead can be reduced by our method.

In terms of test set up time, it can be seen from FIG. 18 that the method and apparatus compares the proportion of time required to regenerate a test suite. It can be seen that the time taken to perform the test is always less than the regeneration.

Claims (10)

1. A regression testing method based on a finite state machine is characterized by comprising the following steps:

inputting an old model, an original test suite and a new model of a system to be tested, wherein the old model M = (S, S)₀X, Y, δ, λ), new model M '= (S', S)₀', X ', Y ', delta ', lambda '), the test kit includes an overlay set including a state overlay set Q and a migration overlay set P, and a state recognizer including a feature set W and an identity set W for each state_i；

Comparing the old model with the new model, and analyzing the change of the model to obtain a model change result;

analyzing the change of the test suite according to the model change result to obtain a test suite change result; combining the changed test kits according to the test kit change result to generate a new test kit;

a regression test was performed.

2. The regression testing method based on the finite-state machine according to claim 1, wherein the change of the test suite is analyzed according to the change result of the model to obtain the change result of the test suite, which specifically comprises:

analyzing the change of the coverage set according to the model change result, and segmenting the state coverage set Q ' and the migration coverage set P ' of the M ' to obtain a coverage set change result;

analyzing the change of the recognition state machine according to the model change result to generate a feature set W 'of M' and a recognition set W of each state_i', obtaining a state identifier change result.

3. The regression testing method based on the finite-state machine according to claim 2, wherein the state coverage set Q ' and the migration coverage set P ' of M ' are segmented according to the model change result, and the change of the coverage set is analyzed to obtain the coverage set change result, which specifically includes:

generating a state coverage set Q ' and a migration coverage set P ' of M ';

attributing transition sequences corresponding to states left in Q ' to an invariant portion Q ' of a state overlay set Q '^UThe transition order corresponding to the newly added state is included in different parts Q 'of the state coverage set Q'^D；

Attributing a transfer sequence corresponding to the unchanged migration in the P ' to an unchanged part P ' of the migration overlay set P '^UThe transition sequences corresponding to the newly added transitions and the changed transitions are attributed to different portions P 'of the migration overlay set P'^D；

Calculating the part R ' = P not included in the state coverage set Q ' in the migration coverage set P ' =' -Q ', wherein R '^U=P'^U-Q'^U，R'^D=P'^D-Q'^D；

Analyzing the change of the recognition state machine according to the model change result to generate a feature set W 'of M' and a recognition set W of each state_i' obtaining a state identifier change result, specifically comprising:

for each newly added state S ∈ S ' -S and any other state t ∈ S ', a separate sequence SS ' (S, t) is generated and attributed to a different portion W ' of the corresponding state identifier '^D, W_s'^D, W_t'^D；

For each pair of left-behind states S, t ∈ S ∩ S ', it is determined whether the separation sequence SS (S, t) is still valid in M', and if so, the SS (S, t) is included in the unchanged portion W 'of the corresponding state identifier'^U, W_s'^U, W_t'^UIf not, a new separation sequence SS ' (s, t) is generated and included in a different part W ' of the corresponding state identifier '^D, W_s'^D, W_t'^D；

Different parts W 'of the combined status identifier'^D, W_i'^DAnd an unchanged portion W 'of the state identifier'^U, W_i'^UGenerating a feature set W ' = W ' of M '^U∪W'^DAnd an identification set W for each state_i'= W_i'^U∪W_i'^D。

4. The regression testing method based on the finite-state machine according to claim 3, wherein the test suite after the change is combined according to the change result of the test suite to generate a new test suite, specifically comprising:

obtaining reusable test cases TS^R= r⋅Q'^U⋅W'^U⋃ r⋅R'^U⋅W_i'^U；

Obtaining a complementary test case TS^C= r⋅Q'^U⋅W'^D⋃ r⋅R'^U⋅W_i'^D；

Obtaining newly added test case TS^N= r⋅Q'^D⋅W'⋃ r⋅R'^D⋅W_i'；

Combining the reusable test cases, the supplementary test cases and the newly added test cases to form a new test suite TS' = TS^R⋃ TS^C⋃ TS^N。

5. The regression testing method based on the finite-state machine according to claim 4, wherein the regression testing specifically comprises:

executing the test of the newly added test case;

a test of the supplemental test case is performed.

6. A finite state machine based regression testing apparatus, comprising:

an input module for inputting an old model, an original test suite and a new model of the system to be tested, wherein the old model M = (S, S)₀X, Y, δ, λ), new model M '= (S', S)₀', X ', Y ', delta ', lambda '), the test kit includes an overlay set including a state overlay set Q and a migration overlay set P, and a state recognizer including a feature set W and an identity set W for each state_i；

The model analysis module is used for comparing the old model with the new model and analyzing the change of the model to obtain a model change result;

the test suite analysis module is used for analyzing the change of the test suite according to the model change result to obtain a test suite change result;

the kit generating module is used for combining the changed test kits according to the test kit change result to generate a new test kit;

and the test module is used for performing regression test.

7. The regression test device based on the finite state machine according to claim 6, wherein the test suite analysis module specifically comprises:

the coverage set analysis module is used for analyzing the change of the coverage set according to the model change result, and segmenting the state coverage set Q ' and the migration coverage set P ' of the M ' to obtain a coverage set change result;

the state recognizer analysis module is used for analyzing and recognizing the change of the state recognizer according to the model change result to generate a feature set W 'of M' and an identification set W of each state_i', obtaining a state identifier change result.

8. The finite state machine-based regression test device of claim 7, wherein the coverage set analysis module specifically comprises:

the first generation module is used for generating a state coverage set Q ' and a migration coverage set P ' of the M ';

a first dividing module for classifying the transition sequence corresponding to the left state in Q ' into the unchanged part Q ' of the state coverage set Q '^UTransition sequences corresponding to newly added states are attributed to different portions Q 'of the state overlay set Q'^D；

A second dividing module, configured to attribute a transition sequence corresponding to the unchanged transition in P ' to an unchanged portion P ' of the transition overlay set P '^UThe transition sequences corresponding to the newly added transitions and the changed transitions are attributed to different portions P 'of the migration overlay set P'^D；

A calculation module for calculating a portion R '= P' -Q 'in the migration overlay set P' that is not comprised by the state overlay set Q ', wherein R'^U=P^U-Q'^U，R'^D=P^D-Q'^D;

The state recognizer analysis module specifically comprises:

a placement module for generating a separate sequence SS ' (S, t) for each newly added state S ∈ S ' -S and any other state t ∈ S ' and attributing to a different portion W ' of the corresponding state identifier '^D, W_s'^D, W_t'^D；

A judging module for judging whether the separation sequence SS (S, t) is still valid in M 'for each pair of the left states S, t ∈ S ∩ S', and if so, attributing the SS (S, t) to the unchanged part W 'of the corresponding state identifier'^U, W_s'^U, W_t'^UIf it is not validA new separation sequence SS '(s, t) is generated and included in a different part W of the corresponding state identifier'^D, W_s'^D, W_t'^D;

A second generating module for combining different portions W 'of the status identifier'^D, W_i'^DAnd an unchanged portion W 'of the state identifier'^U, W_i'^UGenerating a feature set W ' = W ' of M '^U∪W'^DAnd an identification set W for each state_i'= W_i'^U∪W_i'^D。

9. The finite state machine-based regression testing device of claim 8, wherein the suite generation module specifically comprises:

a first obtaining module for obtaining reusable test case TS^R= r⋅Q'^U⋅W'^U⋃ r⋅R'^U⋅W_i'^U；

A second obtaining module for obtaining the supplemented test case TS^C= r⋅Q'^U⋅W'^D⋃ r⋅R'^U⋅W_i'^D；

A third obtaining module for obtaining the newly added test case TS^N= r⋅Q'^D⋅W'⋃ r⋅R'^D⋅W_i'；

A combination module for combining the reusable test cases, the supplemented test cases and the newly added test cases to form a new test suite TS' = TS^R⋃ TS^C⋃ TS^N。

10. The finite state machine-based regression testing device of claim 9, wherein performing regression testing specifically comprises:

the new case testing module is used for executing the test of the new test case;

and the supplementary case testing module is used for executing the test of the supplementary test case.

Images