CN117971684A - A semantically-aware whole-machine regression test case recommendation method - Google Patents

Abstract

The invention discloses a whole machine regression test case recommendation method for changing semantic perception, which comprises the following steps: (1) Acquiring code submission information and a test case, and cleaning to acquire a text data set D _{Submitting information} of the code submission information and a description data set D _{Test case} of the test case; (2) Classifying each submitted information text in the text dataset D _{Submitting information} into different characteristic tags according to a hierarchical structure by adopting a hierarchical residual multi-granularity classification network model based on a tag relation tree; the feature tag represents a description tag of a corresponding function of the test case test; (3) And screening test cases under the classification from the description dataset D _{Test case} according to the classification of each submitted information text, performing similarity calculation on the submitted information text and the corresponding screened test case text, and selecting the test cases with high similarity scores as recommended test cases. By utilizing the method and the device, the efficiency and the accuracy of test case selection can be effectively improved.

Description

A semantically-aware whole-machine regression test case recommendation method

Technical Field

The present invention belongs to the field of software regression testing, and in particular relates to a change semantics-aware whole-machine regression test case recommendation method.

Background technique

Regression testing is a key step in software development and release. It is performed after modifying the old code to ensure that these modifications have not introduced new defects or affected the normal operation of other functions. Automated regression testing can significantly reduce the cost of system testing, maintenance and update.

As an important part of the software life cycle, regression testing occupies an important position in the entire software testing process and is repeatedly performed many times at each stage of software development. Especially in the incremental and rapid iteration development mode, the frequent release of new versions requires more frequent regression testing, and in projects using extreme programming methods, multiple rounds of regression testing may be required every day. In this context, it is particularly important to choose a suitable regression testing strategy to improve the efficiency and effectiveness of regression testing. For example, Chinese patent document No. CN101178687A discloses an improved software regression testing method.

With the continuous increase and iteration of product functions, the scale of test case sets is also expanding, which directly leads to the increase of regression testing costs. Given the limited testing resources, it is impossible to execute all test cases. In order to improve the efficiency of regression testing, it is urgent to formulate a more scientific regression testing strategy. This requires effective sorting of test cases based on a series of clear test objectives to determine their execution order and ensure that the most critical test cases are executed first.

For example, the paper "Can Code Representation Boost IR-Based Test Case Prioritization" proposed a method to recommend test cases by using the semantic similarity between the changed code and the test code. This method first uses the code representation method to convert the changed code and the test code into vectors, then calculates the similarity between the test method and the changed code based on the method level and class level granularity, and finally combines the similarity of the two granularities to give test cases with high similarity to the changed code.

For example, the paper "Prioritizing Natural Language Test Cases Based on Highly-Used Game Features" proposes a method to recommend artificial test cases described in natural language through zero-shot classification and genetic algorithm multi-objective optimization. This method first uses the zero-shot classification method to automatically identify the functions that the test cases will test from the natural language test case description, and then prioritizes the test cases based on the frequently used functions covered by the test cases and the execution time of the test cases. Here, the genetic algorithm is used to optimize the two goals of covering more functions and short execution time to obtain a better test case recommendation ranking.

Although existing techniques provide basic methods for regression test case selection, they have some significant disadvantages:

1. Scenario Mismatch: Most prioritization methods are not applicable to manual test cases because they require source code information or test execution history to support their algorithms, which are usually not applicable in manual testing scenarios.

2. Insufficient understanding of change semantics: Code submission information contains important semantic information about code changes, which is crucial for identifying affected functions and corresponding test cases. Existing technologies lack an effective mechanism to interpret this semantic information, resulting in test case selection that may not be accurate or relevant enough.

Therefore, a more efficient and intelligent technical solution is needed to overcome the above problems, especially one that can understand the semantic information of code submissions and automatically recommend relevant manual test cases to improve the accuracy and efficiency of software testing.

Summary of the invention

The present invention provides a change semantics-aware whole-machine regression test case recommendation method, which can effectively improve the efficiency and accuracy of test case selection.

A change semantics-aware whole-machine regression test case recommendation method includes the following steps:

(1) Obtain code submission information and test cases. After data cleaning, obtain a text dataset D of code _{submission information} and a description dataset D of _test cases;

(2) Using a hierarchical residual multi-granularity classification network model based on a label relationship tree, each submission information text in the text dataset D _submission information is classified into different feature labels according to the hierarchical structure;

Among them, the feature label represents the description label of the corresponding function tested by the test case, and its description form is a tree-like multi-level label representation;

(3) According to the classification of each submitted information text, a subset of test cases under the classification is selected from _{the test} cases in the description data set D, and the similarity between the submitted information text and the text in the corresponding subset of test cases is calculated, and the cases with high similarity scores are selected as recommended test cases.

Step (2) specifically includes:

(2-1) Through the fine-tuned BERT model, each submission information text in the text dataset D _{submission information} is converted into a high-dimensional vector;

(2-2) Convert the label hierarchical relationship in the test case description data set D _{test case} into a tree structure and generate a state vector based on binary labels;

(2-3) The high-dimensional text vector obtained in step (2-1) is input into the hierarchical residual multi-granularity classification network model, and the state vector obtained in step (2-2) is constrained to perform hierarchical feature classification on the high-dimensional text vector to determine the hierarchical feature label of each submitted information text.

The specific process of step (2-1) is:

(2-1-1) Select the BERT model and use the submitted information text data with labeled features for training and fine-tuning;

(2-1-2) Using the word segmenter corresponding to the BERT model, convert the submission information text in the text dataset D _{submission information} into a word list and a corresponding mask list;

(2-1-3) Input the word list and the corresponding mask list into the BERT model trained and fine-tuned in step (2-1-1) and convert them into high-dimensional vectors;

(2-1-4) Select the pooled vector output of the BERT model as the vector representation of the submitted information text.

The specific process of step (2-2) is:

(2-2-1) Design a structured format to store the hierarchical information of the feature tree. The feature label represents the description label of the corresponding function tested by the test case, and its description form is a tree-like multi-level label representation; wherein the test case is divided into any layer of nodes in the feature tree, and at the leaf node or the middle node, different feature nodes represent different test case classifications;

(2-2-2) The feature tree calculates all binary label-based state vectors that meet the feature tree constraints to form a reasonable state space; if the total number of feature nodes is N, an N×N matrix is formed. By traversing the feature tree structure, each column of the matrix represents a feature node, and each row represents all possible legal paths in the feature tree, so that the binary-valued hierarchical category label represents the feature node label, that is, the state space is used to define the legal constraints on the relationship between parent-child nodes and sibling nodes.

The specific process of step (2-3) is:

(2-3-1) Calculating the number of layers of the feature tree and the number of features in each layer according to the state space in step (2-2-2) to construct a network structure of a hierarchical residual multi-granularity classification network model; wherein the hierarchical residual multi-granularity classification network model comprises a hierarchical feature transfer module, a residual connection part and a hierarchical classification module;

(2-3-2) Establish a hierarchical feature transfer module, which is composed of several fully connected networks or convolutional networks, and is used for feature extraction between different levels and downward transmission; establish a residual connection part, through the sequential connection of coarse-grained parent-class level features and fine-grained child-class level feature layers, to obtain a combined feature vector that is combined from the parent-class feature information layer by layer to the child-class;

(2-3-3) Applying a hierarchical classification module based on the combined feature vector, the combined feature vector passes through each level of the classification network, and each dimension of the classification network output corresponds to each feature label;

(2-3-4) Set the hierarchical threshold and determine the hierarchical classification of the text based on the vector output by each layer of the classification network.

Step (3) specifically includes:

(3-1) According to the classification of each submission information text, a subset of test cases under the classification is selected from _{the test} cases in the description dataset D; each submission information text in _{the submission information} of the dataset D and the text in the corresponding subset of test cases are converted into high-dimensional vectors through a fine-tuned BERT model;

(3-2) Based on the high-dimensional vector representation of the text obtained in step (3-1), calculate the similarity between the vector representation of the submitted information text and the vector representation of each test case description in the corresponding test case set to obtain a list of recommended test cases.

The specific process of step (3-1) is:

(3-1-1) Select the BERT model and train and fine-tune it based on the annotated submission information and test case text data;

(3-1-2) According to the classification of each submitted information text, a subset of test cases under the classification is selected from _{the test} cases in the description dataset D; through the word segmenter corresponding to the BERT model, the submitted information text and the text of the corresponding subset of test cases are converted into a word list and a corresponding mask list;

(3-1-3) Input the word list and the corresponding mask list in step (3-1-2) into the BERT model trained and fine-tuned in step (3-1-1), and convert them into high-dimensional vectors of submission information and test case descriptions;

(3-1-4) Select the pooled vector output of the BERT model as the vector representation of the corresponding text.

The specific process of step (3-2) is:

(3-2-1) Calculate the similarity between the vector representation of the submitted information and the vector representation of each test case description in the corresponding test case set, where the similarity calculation uses the cosine similarity formula:

(3-2-2) According to the similarity scores given in step (3-2-1), test cases with higher similarity are selected as recommended test cases, and a list of recommended test cases is output.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention designs a two-stage test case selection framework. In the first stage, according to the code submission semantic information, a hierarchical residual multi-granularity classification model based on a label tree structure is used to classify the code into corresponding test case feature labels, thereby screening out a subset of test cases from a massive test case set, reducing the scope of test cases that need to be calculated and matched in the second stage, and improving the test case selection speed. In the second stage, according to the code submission semantic information and the test case description information, the SBERT model is used to calculate the semantic similarity between the code submission semantic information and the description information of the subset of test cases screened out in the first stage, and a recommended test case list sorted by the change semantic relevance is output according to the similarity score. Through the processing and intelligent analysis of the code submission information text, the method provides an efficient and accurate way to automatically select relevant regression test cases in the software development process, which is suitable for fast-iteration software development projects.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a method for recommending whole-machine regression test cases based on semantic change awareness according to the present invention;

FIG2 is a schematic diagram of the residual connection part of the hierarchical residual multi-granularity classification network model of the present invention;

FIG3 is a flow chart of step S200 in an embodiment of the present invention;

FIG. 4 is a flow chart of step S300 in an embodiment of the present invention.

Detailed ways

The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be pointed out that the embodiments described below are intended to facilitate the understanding of the present invention and do not have any limiting effect on the present invention.

As shown in FIG1 , a change semantics-aware whole-machine regression test case recommendation method includes the following steps:

S100, data preprocessing, obtaining the text of code submission information in the historical version to form the original data set, and then cleaning the data set to output the data set D _{submission information} that can be used in the following steps. And according to the same method, obtain the description data set D _{test case} of the cleaned test case.

In this step, the specific process of cleaning the original data set is:

S101, collect text from a database.

S102, removing irrelevant information such as special characters and spaces in the text.

S103, normalizing the text. Specifically, firstly, the upper and lower case letters in the text are unified, and then key fields are extracted according to the template of the submitted information format.

S104, output the cleaned and normalized text data.

S200, obtain the structure of the label tree in the test case library, and convert the tree structure into a label relationship state vector for constructing a classification network model. A hierarchical residual multi-granularity classification network model based on a label relationship tree is used to classify the text data of the code submission information in step S100 into different feature labels according to a hierarchical structure. The feature label represents the description label of the corresponding function tested by the test case, and its description is represented by a tree-like multi-level label. The hierarchical residual multi-granularity classification network model based on the label relationship tree in this step needs to be trained with a certain amount of submission information text data with annotated features to adapt to the specific environment of a specific project.

The specific steps of S200 are shown in FIG3 , and specifically include:

S210, a text vectorization module (BERT model), converts the output text in step S100 into a high-dimensional vector through a fine-tuned BERT model.

S220, a label relationship tree construction module, converts the label hierarchy of the use case library into a tree structure and generates a binary label state vector.

S230, a hierarchical classification module, inputs the high-dimensional text vector obtained in step S210 into the classification model, constrains it through the state vector obtained in step S220, performs hierarchical feature classification on the high-dimensional text vector, and determines the hierarchical feature label of each submitted information text.

Step S210 specifically includes:

S211, select a suitable BERT pre-trained model, and perform training and fine-tuning based on the text data of the submitted information with labeled features to adapt to the project-specific text data.

S212, the text output in step S100 is converted into a word list and a corresponding mask list through the word segmenter corresponding to the BERT model.

S213, input the word list and the corresponding mask list in step S212 into the BERT model and convert them into high-dimensional vectors.

S214, select the pooled vector output of the BERT model as the vector representation of the text and output it to the next step.

Wherein step S220 specifically includes:

S221, extract the hierarchical information of the feature tree to form a multi-level feature label tree. Design a structured format to store the hierarchical information of the feature tree. The feature label represents the description label of the corresponding function tested by the test case, and its description form is a tree-like multi-level label representation. Test cases can be divided into any layer of nodes in the feature tree, either in leaf nodes or in intermediate nodes. In most application scenarios, different feature nodes represent different test case classifications.

S222, the feature tree in step S221 calculates all binary label-based state vectors that meet the feature tree constraints to form a reasonable state space. If the total number of feature nodes is N, an N×N matrix is formed. By traversing the feature tree structure, each column of the matrix represents a feature node, and each row represents all possible legal paths in the feature tree, so that the binary-valued hierarchical category label represents the feature node label, that is, the state space is used to define the legal constraints of the relationship between parent-child nodes and sibling nodes.

Wherein step S230 specifically includes:

S231, calculate the hierarchical attributes required for building the network, and calculate the number of layers of the feature tree and the number of features in each layer according to the state space in step S222 to build a hierarchical residual network structure.

S232, establish a hierarchical feature transfer module, which is composed of several fully connected networks or convolutional networks, and is used for feature extraction and downward transfer between different levels. Establish a residual connection part, as shown in Figure 2, by sequentially connecting the coarse-grained parent-class level feature and the fine-grained child-class level feature layer, to obtain a combined feature vector that is combined from the parent-class feature information layer by layer to the child class.

S233, according to step S232, the combined feature vector is applied to the hierarchical classification module, and the combined feature vector passes through the classification network of each level. First, the number of feature labels of each layer is determined according to the feature tree, and the fully connected network hierarchy of the classification model is determined. The calculation process of the fully connected network of one layer is:

x _i+1 =σ( _Wixi + _{bi )}

Where x _i+1 represents the output vector, σ represents the activation function, _Wi represents the network weight of the current layer, x _i represents the input vector, and _bi represents the bias. The Sigmoid function is used as the activation function. Each dimension of the classification network output corresponds to each feature label.

S234, determine the hierarchical classification of the text based on the vector output by each layer of the classification network. First, it is necessary to set the threshold of each layer. When the value of a certain dimension of the vector output in step S233 is higher than the threshold of the current layer, the text is classified into the feature label corresponding to this dimension of the layer. The tree structure is used to determine whether the classification feature label is reasonable, and finally the overall hierarchical classification label is formed.

S300, according to the classification of each submitted information text in step S200, a subset of test cases under the classification is screened from the test case description data set D _{test case} output in step S100, and the similarity between the submitted information text and the test case text is calculated through the SBERT model, and the test cases with high similarity scores are selected as recommended test cases. The SBERT-based model in this step needs to be trained and fine-tuned with a certain amount of data corresponding to the annotated submitted information text and test case text to adapt to the specific environment of the specific project.

As shown in FIG. 4 , the specific steps of S300 include:

S310, converting the submitted information in step S100 and the text data of the filtered test case subset into a high-dimensional vector through the fine-tuned BERT model.

S320, based on the high-dimensional vector representation of the text obtained in step S310, calculate the similarity between the vector representation of the submitted information and the vector representation of each test case description to obtain a recommended test case list.

Step S310 specifically includes:

S311, select the same pre-trained model as step S211, and perform training and fine-tuning based on the labeled submission information and test case text data to adapt to the project-specific text data.

S312, the text output in step S100 is converted into a word list and a corresponding mask list by using the word segmenter corresponding to the BERT model.

S313, input the word list and the corresponding mask list in step S312 into the BERT model and convert them into high-dimensional vectors of submission information and test case descriptions.

S314, select the pooled vector output of the BERT model as the vector representation of the corresponding text.

Step S320 specifically includes:

S321, calculating the similarity between the vector representation of the submitted information and the vector representation of each test case description, wherein the similarity calculation uses the cosine similarity formula:

S322, according to the similarity scores given in step S321, select test cases with higher similarity as recommended test cases, and output a list of recommended test cases.

The embodiments of the present invention can effectively improve the efficiency and accuracy of test case selection. Through intelligent text analysis and similarity calculation, the method can quickly identify test cases closely related to the most recent code submission among a large number of test cases, thereby significantly improving the quality and reliability of software testing.

The embodiments described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims (8)

1. A change semantics-aware whole-machine regression test case recommendation method, characterized by comprising the following steps: (1) Obtain code submission information and test cases. After data cleaning, obtain a text dataset D of code _{submission information} and a description dataset D of _test cases; (2) Using a hierarchical residual multi-granularity classification network model based on a label relationship tree, each submission information text in the text dataset D _submission information is classified into different feature labels according to the hierarchical structure; Among them, the feature label represents the description label of the corresponding function tested by the test case, and its description form is a tree-like multi-level label representation; (3) According to the classification of each submitted information text, a subset of test cases under the classification is selected from _{the test} cases in the description data set D, and the similarity between the submitted information text and the text in the corresponding subset of test cases is calculated, and the cases with high similarity scores are selected as recommended test cases. 2. The change semantics-aware whole-machine regression test case recommendation method according to claim 1, wherein step (2) specifically comprises: (2-1) Through the fine-tuned BERT model, each submission information text in the text dataset D _{submission information} is converted into a high-dimensional vector; (2-2) Convert the label hierarchical relationship in the test case description data set D _{test case} into a tree structure and generate a state vector based on binary labels; (2-3) The high-dimensional text vector obtained in step (2-1) is input into the hierarchical residual multi-granularity classification network model, and the state vector obtained in step (2-2) is constrained to perform hierarchical feature classification on the high-dimensional text vector to determine the hierarchical feature label of each submitted information text. 3. The change semantics-aware whole-machine regression test case recommendation method according to claim 2, characterized in that the specific process of step (2-1) is: (2-1-1) Select the BERT model and use the submitted information text data with labeled features for training and fine-tuning; (2-1-2) Using the word segmenter corresponding to the BERT model, convert the submission information text in the text dataset D _{submission information} into a word list and a corresponding mask list; (2-1-3) Input the word list and the corresponding mask list into the BERT model trained and fine-tuned in step (2-1-1) and convert them into high-dimensional vectors; (2-1-4) Select the pooled vector output of the BERT model as the vector representation of the submitted information text. 4. The change semantics-aware whole-machine regression test case recommendation method according to claim 2, characterized in that the specific process of step (2-2) is: (2-2-1) Design a structured format to store the hierarchical information of the feature tree. The feature label represents the description label of the corresponding function tested by the test case, and its description form is a tree-like multi-level label representation; wherein the test case is divided into any layer of nodes in the feature tree, and at the leaf node or the middle node, different feature nodes represent different test case classifications; (2-2-2) The feature tree calculates all binary label-based state vectors that meet the feature tree constraints to form a reasonable state space; if the total number of feature nodes is N, an N×N matrix is formed. By traversing the feature tree structure, each column of the matrix represents a feature node, and each row represents all possible legal paths in the feature tree, so that the binary-valued hierarchical category label represents the feature node label, that is, the state space is used to define the legal constraints on the relationship between parent-child nodes and sibling nodes. 5. The change semantics-aware whole-machine regression test case recommendation method according to claim 4 is characterized in that the specific process of step (2-3) is: (2-3-1) Calculating the number of layers of the feature tree and the number of features in each layer according to the state space in step (2-2-2) to construct a network structure of a hierarchical residual multi-granularity classification network model; wherein the hierarchical residual multi-granularity classification network model comprises a hierarchical feature transfer module, a residual connection part and a hierarchical classification module; (2-3-2) Establish a hierarchical feature transfer module, which is composed of several fully connected networks or convolutional networks, and is used for feature extraction between different levels and downward transmission; establish a residual connection part, through the sequential connection of coarse-grained parent-class level features and fine-grained child-class level feature layers, to obtain a combined feature vector that is combined from the parent-class feature information layer by layer to the child-class; (2-3-3) Applying a hierarchical classification module based on the combined feature vector, the combined feature vector passes through each level of the classification network, and each dimension of the classification network output corresponds to each feature label; (2-3-4) Set the hierarchical threshold and determine the hierarchical classification of the text based on the vector output by each layer of the classification network. 6. The change semantics-aware whole-machine regression test case recommendation method according to claim 1, wherein step (3) specifically comprises: (3-1) According to the classification of each submission information text, a subset of test cases under the classification is selected from _{the test} cases in the description dataset D; each submission information text in _{the submission information} of the dataset D and the text in the corresponding subset of test cases are converted into high-dimensional vectors through a fine-tuned BERT model; (3-2) Based on the high-dimensional vector representation of the text obtained in step (3-1), calculate the similarity between the vector representation of the submitted information text and the vector representation of each test case description in the corresponding test case set to obtain a list of recommended test cases. 7. The change semantics-aware whole-machine regression test case recommendation method according to claim 6, characterized in that the specific process of step (3-1) is: (3-1-1) Select the BERT model and train and fine-tune it based on the annotated submission information and test case text data; (3-1-2) According to the classification of each submitted information text, a subset of test cases under the classification is selected from _{the test} cases in the description dataset D; through the word segmenter corresponding to the BERT model, the submitted information text and the text of the corresponding subset of test cases are converted into a word list and a corresponding mask list; (3-1-3) Input the word list and the corresponding mask list in step (3-1-2) into the BERT model trained and fine-tuned in step (3-1-1), and convert them into high-dimensional vectors of submission information and test case descriptions; (3-1-4) Select the pooled vector output of the BERT model as the vector representation of the corresponding text. 8. The change semantics-aware whole-machine regression test case recommendation method according to claim 7, characterized in that the specific process of step (3-2) is: (3-2-1) Calculate the similarity between the vector representation of the submitted information and the vector representation of each test case description in the corresponding test case set, where the similarity calculation uses the cosine similarity formula: (3-2-2) According to the similarity scores given in step (3-2-1), test cases with higher similarity are selected as recommended test cases, and a list of recommended test cases is output.