WO2021253904A1

WO2021253904A1 - Test case set generation method, apparatus and device, and computer readable storage medium

Info

Publication number: WO2021253904A1
Application number: PCT/CN2021/081873
Authority: WO
Inventors: 袁文静; 周杰; 卢道和; 方镇举; 翁玉萍; 陈文龙; 黄涛; 韩海燕
Original assignee: 深圳前海微众银行股份有限公司
Priority date: 2020-06-18
Filing date: 2021-03-19
Publication date: 2021-12-23
Also published as: CN111708703A

Abstract

The present application relates to the technical field of fintech. Disclosed are a test case set generation method, apparatus and device, and a computer readable storage medium. The test case set generation method comprises: obtaining a case keyword and performing semantic analysis on the case keyword to obtain the semantic analysis result; searching a test case set knowledge base according to the semantic analysis result to obtain the search result, wherein the test case set knowledge base is generated by training a preset training model obtained by construction by a BERT model by combining a knowledge map; according to the search result, if it is determined that a similar case set exists in the test case set knowledge base, obtaining a target similar case set; and using the knowledge map to analyze the case keyword and the target similar case set to generate a test case set.

Description

Method, device, equipment and computer readable storage medium for generating test case set

Priority information

This application claims the priority of the Chinese patent application filed on June 18, 2020 with the application number 202010563141.X, the entire content of which is incorporated into this application by reference.

Technical field

This application relates to the technical field of financial technology (Fintech), and in particular to a method, device, device, and computer-readable storage medium for generating a test case set.

Background technique

With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually transforming to Fintech. However, due to the security and real-time requirements of the financial industry, higher technology is also proposed. Requirements.

In the process of product development and testing, product developers often need some test cases for development and testing. Test cases usually include usage scenarios and their corresponding test results. At present, test cases are mainly written and maintained manually, which is inefficient and consumes a lot of manpower. At the same time, the existing automated case writing solutions need to save historical cases in the database first, and generate test case sets through database search and matching. Although there is a certain automated process, the database retrieval process can only retrieve existing cases, and the database cannot automatically produce new cases. The input of historical cases in the database needs to be manually entered and the scope of case retrieval is very limited, which is inefficient and impossible Realize the automatic generation of test cases.

Summary of the invention

The main purpose of this application is to provide a test case generation method, device, equipment, and computer-readable storage medium, aiming to realize the automatic generation of test cases and improve the efficiency of test case generation.

In order to achieve the foregoing objective, the present application provides a method for generating a test case set, and the method for generating a test case set includes:

Obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results;

According to the semantic analysis result, search the test case set knowledge base to obtain the retrieval result, wherein the test case set knowledge base is generated by training a preset training model constructed by combining the BERT model and the knowledge graph;

If it is determined according to the search result that there is a similar case set in the test case set knowledge base, then obtain a target similar case set;

The knowledge graph is used to analyze the case keywords and the target similar case set to inferentially generate a test case set.

In an embodiment, before the step of obtaining case keywords, performing semantic analysis on the case keywords, and obtaining a semantic analysis result, the method further includes:

Perform preprocessing training on the preset training model according to the unlabeled first training test case set to obtain the initial training model, where the preset training model is constructed based on the BERT model combined with the knowledge graph;

Performing fine-tuning training on the initial training model according to the labeled second training test case set to obtain a language representation model;

The first training test case set and the second training test case set are classified by the language representation model, and a test case set knowledge base is generated according to the classification result.

In an embodiment, the step of performing preprocessing training on the preset training model according to the unlabeled first training test case set to obtain the initial training model includes:

Acquiring the first attribute information of the unlabeled first training test case set;

Dividing the first training test case set according to the first attribute information to obtain a plurality of first training test case subsets;

Performing preprocessing training on the preset training model according to a plurality of first training test case subsets respectively to obtain a plurality of corresponding initial training models, wherein the preset training model is constructed based on the BERT model combined with the knowledge map;

The step of performing fine-tuning training on the initial training model according to the labeled second training test case set, and obtaining a trained language representation model includes:

Acquiring the labeled second training test case set and its second attribute information;

Divide according to the second attribute information and the second training test set to obtain a plurality of second training test case subsets;

Performing fine-tuning training on the corresponding initial training model according to a plurality of second training test case subsets, respectively, to obtain a plurality of language representation models corresponding to the first attribute information;

The step of classifying the first training test case set and the second training test case set according to the language representation model, and generating a test case set knowledge base according to the classification result includes:

Classify the corresponding first training test case subset and the second training test case subset through multiple language representation models to obtain multiple test case sets corresponding to the first attribute information, and based on the multiple test cases The case set generates a test case set knowledge base.

In one embodiment, obtain candidate attribute information;

According to the semantic analysis result, the step of retrieving the test case collection knowledge base to obtain the retrieval result includes:

Determining a target test case set corresponding to the candidate attribute information in the test case set knowledge base;

The target test case set is retrieved according to the semantic analysis result to obtain the similarity between the case keywords and the test cases in the target test case set, and the retrieval result is obtained.

In one embodiment, after the step of retrieving the target test case set according to the semantic analysis result to obtain the similarity between the case keywords and the test cases in the target test case set, and obtaining the retrieval result, further include:

Detecting whether the similarity in the retrieval result is greater than or equal to a first preset threshold;

If the similarity is greater than or equal to the first preset threshold, it is determined that the same case set corresponding to the case keyword exists in the test case set knowledge base, and the target same case set is obtained and output.

In an embodiment, after the step of detecting whether the similarity in the retrieval result is greater than or equal to a first preset threshold, the method further includes:

If the similarity is less than the first preset threshold, detecting whether the similarity is greater than a second preset threshold, where the second preset threshold is less than the first preset threshold;

If the similarity is greater than the second preset threshold, it is determined that there is a similar case set in the test case set knowledge base, and then the step is performed: obtaining a target similar case set;

If the similarity is less than or equal to the second preset threshold, output prompt information to prompt the user to manually generate a test case set.

In an embodiment, after the step of outputting prompt information to prompt the user to manually generate a test case set, the method further includes:

Obtain a set of labeled test cases manually generated by the user;

Update the test case set knowledge base according to the labeled test case set.

In addition, in order to achieve the above-mentioned purpose, the present application also provides a test case set generating device, and the test case set generating device includes:

The analysis module is used to obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results;

The retrieval module is configured to retrieve the test case set knowledge base according to the semantic analysis result to obtain the retrieval result, wherein the test case set knowledge base is generated by training a preset training model constructed by combining the BERT model and the knowledge graph;

The first obtaining module is configured to obtain a target similar case set if it is determined that there is a similar case set in the test case set knowledge base according to the search result;

The first generating module is configured to analyze the case keywords and the target similar case set by using the knowledge graph to generate a test case set.

In addition, in order to achieve the above object, the present application also provides a test case set generating device, the test case set generating device including: a memory, a processor, and a test stored on the memory and running on the processor A case set generation program, when the test case set generation program is executed by the processor, the steps of the test case set generation method described above are implemented.

In addition, in order to achieve the above object, the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a test case set generation program, and the test case set generation program is executed by a processor to achieve the above The steps of the test case set generation method described.

This application provides a test case collection method, device, equipment, and computer readable storage medium to obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results; and retrieve test cases based on the semantic analysis results A knowledge base to obtain retrieval results, wherein the test case set knowledge base is generated by the training of a preset training model constructed by the BERT model combined with the knowledge graph; if the test case set knowledge base is determined according to the retrieval result If there is a set of similar cases, the target similar case set is obtained; the case keywords and the target similar case set are analyzed using the knowledge graph to generate a test case set. Through the above method, the existing test case set knowledge base can be used to search in the test case set knowledge base according to the case keywords to obtain the search results. Then, according to the target similar case set in the search results, the knowledge graph is used to automatically Reasoning to generate a new set of test cases. Therefore, compared with the prior art, the present application can automatically generate a new test case set based on the existing test case set knowledge base, thereby improving the generation efficiency of the test case set.

Description of the drawings

FIG. 1 is a schematic diagram of a device structure of a hardware operating environment involved in a solution of an embodiment of the application;

FIG. 2 is a schematic flowchart of a first embodiment of a method for generating a test case set of this application;

FIG. 3 is a schematic diagram of the functional modules of the first embodiment of the apparatus for generating a test case set of this application.

The realization, functional characteristics, and advantages of the purpose of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

detailed description

It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.

Referring to FIG. 1, FIG. 1 is a schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the application.

The test case set generating device in the embodiment of the present application may be a smart phone, or a terminal device such as a PC (Personal Computer, personal computer), a tablet computer, and a portable computer.

As shown in FIG. 1, the test case set generating device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to implement connection and communication between these components. The user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a magnetic disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

Those skilled in the art can understand that the structure of the test case set generating device shown in FIG. 1 does not constitute a limitation on the test case set generating device, and may include more or less components than shown in the figure, or a combination of certain components, Or different component arrangements.

As shown in FIG. 1, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a test case set generation program.

In the terminal shown in FIG. 1, the network interface 1004 is mainly used to connect to a back-end server and communicate with the back-end server; the user interface 1003 is mainly used to connect to a client and communicate with the client; and the processor 1001 can be used to Call the test case set generation program stored in the memory 1005, and execute each step of the following test case set generation method.

Based on the above hardware structure, various embodiments of the method for generating a test case set of the present application are proposed.

This application provides a method for generating a test case set.

Referring to FIG. 2, FIG. 2 is a schematic flowchart of a first embodiment of a method for generating a test case set of this application.

In this embodiment, the method for generating a test case set includes:

Step S10, obtaining case keywords, performing semantic analysis on the case keywords, and obtaining semantic analysis results;

The method for generating a test case set of this embodiment is implemented by a test case set generating device, which is equipped with a test case set generator.

In this embodiment, the test case set is some test scenarios that may be used by developers and testers in the product development process and the corresponding expected test results. In one embodiment, it includes test information related to some abnormal scenarios, such as For a login interface, the test case set can include different usage scenarios such as Chinese login name, English login name, and special character login name, as well as test results when tested in these different test scenarios. When a new test case set needs to be generated, the staff can input the case keywords of the test case set they want to generate through the corresponding software on the working end, such as "login", to trigger the test case set generation instruction. At this time, the test case set When the generator receives the test case set generation instruction, it obtains the case keywords input by the staff, and then performs semantic analysis on the case keywords to use the semantic analysis to conduct context-sensitive examinations, thereby obtaining the corresponding semantic analysis results. , The semantic analysis method used in this application is a commonly used semantic analysis method, such as latent semantic analysis.

Step S20: According to the semantic analysis result, search the test case set knowledge base to obtain the search result, wherein the test case set knowledge base is generated by training a preset training model constructed by combining the BERT model and the knowledge graph;

Then, according to the semantic analysis result, the test case set knowledge base is retrieved to obtain the retrieval result.

Among them, the test case set knowledge base contains all product-related case sets, and the test case set knowledge base is generated by training a preset training model constructed by combining the BERT model with the knowledge graph. Among them, BERT (Bidirectional Encoder Representations from Transformers, a natural language processing pre-training technology based on neural networks) can handle natural language semantic analysis, classification and other scenarios very well, but there are some shortcomings, for example, the lack of common sense. Test cases require a large amount of test background knowledge support. What BERT learns is a text matching model. A large amount of test background common sense is implicit and vague, and it is difficult to reflect in the pre-training data. At the same time, it lacks semantic understanding and reasoning. Therefore, the knowledge map information is incorporated in the pre-training process to organize the knowledge under test. The calculation model based on symbolic semantics can provide prior knowledge for BERT, so that it has certain test common sense and reasoning ability. Therefore, in this application, BERT is used in combination with the preset training model training of the knowledge graph to generate a test case set knowledge base, which can enable the test case set knowledge base to know different test common sense and use the test common sense inference to generate a new test case set.

The search result is the similarity between the case keywords and the existing cases in the test case set knowledge base after semantic analysis. According to the similarity, the search results generally include three different search results: identical, similar, and basically different.

Step S30: If it is determined according to the search result that there is a similar case set in the test case set knowledge base, then a target similar case set is obtained.

If it is determined according to the search result that there is a similar case set in the test case set knowledge base, then the target similar case set is obtained. Specifically, for the input case keywords, according to the semantic analysis results, compare with the test set in the test case set knowledge base to obtain the similarity with each test set. Among them, the similarity value with different test sets will be obtained at the same time. The one with the largest similarity is used as the final retrieval result, and the case set corresponding to the largest similarity value is used as the target similar case set.

Step S40, using the knowledge graph to analyze the case keywords and the target similar case set to generate a test case set;

Finally, use the knowledge graph to analyze case keywords and target similar case sets, and generate test case sets by reasoning. The knowledge map is a knowledge domain visualization or a knowledge domain mapping map, which is a series of various graphs showing the relationship between the development process of knowledge and the structure. The reasoning of the knowledge graph includes deductive reasoning and inductive reasoning. Since inductive reasoning can add new knowledge, inductive reasoning is mainly used in this application. Inductive reasoning can also use FOIL (First Order Inductive Learner) algorithm, association rule mining algorithm of incomplete knowledge base, and path sorting algorithm. Specifically, first use one or more of the above algorithms to learn or construct rules for the target similar case set, and then infer new entities based on the case keywords and the entities in the target similar case set according to the learned or constructed rules. The new entity can be constructed by reasoning to get the test case set. If the entered case keyword is the login password, and the existing target similar case set is the login name, the login password and login name can be used to inferentially generate a test case set related to the login password.

The embodiment of the application provides a method for generating a test case set, obtaining case keywords, performing semantic analysis on the case keywords, and obtaining a semantic analysis result; according to the semantic analysis result, searching the test case set knowledge base to obtain the retrieval result , Wherein the test case set knowledge base is generated by training a preset training model constructed by the BERT model combined with the knowledge graph; if it is determined that there is a similar case set in the test case set knowledge base according to the search result, then obtain Target similar case set; use the knowledge graph to analyze the case keywords and the target similar case set, and reason to generate a test case set. Through the above method, the existing test case set knowledge base can be used to search in the test case set knowledge base according to the case keywords to obtain the search results. Then, according to the target similar case set in the search results, the knowledge graph is used to automatically Reasoning to generate a new set of test cases. Therefore, compared with the prior art, the present application can automatically generate a new test case set based on the existing test case set knowledge base, thereby improving the generation efficiency of the test case set.

Further, based on the first embodiment shown in FIG. 2, a second embodiment of the method for generating a test case set of the present application is proposed.

In this embodiment, before step S10, the method for generating a test case set further includes:

Step A: Perform preprocessing training on the preset training model according to the unlabeled first training test case set to obtain the initial training model, where the preset training model is constructed based on the BERT model combined with the knowledge graph;

In this embodiment, the preset training model is preprocessed according to the unlabeled first training test case set to obtain the initial training model, where the preset training model is constructed based on the BERT model combined with the knowledge graph.

Among them, the unlabeled first training test case set is the test case set that has been stored before. The preset training model is constructed based on the BERT model combined with the knowledge graph. BERT can handle natural language semantic analysis, classification and other scenarios very well, but there are some shortcomings, such as the lack of common sense. Test cases require a large amount of test background knowledge support. What BERT learns is a text matching model. A large amount of test background common sense is implicit and vague, and it is difficult to reflect in the pre-training data. At the same time, it lacks semantic understanding and reasoning. Therefore, the knowledge map information is incorporated in the pre-training process to organize the knowledge under test. The calculation model based on symbolic semantics can provide prior knowledge for BERT, so that it has certain test common sense and reasoning ability. BERT conducts pre-training on a large number of test case corpora to realize the understanding of the semantics of the test case text. Specifically, BERT first randomly hides some words under test, and then implements language representation through context prediction to obtain the initial training model. For example, for the sentence "Dylan wrote "Answers in the Wind" in 1962 and "Chronicles: Book One" in 2004, BERT can randomly hide "Dylan" and " 1962", "Answer is flying in the wind" and other words, but through continuous training, the model can determine the relationship between these words and can store the relationship between these words, so that the model can know the relationship between words The relationship of the linguistic representation. It should be noted that before the pre-processing training process, the BERT is combined with the knowledge graph, and the multi-information entities in the knowledge graph are used (such as Dylan in the above example, "Answers in the Wind" and other specific examples) Used as external knowledge to improve language representation, so that the model can know the meaning of each word itself, not just the relationship between multiple words, and at the same time achieve structured knowledge coding and heterogeneous information fusion (among which, structured knowledge The way of encoding is to transform abstract knowledge into vectors and other forms for language representation. However, because the vector representation spaces of knowledge and text are different, the text and knowledge are merged into a unified feature space through a preset efficiency model; Heterogeneous information refers to different types of information such as vocabulary, syntax, and knowledge information), which realizes that on the basis of the BERT model, a preset training model is constructed by fusing the knowledge graph. Specifically, for abstract knowledge information, they need to be encoded so that knowledge can be used for language representation. At the same time, the encoding of words and the encoding of knowledge during BERT pre-training are different, although they are both converted into The vector is located in a different vector space. Therefore, it is necessary to design the model to realize the fusion of heterogeneous information such as vocabulary, syntax and knowledge information. The BERT combined with the knowledge graph model can solve the above problems.

Step B: Perform fine-tuning training on the initial training model according to the labeled second training test case set to obtain a language representation model;

Then, the initial training model is fine-tuned and trained according to the labeled second training test case set to obtain the language representation model.

Among them, the labeled second training test set is a training test set that is not in the existing first training test case set, such as a test case set of a new login name. The labeled second training test case set can supplement the first training test set, so that the resulting language representation model is more comprehensive, and a test case set knowledge base that is more in line with the real test scenario can be constructed.

The fine-tuning training process is completed by two modules: text encoder and knowledge encoder. The text encoder is responsible for obtaining the semantic information such as the morphology and syntax of the input tags of the second training test case set, and the tag vector, segmentation vector and position vector are summed to obtain the input vector, and then implemented by the multi-layer two-way conversion encoder For the extraction of semantic features. The knowledge encoder integrates additional entity-oriented knowledge information into the text information from the bottom layer, so that the heterogeneous information of tags and entities can be represented in a unified feature space. Represents vector sequences labeled by _{_{{w 1, ..., w n}} }, with _{_{{e 1, ..., e n}} } be a vector representing the sequence entity. The two sequences are calculated according to the following formula:

Among them, MH-ATT is the attention layer.

Then align the marks in the sequence with the corresponding entities (the entities are aligned with the corresponding first mark), and then input such a sequence into the information fusion layer. The calculation steps of the information fusion layer are as follows:

Among them, h _j represents the internal hidden state of the fusion mark and entity information, b represents the bias, W _t represents the weight in the hidden layer, and σ() is the non-linear activation function.

Through the above-mentioned fine-tuning process, the language representation model is obtained, so that the test case set knowledge base can be obtained subsequently based on the language representation model.

Step C: Classify the first training test case set and the second training test case set through the language representation model, and generate a test case set knowledge base according to the classification result;

Finally, the first training test case set and the second training test case set are classified through the language representation model, and the test case set knowledge base is generated according to the classification results. Specifically, the language representation model obtains the classification of different training test case sets based on the probability distribution calculation formula, and finally generates the test case set knowledge base, where the predicted probability distribution calculation formula is as follows:

Among them, linear() represents the linear layer.

That is, to classify each case in different training test case sets, combine the cases belonging to the same type into the same type of case, and then construct the test case set knowledge base from different types of cases, and then, for example, for the user name and password, both are It can be a landing case.

In this embodiment, preprocessing training is performed on the preset training model according to the unlabeled first training test case set to obtain the initial training model, where the preset training model is constructed based on the BERT model combined with the knowledge graph; Perform fine-tuning training on the initial training model according to the labeled second training test case set to obtain a language representation model; classify the first training test case set and the second training test case set through the language representation model , According to the classification results to generate a test case set knowledge base. According to the pre-training process and fine-tuning the training process, the training model is trained to obtain the test case set knowledge base, which realizes the classification of different test case sets to facilitate subsequent retrieval, thereby improving the efficiency of subsequent test case set generation. In addition, through pre-training The training process and fine-tuning the training process to train the model, and then classify the training test case set according to the trained model, and finally generate a test case set knowledge base. Compared with the traditional manual writing test case, this application is more relevant to the case of the example. Improved the accuracy of test case generation.

Further, based on the foregoing embodiments, a third embodiment of the method for generating a test case set of the present application is proposed.

In this embodiment, step A includes:

Step a1: Obtain the first attribute information of the unlabeled first training test case set;

Step a2, dividing the first training test case set according to the first attribute information to obtain multiple first training test case subsets;

Step a3: Perform preprocessing training on the preset training model according to a plurality of first training test case subsets to obtain corresponding multiple initial training models, wherein the preset training model is constructed based on the BERT model combined with the knowledge map of.

In this embodiment, the training test case set (including the first training test case set and the second training test case set) can be divided according to the attribute information, so as to train to obtain multiple language representation models corresponding to different attribute information, and then combine the language The classification results and attribute information of the characterization model are classified, and the test case set knowledge base is constructed.

Specifically, first obtain the first attribute information of the unlabeled first training test case set; then, divide the first training test case set according to the first attribute information to obtain a plurality of first training test case subsets.

In this embodiment, the training model input source is composed of four parts: the test knowledge public database, the BUG database, the business scenario database, and the training database. The test knowledge public database is mainly common test cases with business commonality, such as login and password verification. The BUG database is a set of BUG use cases found in production; the business scenario library is a collection of test cases written in a specific business scenario, and the training database is a set of test cases manually annotated on the TCTP platform. The training parameter case set will be trained according to the training data of three latitudes: full product cases, specific project product cases, and personalized writing cases. Correspondingly, the first attribute information can include full product cases, specific project product cases, and personalized writing cases. And other different attributes. The full product case is, for example, a set of test cases for a type of product such as insurance, and the project product case is a set of test cases for a specific product such as login, and a personalized insurance case can be a set of test cases associated with each writer.

For the first training test case subset with different attributes, the classification results of the same case in the initial training model formed by it may be different, and the association relationship between different entities may be different. In this way, the preset training models are preprocessed according to a plurality of first training test case subsets respectively to obtain corresponding multiple initial training models, where the preset training models are constructed based on the BERT model combined with the knowledge graph.

At this point, step B includes:

Step b1: Obtain the labeled second training test case set and its second attribute information;

Step b2, dividing according to the second attribute information and the second training test set to obtain a plurality of second training test case subsets;

Step b3: Perform fine-tuning training on the corresponding initial training model according to a plurality of second training test case subsets, respectively, to obtain multiple language representation models corresponding to the second attribute information.

For the labeled second training test case set, similar to the first training test case set, according to the second attribute information of the second training test case set, the initial training model that matches the second attribute information is determined, and the second training test In the case set, the second attribute information is added to the corresponding initial training model for fine-tuning training, and multiple language representation models are obtained.

Generate a language representation model based on the above method, and determine different language representation models based on the attribute information, so that models of different attributes can be obtained, and the classification results of the obtained language representation models are more accurate, and the diversity of language representation models is increased to adapt to different tests. Scenes.

At this point, step C includes:

Step c1: Classify the corresponding first training test case subset and the second training test case subset through multiple language representation models to obtain multiple test case sets corresponding to the first attribute information, and based on the Multiple test case sets generate test case set knowledge base;

Multiple language representation models with different attributes are combined to form the device's test case set knowledge base. The test case set knowledge base includes cases, attributes (full product cases, specific project product cases, personalized product cases, personalized writing cases) and test sets. For different test sets, they will be classified into corresponding cases. At the same time, for the same test set, according to different attributes, different cases may correspond to different cases in the test case set sub-knowledge base of different attributes.

In this embodiment, multiple speech representation models with different attributes are formed into the final test case set knowledge base, so as to ensure the integrity of the test case set knowledge base, and also enable the test case set to match more usage scenarios according to different attributes. It further improves the accuracy of the test case set generation. At the same time, when the staff searches, the search range can be narrowed based on the input candidate attribute information, and the retrieval efficiency is improved, thereby improving the generation efficiency of the test case set.

Further, based on the foregoing embodiments, a fourth embodiment of the method for generating a test case set of the present application is proposed.

In this embodiment, after the above step S20, the method for generating a test case set further includes:

Step D, obtain candidate attribute information;

In this embodiment, when the worker triggers the test case set generation instruction, in addition to the case keywords, he can also input candidate attribute information, where the candidate attribute information is the attribute information corresponding to the test case set to be generated At the same time, the candidate attribute information corresponds to the attribute information of each language representation model of the test case set knowledge base, that is, the candidate attribute information is used to give the associated test case set during retrieval. Correspondingly, the test case set generator can first Get candidate attribute information.

Step S20 includes:

Step E: Determine a target test case set corresponding to the candidate attribute information in the test case set knowledge base;

Step F: Retrieve the target test case set according to the semantic analysis result to obtain the similarity between the keyword of the case and the test case in the target test case set to obtain the retrieval result;

Then, determine the target test case set corresponding to the candidate attribute information in the test case set knowledge base, and then retrieve the target test case set according to the semantic analysis result to obtain the similarity between the case keywords and the test cases in the target test case set, and get search result.

When one or more candidate attribute information is selected, the output result gives a set of test cases that conform to the attribute according to the cases associated with the test attribute. At the same time, in the retrieval process, according to the selected candidate attribute information, only the language representation model in the test case set knowledge base whose attribute information is the same as the candidate attribute information is retrieved. If the candidate attribute is a specific project product case set, only the test case set knowledge base whose attribute is a specific product case set is retrieved, instead of retrieving the full product case and personalized writing case, the results can be retrieved at the same time through the efficiency of the retrieval process Also more accurate. According to the similarity between the case keywords and the test cases in the test case set of the corresponding attributes, the corresponding retrieval results are determined.

Through the above method, this embodiment can narrow the range of the test case set knowledge base that needs to be retrieved according to the candidate attribute information in the retrieval process, thereby passing the efficiency and accuracy of retrieval.

Further, based on the foregoing embodiments, a fifth embodiment of the method for generating a test case set of the present application is proposed.

In this embodiment, step S20 includes:

Step G, detecting whether the similarity in the retrieval result is greater than or equal to a first preset threshold;

Step H: If the similarity is greater than or equal to the first preset threshold, it is determined that the same case set corresponding to the case keyword exists in the test case set knowledge base, then the target same case set is obtained, and Output

In this embodiment, it is detected whether the similarity in the search result is greater than or equal to the first preset threshold. When the similarity is greater than or equal to the first preset threshold, it indicates that the current test case set knowledge base already exists and the input case keyword For the same case set, the target same case set is directly determined according to the similarity and output, and then the required test case set can be output.

Further, after step H, it also includes:

Step 1: If the similarity is less than the first preset threshold, detecting whether the similarity is greater than a second preset threshold, where the second preset threshold is less than the first preset threshold;

When the similarity is less than the first preset threshold, it means that the same test case set does not exist in the test case set knowledge base, but the test case set knowledge base generated by the BERT combined with the training model of the knowledge graph has a certain learning ability. Next, determine whether there are similar cases, that is, determine whether the similarity is greater than the second preset threshold.

Step J: If the similarity is greater than the second preset threshold, it is determined that there is a similar case set in the test case set knowledge base, and step S30 is executed: obtaining a target similar case set;

When it is greater than the second preset threshold, although the same test set does not exist in the test case set knowledge base, but there is a similar case set associated with the case keyword, then based on the case keyword and the target similar case set, the knowledge graph is used for reasoning Ability reasoning generates and generates a set of test cases. If the case keyword is the user password, and it is determined that there is a similar test set in the test case set knowledge base as the user name related test set, if it does not contain special characters, the length is at least six characters, etc., it can be inferred that the user password does not contain A set of test cases with special characters and a length of at least six characters.

Step K: If the similarity is less than or equal to the second preset threshold, output prompt information to prompt the user to manually generate a test case set;

When the similarity is less than or equal to the second preset threshold, it means that the test case in the test case set knowledge base differs greatly from the input case keywords. The case corresponding to the input case keywords should be a brand new case. It is impossible to use the existing test case set for direct output or reasoning to generate a test case set. If the user needs to manually generate the test case set, then manually add the test case set.

Further, after step K, it also includes:

Step k1: Obtain a set of labeled test cases manually generated by the user;

Step k2, update the test case set knowledge base according to the labeled test case set;

When the user manually enters a new labeled test case set, record the labeled test case set manually generated by the user, and input the labeled test case set as a new labeled test case set to update the test case set knowledge base, so that The test case set knowledge base can learn from the labeled test case set, thereby expanding the test case set knowledge base.

In this embodiment, the same test case set can be directly output according to the similarity according to the retrieval result, or the test case set can be generated by reasoning based on the similar test case set. When the test case set cannot be output according to the test case set knowledge base, it can be manually generated Annotated test case set is generated in the method, and then the test case set knowledge base is updated by annotated test case set to expand the test case set knowledge base.

The application also provides a device for generating a test case set.

Referring to FIG. 3, FIG. 3 is a schematic diagram of the functional modules of the first embodiment of the apparatus for generating a test case set according to the present application.

As shown in Figure 3, the test case set generating device includes:

The analysis module 10 is used to obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results;

The retrieval module 20 is configured to retrieve the test case set knowledge base according to the semantic analysis result to obtain the retrieval result, wherein the test case set knowledge base is generated by training a preset training model constructed by combining the BERT model and the knowledge graph ；

The first obtaining module 30 is configured to obtain a target similar case set if it is determined that there is a similar case set in the test case set knowledge base according to the search result;

The first generation module 40 is configured to analyze the case keywords and the target similar case set by using the knowledge graph, and generate a test case set by reasoning.

Further, the test case set generating device further includes:

The pre-training module performs pre-processing training on the preset training model according to the unlabeled first training test case set to obtain the initial training model, where the preset training model is constructed based on the BERT model combined with the knowledge map;

The fine-tuning training module is configured to perform fine-tuning training on the initial training model according to the labeled second training test case set to obtain a language representation model;

The second generation module is configured to classify the first training test case set and the second training test case set through the language representation model, and generate a test case set knowledge base according to the classification result.

Further, the pre-training module further includes:

The first acquiring unit is configured to acquire the first attribute information of the unlabeled first training test case set;

The first dividing unit is configured to divide the first training test case set according to the first attribute information to obtain multiple first training test case subsets;

The pre-training unit is used to perform pre-processing training on the preset training model according to a plurality of first training test case subsets to obtain corresponding multiple initial training models, wherein the preset training model is based on the BERT model combined with knowledge The map is constructed;

The fine-tuning training module further includes:

The second acquiring unit is used to acquire the labeled second training test case set and its second attribute information;

The second dividing unit is configured to divide according to the second attribute information and the second training test set to obtain a plurality of second training test case subsets;

The fine-tuning training unit is configured to perform fine-tuning training on the corresponding initial training model according to a plurality of second training test case subsets to obtain multiple language representation models corresponding to the second attribute information;

The second generating module further includes:

The first generating unit is configured to classify the corresponding first training test case subset and the second training test case subset through multiple language representation models to obtain multiple test case sets corresponding to the first attribute information, And generate a test case set knowledge base based on the multiple test case sets.

Further, the test case set generating device further includes:

The second obtaining unit is used to obtain candidate attribute information;

The first acquisition module further includes:

A determining unit, configured to determine a target test case set corresponding to the candidate attribute information in the test case set knowledge base;

The third obtaining unit is configured to retrieve the target test case set according to the semantic analysis result to obtain the similarity between the case keywords and the test cases in the target test case set to obtain the retrieval result.

Further, the test case set generating device further includes:

The first detection module is configured to detect whether the similarity in the retrieval result is greater than or equal to a first preset threshold;

The first output module is configured to determine that the same case set corresponding to the case keyword exists in the test case set knowledge base if the similarity is greater than or equal to the first preset threshold, and the acquisition target is the same Case collection and output.

Further, the test case set generating device further includes:

The second detection module is configured to detect whether the similarity is greater than a second preset threshold if the similarity is less than the first preset threshold, wherein the second preset threshold is less than the first preset Set threshold

The fourth obtaining module is configured to determine that there is a similar case set in the test case set knowledge base if the similarity is greater than the second preset threshold, and then execute the step of: obtaining a target similar case set;

The second generation module is configured to output prompt information to prompt the user to manually generate a test case set if the similarity is less than or equal to the second preset threshold.

Further, the test case set generating device further includes:

The fifth acquisition module is used to acquire a set of labeled test cases manually generated by the user;

The update module is used to update the test case set knowledge base according to the labeled test case set.

Among them, the function realization of each module in the above-mentioned test case set generation device corresponds to each step in the above-mentioned test case set generation method embodiment, and its functions and realization processes are not repeated here.

The present application also provides a computer-readable storage medium with a test case set generation program stored on the computer-readable storage medium. The test case set generation program is executed by a processor to achieve the above The steps of the test case set generation method.

The specific embodiments of the computer-readable storage medium of the present application are basically the same as the embodiments of the above-mentioned test case set generation method, and will not be repeated here.

It should be noted that in this article, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, It also includes other elements not explicitly listed, or elements inherent to the process, method, article, or system. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the process, method, article, or system that includes the element.

The serial numbers of the foregoing embodiments of the present application are for description only, and do not represent the superiority or inferiority of the embodiments.

Through the description of the above implementation manners, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is better.的实施方式。 Based on this understanding, the technical solution of this application essentially or the part that contributes to the existing technology can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as ROM/RAM) as described above. , Magnetic disk, optical disk), including several instructions to make a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the method described in each embodiment of the present application.

The above are only the preferred embodiments of this application, and do not limit the scope of this application. Any equivalent structure or equivalent process transformation made using the content of the description and drawings of this application, or directly or indirectly used in other related technical fields , The same reason is included in the scope of patent protection of this application.

Claims

A method for generating a test case set, wherein the method for generating a test case set includes:

Obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results;

According to the semantic analysis result, search the test case set knowledge base to obtain the search result, wherein the test case set knowledge base is generated by training a preset training model constructed by combining the BERT model and the knowledge graph;

If it is determined according to the search result that there is a similar case set in the test case set knowledge base, then obtain a target similar case set;

Use the knowledge graph to analyze the case keywords and the target similar case set to generate a test case set.
The method for generating a test case set according to claim 1, wherein before the step of obtaining case keywords, performing semantic analysis on the case keywords, and obtaining a semantic analysis result, the method further comprises:

Perform preprocessing training on the preset training model according to the unlabeled first training test case set to obtain the initial training model, where the preset training model is constructed based on the BERT model combined with the knowledge graph;

Performing fine-tuning training on the initial training model according to the labeled second training test case set to obtain a language representation model;

The first training test case set and the second training test case set are classified by the language representation model, and a test case set knowledge base is generated according to the classification result.
The method for generating a test case set according to claim 2, wherein the step of performing preprocessing training on a preset training model according to the unlabeled first training test case set to obtain an initial training model comprises:

Acquiring the first attribute information of the unlabeled first training test case set;

Dividing the first training test case set according to the first attribute information to obtain a plurality of first training test case subsets;

Performing preprocessing training on the preset training model according to a plurality of first training test case subsets respectively to obtain a plurality of corresponding initial training models, wherein the preset training model is constructed based on the BERT model combined with the knowledge map;

The step of performing fine-tuning training on the initial training model according to the labeled second training test case set, and obtaining a trained language representation model includes:

Acquiring the labeled second training test case set and its second attribute information;

Divide according to the second attribute information and the second training test set to obtain a plurality of second training test case subsets;

Performing fine-tuning training on the corresponding initial training model according to a plurality of second training test case subsets, respectively, to obtain a plurality of language representation models corresponding to the second attribute information;

The step of classifying the first training test case set and the second training test case set according to the language representation model, and generating a test case set knowledge base according to the classification result includes:

Classify the corresponding first training test case subset and the second training test case subset through multiple language representation models to obtain multiple test case sets corresponding to the first attribute information, and based on the multiple test cases The case set generates a test case set knowledge base.
The method for generating a test case set according to claim 3, wherein, before the step of retrieving the test case set knowledge base to obtain the retrieval result according to the semantic analysis result, the method further comprises:

Obtain candidate attribute information;

According to the semantic analysis result, the step of retrieving the test case collection knowledge base to obtain the retrieval result includes:

Determining a target test case set corresponding to the candidate attribute information in the test case set knowledge base;

The target test case set is retrieved according to the semantic analysis result to obtain the similarity between the case keywords and the test cases in the target test case set, and the retrieval result is obtained.
4. The method for generating a test case set according to claim 4, wherein the retrieval of the target test case set according to the semantic analysis result to obtain the similarity between the case keywords and the test cases in the target test case set , After the step of obtaining the search results, it also includes:

Detecting whether the similarity in the retrieval result is greater than or equal to a first preset threshold;

If the similarity is greater than or equal to the first preset threshold, it is determined that the same case set corresponding to the case keyword exists in the test case set knowledge base, and the target same case set is obtained and output.
5. The test case generation method according to claim 5, wherein after the step of detecting whether the similarity in the retrieval result is greater than or equal to a first preset threshold, the method further comprises:

If the similarity is less than the first preset threshold, detecting whether the similarity is greater than a second preset threshold, where the second preset threshold is less than the first preset threshold;

If the similarity is greater than the second preset threshold, it is determined that there is a similar case set in the test case set knowledge base, and then the step is performed: obtaining a target similar case set;

If the similarity is less than or equal to the second preset threshold, output prompt information to prompt the user to manually generate a test case set.
8. The method for generating a test case set according to claim 6, wherein after the step of outputting prompt information to prompt the user to manually generate the test case set, the method further comprises:

Obtain a set of labeled test cases manually generated by the user;

Update the test case set knowledge base according to the labeled test case set.
A test case set generating device, wherein the test case set generating device includes:

The analysis module is used to obtain case keywords, perform semantic analysis on the case keywords, and obtain semantic analysis results;

The retrieval module is configured to retrieve the test case set knowledge base to obtain the retrieval result according to the semantic analysis result, wherein the test case set knowledge base is generated by training a preset training model constructed by combining the BERT model with the knowledge graph;

The first obtaining module is configured to obtain a target similar case set if it is determined that there is a similar case set in the test case set knowledge base according to the search result;

The first generating module is used to analyze the case keywords and the target similar case set by using the knowledge graph to generate a test case set.
A test case set generating device, wherein the test case set generating device includes: a memory, a processor, and a test case set generating program stored on the memory and running on the processor, the test case When the set generation program is executed by the processor, the steps of the test case set generation method according to any one of claims 1 to 7 are realized.
A computer-readable storage medium, wherein a test case set generation program is stored on the computer-readable storage medium, and when the test case set generation program is executed by a processor, the test case set generation program is implemented as described in any one of claims 1 to 7 The steps of the test case set generation method described.