CN115756907A - Uncertain test result judgment method and system based on keywords - Google Patents

Abstract

Some embodiments of the application provide a method and a system for judging uncertain test results based on keywords. The keyword library is a keyword classification set formed by keywords of the product in each field. And inputting the target use case into the equipment to be tested to obtain target return information of the target use case. And analyzing the target return information, and decomposing the target return information into a plurality of sub-target return information. And extracting corresponding return information from the keyword library according to the test object and the error type to generate a verification set. And verifying the sub-target return information through the verification set respectively, and if the sub-target return information is in the range of the verification set, judging that the test result is correct, so that the judgment efficiency of the test result is improved.

Description

Uncertain test result judgment method and system based on keywords

Technical Field

The application relates to the technical field of computer network communication, in particular to an uncertain test result judgment method and system based on keywords.

Background

Product testing is an important link for ensuring product quality and user experience. In product testing, automated testing is a common testing method. The automatic test is to execute a test case in a product to be tested, and further to judge a test conclusion according to an actual measurement result. When writing the test case, the test result can be clarified, so that the test result can be verified conveniently.

In some scenarios, such as validity check of input information, different error cases may be automatically generated based on the input type. However, due to different test scenarios, when the same test type, test object, and return information of the same error case express related errors from a semantic level, there is a problem that text contents are not completely consistent, and further, it is difficult to accurately determine the result part of the case.

In order to judge the accuracy of the test result, the verification information is acquired after the case is generated, and the test result can be verified and identified one by one through manual work. However, the manual inspection method consumes a lot of human resources, and the verification speed is slow, which results in low efficiency of determining the test result of the use case.

Disclosure of Invention

The application provides an uncertain test result judgment method and system based on keywords, and aims to solve the problem of low test result judgment efficiency of automatic testing.

In a first aspect, the present application provides a method for determining an uncertain test result based on a keyword, comprising:

acquiring a keyword library and input information, wherein the keyword library is a keyword classification set formed by keywords of products in various fields;

generating a target case according to the input information, wherein the target case comprises the title information, the error type, the test object and the data to be tested of the test object;

inputting the target use case into equipment to be tested to obtain target return information;

analyzing the target return information and decomposing the target return information into a plurality of sub-target return information;

extracting return information from the keyword library according to the test object and the error type to generate a verification set;

if the sub-target return information is in the range of the verification set, judging that the test result is correct;

and if the sub-target return information is not in the range of the verification set, judging that the test result is an error.

The target return information is decomposed into a plurality of sub-target return information, and each sub-target return information is checked based on the keyword library. When all the sub-target return information is in the range of the verification set, judging that the test result is correct; if any sub-target return information is not in the range of the verification set, the testing result is judged to be an error, and further the testing efficiency can be improved.

With reference to the first aspect, in an implementation manner of the first aspect, return information of different use cases is obtained; decomposing the return information into a plurality of sub-return information, wherein the sub-return information comprises title information, operation information, re-operation information and prompt information; and storing the sub-return information into the keyword library so as to extract the sub-return information from the keyword library.

With reference to the first aspect, in an implementation manner of the first aspect, the obtaining return information of different use cases further includes: analyzing the return information; marking the test object information in the return information as header information; marking the operation key words, the operation objects and the operation results in the return information as operation information; the re-operation key words and the re-operation objects in the return information are marked as re-operation information; and marking the prompting words and the allowable range information in the return information as prompting information.

With reference to the first aspect, in an implementation manner of the first aspect, the decomposing the target return information into a plurality of sub-target return information includes: and decomposing the target return information into target title information, target operation information, target re-operation information and target prompt information according to a preset information format so as to check the target sub-return information conveniently.

With reference to the first aspect, in one possible implementation manner of the first aspect, the target header information is matched with header information in the verification set; and if the target title information does not accord with the title information of the test object, judging that the test result is an error.

With reference to the first aspect, in one implementable manner of the first aspect, the target operation information is matched with the operation information in the verification set; and if the operation information is not in the operation range of the test object and the error type, judging that the test result is an error.

With reference to the first aspect, in one implementable manner of the first aspect, the target re-operation information is matched with the re-operation information in the verification set; and if the heavy operation information is not in the heavy operation range of the test object and the error type, judging that the test result is an error.

With reference to the first aspect, in an implementation manner of the first aspect, the target hint information is matched with hint information in the verification set; and if the target title information does not accord with the prompt information of the test object, judging that the test result is wrong.

With reference to the first aspect, in an implementation manner of the first aspect, the validity of the input information is checked according to a data class, and the input information is parsed; and generating an error case according to the input type of the input information.

In a second aspect, the present application further provides an uncertain test result determination system based on a keyword, including: the device comprises an input module, a processing module and an output module. Wherein:

the input module is configured to acquire a keyword library and input information, wherein the keyword library is a keyword classification set formed by keywords of products in various fields; generating a target case according to the input information, wherein the target case comprises the title information, the error type, the test object and the data to be tested of the test object; inputting the target use case into equipment to be tested to obtain target return information;

the processing module is configured to analyze the target return information according to a preset format and decompose the return information into a plurality of sub-target return information; extracting return information from the keyword library according to the test object and the error type to generate a verification set;

the output module is configured to judge that the test result is correct if the sub-target return information is in the range of the verification set; and if the sub-target return information is not in the range of the verification set, judging that the test result is an error.

According to the technical scheme, the uncertain test result judging method and system based on the keywords can generate the target case according to the input information by acquiring the keyword library and the input information. The keyword library is a keyword classification set formed by keywords of the product in each field. And inputting the target use case into the equipment to be tested to obtain target return information of the target use case. And analyzing the target return information, and decomposing the target return information into a plurality of sub-target return information. And extracting corresponding return information from the keyword library according to the test object and the error type to generate a verification set. And verifying the sub-target return information through the verification set respectively, and if the sub-target return information is in the range of the verification set, judging that the test result is correct, so that the judgment efficiency of the test result is improved.

Drawings

In order to more clearly explain the technical solution of the present application, the drawings needed to be used in the embodiments are briefly described below, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

Fig. 1 is a schematic flowchart of a method for determining an uncertain test result based on a keyword according to an embodiment of the present application;

fig. 2 is a schematic flowchart of decomposing target return information according to the embodiment of the present application;

fig. 3 is a schematic flowchart of matching header information according to an embodiment of the present application;

fig. 4 is a schematic flowchart of matching operation information provided in an embodiment of the present application;

fig. 5 is a schematic flow chart illustrating matching re-operation information according to an embodiment of the present disclosure;

fig. 6 is a schematic flow chart of matching prompt information provided in the embodiment of the present application;

fig. 7 is an exemplary diagram of an uncertain test result determination process based on a keyword according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a system for determining uncertain test results based on keywords according to an embodiment of the present application.

Detailed Description

Reference will now be made in detail to embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following examples do not represent all embodiments consistent with the present application. But merely as exemplifications of systems and methods consistent with certain aspects of the application, as recited in the claims.

The automatic test is an important means for product detection, and mainly determines a test conclusion according to an actual measurement result by executing a case in equipment to be tested so as to realize the research and development confirmation of a product and the quality inspection of the product. Before testing, it is necessary to analyze how the test is performed and output a text scheme for use in the test operation, where the analysis process is called test analysis and the output text scheme is called test case.

Thus, in some embodiments, a target use case is generated from the input information; and inputting the target case into the equipment to be tested, and obtaining the test result of the target case, so that the test conclusion of the equipment to be tested can be judged according to the test result. The target case can determine the range, depth, breadth, normal and abnormal conditions and the like of the test operation.

However, in some scenarios, when the return information of the same test type, the same test object, and the same error case express a relevant error from a semantic level, the text content is not completely consistent, which further causes that the test result of the case cannot be accurately generated, and reduces the test efficiency.

Therefore, the test result of the use case needs to be determined. However, when the test result is determined, if the test result is verified manually, the efficiency of the test may be reduced. If the test results are identified through a machine learning algorithm, the amount of data generated for each data type under different scenes is small. It is difficult to generate reliable results for unknown information, since the amount of data generated by machine learning algorithms is insufficient to support the building of the model, resulting in overfitting of the data.

Based on the application scenario, in order to alleviate the problem of low judgment efficiency of case test results, as shown in fig. 1, the application provides a method for judging uncertain test results based on keywords, which includes the following steps:

s100: and acquiring a keyword library and input information.

Before testing, input information and a keyword library are obtained, a target case to be executed by the device to be tested is determined through the input information, and the keyword library is used as a judgment basis for a test result. The keyword library is an experience database and is a keyword classification set formed by keywords of products in various fields. That is, the keyword library is formed according to the conventional terms in each field of products, and the use case is returned as a proper supplement.

In order to facilitate the judgment of the test result through the keyword library, in some embodiments, the return information of different use cases, that is, the return information of different use case devices, is obtained. The return information may be decomposed into a plurality of sub-return information, wherein the sub-return information includes header information, operation information, re-operation information, and prompt information. The returned information can be decomposed into four parts of title information, operation information, re-operation information and prompt information by analyzing the returned information of different cases, and then the decomposed four parts of information are stored in a keyword library to be used as a judgment basis.

In order to facilitate analysis of return information of different use cases, in some embodiments, when the return information of different use cases is obtained, the return information is also analyzed; marking the test object information in the return information as header information; marking the operation keywords, the operation objects and the operation results in the return information as operation information; marking the re-operation keywords and the re-operation objects in the return information as re-operation information; and marking the prompting words and the allowable range information in the return information as prompting information.

That is, the return information stored in the keyword library is divided into four parts, which are header information, operation information, re-operation information, and prompt information. Wherein, the title information is the related information of the tested object; the operation information consists of three parts of an operation keyword, an operation object and an operation result; the re-operation information consists of a re-operation keyword and a re-operation object; the prompt message is composed of a prompt word and allowable range information.

S200: and generating a target use case according to the input information.

And after the input information is acquired, generating a target case corresponding to the input information according to the input information. The target case comprises the title information, the error type, the test object and the data to be tested of the test object.

S300: and inputting the target use case into the equipment to be tested to obtain target return information.

And after the target use case is generated, executing the target use case on the equipment to be tested. After the target use case is executed by the equipment to be tested, the execution result of the target use case can be automatically output, and the execution result is returned. In the embodiment of the application, the target return information is information data of the execution result of the target use case, and the target return information can also represent the test result of the target use case.

S400: and analyzing the target return information and decomposing the target return information into a plurality of sub-target return information.

After the target return information is obtained, the test result needs to be judged, and whether the current test result is correct or not is further judged. Therefore, it is necessary to analyze the target return information and decompose the target return information into a plurality of pieces of sub-target return information. And taking the returned information of each sub-target as a judgment unit to check the returned information of each sub-target through the keyword library, so that the judgment efficiency of the test result is improved.

In some embodiments, in decomposing the target return information, as shown in fig. 2, the target return information is decomposed into target header information, target operation information, target re-operation information, and target prompt information according to a preset information format. Wherein, the target title information is the related information of the target test object; the target operation information comprises a target operation keyword, a target operation object and a target operation result; the target heavy operation information comprises target heavy operation keywords and target heavy operation objects; the prompt message comprises a target prompt word and target allowable range information.

For example: the preset information format comprises title information, operation information, re-operation information and prompt information. After the target return information is obtained, the target return information is decomposed into four parts according to the formats of the title information, the operation information, the re-operation information and the prompt information, so that four keyword sets corresponding to the information are generated.

S500: and extracting return information from the keyword library according to the test object and the error type to generate a verification set.

And after the target return information is analyzed, extracting corresponding return information from the keyword library by taking the test object and the error type as indexes. The test object can be object information such as character strings, numbers, IP addresses and the like; the error type may be error information such as length error, character set error, range error, etc. According to the indexes of the two information, corresponding return information can be extracted from the keyword library. And combining the keyword set formed by the extracted return information into a verification set, namely verifying the sub-target return information through the verification set respectively.

S600: and if the sub-target return information is in the range of the verification set, judging that the test result is correct.

And after the verification set is generated, matching the sub-target return information with the corresponding keywords in the verification set. And if all the sub-target return information is in the range of the verification set, judging that the test result is correct.

S700: and if the sub-target return information is not in the range of the verification set, judging that the test result is an error.

Similarly, after the verification set is generated, the sub-target return information is matched with the corresponding keywords in the verification set. And if any sub-target return information is not in the range of the verification set, judging that the test result is an error.

It can be known from steps S600 and S700 that the method for determining an uncertain test result based on a keyword in the embodiment of the present application can respectively determine sub-target return information, and determine that a test result is correct when all sub-target return information is within a verification set range, so that the test result can be automatically determined under the condition that the test result is uncertain, thereby improving the test efficiency.

Since the information types of the returned information of the sub-targets are different, the returned information of each sub-target needs to be checked and compared separately. Thus, in some embodiments, the target header information is matched to header information in the verification set, as shown in FIG. 3. And if the target title information does not accord with the title information of the test object, judging that the test result is an error. After the target title information and the title information of the verification set are obtained, the target title information is compared with the title information of the verification set. If the header information of the two are consistent, the target header information is judged to be correct. It should be noted that, at this time, only the header information can be determined to be correct, and whether the test result is correct cannot be determined. However, if the header information of the two are not consistent, the test result can be directly judged to be an error.

For example: the target title information is ABCD, and the title information included in the extracted verification set is ABDD. After the target title information is matched with the title information of the verification array, the current test result can be judged to be wrong due to the fact that the text contents of the ABCD and the text contents of the ABDD do not accord with each other.

Similarly, in some embodiments, as shown in FIG. 4, the target operational information is matched with the operational information in the verification set; and if the operation information is not in the operation range of the test object and the error type, judging that the test result is an error. After the target operation information and the operation information of the verification set are obtained, the operation object and the result key words in the target operation information are compared with the operation information in the verification set. If the operation, the operation object and the result key word of the target operation information are all in the key word range of the verification set, the target operation information can be judged to be correct. It should be noted that, at this time, only the operation information can be determined to be correct, and it cannot be determined whether the test result is correct. However, if the operation of the target operation information, or the operation object, or the result keyword is not in the range of the keywords of the verification set, i.e. the corresponding keywords do not match, the test result can be directly determined to be an error.

Further, in some embodiments, as shown in FIG. 5, the target re-operation information is matched with the re-operation information in the verification set; and if the heavy operation information is not in the heavy operation range of the test object and the error type, judging that the test result is an error. After the target re-operation information and the re-operation information of the verification set are obtained, the re-operation and the re-operation object in the target re-operation information are compared with the re-operation information of the verification set. And if the re-operation and the re-operation object of the target re-operation information are both in the key word range of the verification set, judging that the target re-operation information is correct. It should be noted that, at this time, only the correctness of the re-operation information can be determined, and whether the test result is correct or not cannot be determined. However, if the target re-operation information is re-operated, or the re-operation object is not in the range of the keywords in the verification set, that is, the corresponding keywords do not match, the test result may be directly determined as an error.

And, in some embodiments, as shown in FIG. 6, the target hint information is also matched to the hint information in the validation set; and if the target title information does not accord with the prompt information of the test object, judging that the test result is wrong. After the target prompt information and the prompt information of the verification set are obtained, matching keywords included in the target prompt information with the prompt information keywords of the verification set, and if the texts are consistent, judging that the target prompt information is correct. It should be noted that, at this time, only the prompt information can be determined to be correct, and it cannot be determined whether the test result is correct. However, if the two prompt messages are not consistent, the test result can be directly judged to be an error.

It can be understood that, in the above embodiment, the keyword matching process for the target operation information, the target re-operation information and the prompt information is the same as the principle of matching the header information. Therefore, the detailed process of matching is not described in detail in other embodiments.

In some embodiments of the present application, each class of data has a corresponding validity check, since the data information belongs to the class of data corresponding to the class of data. Therefore, the data class to which the input information belongs is also determined, the legality of the input information is checked according to the data class, and the input information is analyzed; and generating an error case according to the input type of the input information. That is, when a use case is generated from input information, different error use cases can be generated according to the input type based on the validity check of the input information.

Based on the above embodiment, as shown in fig. 7, the method for determining uncertain test results based on keywords according to the embodiment of the present application is illustrated by the following example.

Firstly, based on analysis of return information of each product, different case return information is decomposed into four parts, namely a title, operation information, re-operation information and prompt information. And classifying the keyword sets which are used for describing different test objects, operation information, re-operation information and prompt information under different error modes in the returned information according to the title, the operation information, the re-operation information and the prompt information to form a keyword library.

Then, a specific use case is generated according to the input information. And executing the use case on the equipment to obtain the return information. According to the format of the return information, the return information is decomposed into four parts of header information, operation information, re-operation information and prompt information. And extracting corresponding operation, re-operation and prompt information keyword set information from the keyword library according to the test object and the error type.

Finally, the test result is judged according to the following steps:

s601: verifying whether the header information is consistent;

s602: verifying whether the operation information returned by the equipment is in the range of the operation keywords of the test object and the error type;

s603: verifying whether the re-operation information returned by the equipment is in the ranges of the re-operation keywords of the test object and the error type;

s604: and verifying whether the prompt information of the returned information of the equipment conforms to the prompt information in the prompt range corresponding to the test object and the error type.

And if the judgment results of S601-S604 are all yes, judging that the test result is correct. Otherwise, the test result is judged to be wrong.

Based on the method for determining the uncertain test results based on the keywords, some embodiments of the present application further provide a system for determining the uncertain test results based on the keywords, as shown in fig. 8, the system includes: the device comprises an input module, a processing module and an output module. Wherein:

the input module is configured to acquire a keyword library and input information, wherein the keyword library is a keyword classification set formed by keywords of products in various fields; generating a target case according to the input information, wherein the target case comprises the title information, the error type, the test object and the data to be tested of the test object; inputting the target use case into equipment to be tested to obtain target return information;

the processing module is configured to analyze the target return information according to a preset format and decompose the return information into a plurality of sub-target return information; extracting return information from the keyword library according to the test object and the error type to generate a verification set;

the output module is configured to judge that the test result is correct if the sub-target return information is in the range of the verification set; and if the sub-target return information is not in the range of the verification set, judging that the test result is an error.

According to the technical scheme, the method and the system for judging the uncertain test results based on the keywords can generate the target case according to the input information by acquiring the keyword library and the input information. The keyword library is a keyword classification set formed by keywords of the product in each field. And inputting the target use case into the equipment to be tested to obtain target return information of the target use case. And analyzing the target return information, and decomposing the target return information into a plurality of sub-target return information. And extracting corresponding return information from the keyword library according to the test object and the error type to generate a verification set. And verifying the sub-target return information through the verification set respectively, and if the sub-target return information is in the range of the verification set, judging that the test result is correct, so that the judgment efficiency of the test result is improved.

Those skilled in the art will understand that: all or part of the steps for realizing the method embodiments can be completed by hardware related to program instructions, the program can be stored in a computer readable storage medium, and the program executes the steps comprising the method embodiments when executed; and the aforementioned storage medium includes: various media that can store program codes, such as ROM, RAM, magnetic or optical disks.

The above-described embodiments of the apparatuses and the like are merely illustrative, where the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.

The detailed description provided above is only a few examples under the general concept of the present application, and does not constitute a limitation to the scope of the present application. Any other embodiments that can be extended by the solution according to the present application without inventive efforts will be within the scope of protection of the present application for a person skilled in the art.

Claims (10)

1. An uncertain test result judgment method based on keywords is characterized by comprising the following steps:

acquiring a keyword library and input information, wherein the keyword library is a keyword classification set formed by keywords of products in various fields;

generating a target case according to the input information, wherein the target case comprises the title information, the error type, the test object and the data to be tested of the test object;

inputting the target use case into equipment to be tested to obtain target return information;

analyzing the target return information and decomposing the target return information into a plurality of sub-target return information;

extracting return information from the keyword library according to the test object and the error type to generate a verification set;

if the sub-target return information is in the range of the verification set, judging that the test result is correct;

and if the sub-target return information is not in the range of the verification set, judging that the test result is an error.

2. The method for determining uncertain test results based on keywords according to claim 1, further comprising:

obtaining return information of different use cases;

and decomposing the return information into a plurality of sub-return information, wherein the sub-return information comprises header information, operation information, re-operation information and prompt information.

3. The method for determining uncertain test results based on keywords according to claim 2, wherein obtaining the return information of different use cases further comprises:

analyzing the return information;

marking the test object information in the return information as header information;

marking the operation keywords, the operation objects and the operation results in the return information as operation information;

the re-operation key words and the re-operation objects in the return information are marked as re-operation information;

and marking the prompting words and the allowable range information in the return information as prompting information.

4. The method of claim 2, wherein decomposing the target return message into a plurality of sub-target return messages comprises:

and decomposing the target return information into target title information, target operation information, target re-operation information and target prompt information according to a preset information format.

5. The method of claim 4, further comprising:

matching the target title information with the title information in the verification set;

and if the target title information does not accord with the title information of the test object, judging that the test result is an error.

6. The method for determining uncertain test results based on keywords according to claim 4, further comprising:

matching the target operation information with operation information in the verification set;

and if the operation information is not in the operation range of the test object and the error type, judging that the test result is an error.

7. The method of claim 4, further comprising:

matching the target heavy operation information with heavy operation information in the verification set;

and if the heavy operation information is not in the heavy operation range of the test object and the error type, judging that the test result is an error.

8. The method of claim 4, further comprising:

matching the target prompt information with prompt information in the verification set;

and if the target title information does not accord with the prompt information of the test object, judging that the test result is wrong.

9. The method of claim 1, further comprising:

checking the legality of the input information according to the data class, and analyzing the input information;

and generating an error case according to the input type of the input information.

10. An uncertain test result determination system based on keywords, comprising:

the input module is configured to acquire a keyword library and input information, wherein the keyword library is a keyword classification set formed by keywords of products in various fields; generating a target case according to the input information, wherein the target case comprises title information, error types, test objects and to-be-tested data of the test objects; inputting the target use case into equipment to be tested to obtain target return information;

the processing module is configured to analyze the target return information according to a preset format and decompose the return information into a plurality of pieces of sub-target return information; extracting return information from the keyword library according to the test object and the error type to generate a verification set;

the output module is configured to judge that the test result is correct if the sub-target return information is in the range of the verification set; and if the sub-target return information is not in the range of the verification set, judging that the test result is an error.

Images