CN116089251A

CN116089251A - Interface test method, device, storage medium, and program product

Info

Publication number: CN116089251A
Application number: CN202111288835.8A
Authority: CN
Inventors: 刘璐辰; 高玉军; 陆敏慎; 林晓升; 邹意林; 卢凯旋; 曹紫光; 张天泽
Original assignee: Beijing ByteDance Network Technology Co Ltd
Current assignee: Beijing ByteDance Network Technology Co Ltd
Priority date: 2021-11-02
Filing date: 2021-11-02
Publication date: 2023-05-09

Abstract

The embodiment of the disclosure provides an interface testing method, equipment, a storage medium and a program product, wherein the method comprises the steps of obtaining original flow data of an interface to be tested; constructing a test case according to the original request and the corresponding response data included in the original flow data, wherein the test case includes a test request; generating a target strategy according to the original request and the corresponding response data included in the original flow data; triggering a test case to send a test request to a server through an interface to be tested, and receiving target response data fed back by the server; and according to the test request and the target response data, adopting a target strategy to carry out assertion verification on the interface to be tested, and obtaining an assertion verification result of the interface to be tested. By automatically generating test cases and automatically generating target strategies, assertion verification is performed based on the target strategies, the automation degree of interface testing is improved, dependence on manual writing rules is reduced, the problem finding capability is improved, and finer assertion verification of the interface is realized.

Description

Interface test method, device, storage medium, and program product

Technical Field

The embodiment of the disclosure relates to the technical field of computer and network communication, in particular to an interface testing method, equipment, a storage medium and a program product.

Background

The interface test (Application Programming Interface Testing) is one of the software test methods, and is a test that occurs in the information layer. In the interface test process, indexes of performance and the like such as functional integrity, reliability, safety and the like of the service can be verified by calling an API (Application Programming Interface ). For service interface testing, the high-accuracy automatic assertion capability of interface response is always a difficult problem to be solved in the industry due to the influence of a plurality of business practical factors. Wherein asserting is the process of determining whether response data corresponding to an interface request is expected.

In conventional automated interface test practice, the assertion determination is generally implemented by manually writing a verification rule, screening a field to be verified, and comparing the response result of the interface call request under the scene. The process relies on manual writing rules, is only suitable for specified scenes, is complex and cumbersome to configure, and often requires a tester of a business party to have higher programming capability to write and execute scripts to perform relevant verification.

Disclosure of Invention

The embodiment of the disclosure provides an interface testing method, equipment, a storage medium and a program product, which are used for improving the automation degree of interface testing and reducing the dependence on manual writing rules.

In a first aspect, an embodiment of the present disclosure provides an interface testing method, including:

acquiring original flow data of an interface to be tested; constructing a test case according to an original request and corresponding response data included in the original flow data, wherein the test case includes a test request;

generating a target strategy according to the original request and the corresponding response data included in the original flow data, wherein the target strategy is used for representing the mapping relation between the original request and the response data of the normal response original flow data in the original flow data and the mapping relation between the original request and the response data of the request in the abnormal response original flow data;

triggering to send a test request to a server through the interface to be tested by adopting the test case, and receiving target response data fed back by the server;

and according to the test request and the target response data, adopting the target strategy to carry out assertion verification on the interface to be tested, and obtaining an assertion verification result of the interface to be tested.

In a second aspect, an embodiment of the present disclosure provides an interface test apparatus, including:

the acquisition unit is used for acquiring the original flow data of the interface to be tested;

the test case construction unit is used for constructing a test case according to the original request and the corresponding response data included in the original flow data, wherein the test case includes a test request;

the strategy generation unit is used for generating a target strategy according to the original request and the corresponding response data included in the original flow data, wherein the target strategy is used for representing the mapping relation between the original request and the response data of the normal response original flow data in the original flow data and the mapping relation between the original request and the response data of the abnormal response original flow data;

the test unit is used for triggering to send a test request to the server through the interface to be tested by adopting the test case and receiving target response data fed back by the server;

and the assertion unit is used for performing assertion verification on the interface to be tested by adopting the target strategy according to the test request and the target response data, and obtaining an assertion verification result of the interface to be tested.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor and memory;

the memory stores computer-executable instructions;

the at least one processor executes the computer-executable instructions stored by the memory to cause the at least one processor to perform the interface test method as described above in the first aspect and the various possible designs of the first aspect.

In a fourth aspect, embodiments of the present disclosure provide a computer readable storage medium having stored therein computer executable instructions that when executed by a processor implement the interface test method according to the first aspect and the various possible designs of the first aspect.

In a fifth aspect, embodiments of the present disclosure provide a computer program product comprising computer-executable instructions which, when executed by a processor, implement the interface test method according to the first aspect and the various possible designs of the first aspect.

The method, the device, the storage medium and the program product for testing the interface are provided by the embodiment, and the method obtains the original flow data of the interface to be tested; constructing a test case according to an original request and corresponding response data included in the original flow data, wherein the test case includes a test request; generating a target strategy according to the original request and the corresponding response data included in the original flow data, wherein the target strategy is used for representing the mapping relation between the original request and the response data of the normal response original flow data in the original flow data and the mapping relation between the original request and the response data of the request in the abnormal response original flow data; triggering a test case to send a test request to a server through an interface to be tested, and receiving target response data fed back by the server; and according to the test request and the target response data, adopting a target strategy to carry out assertion verification on the interface to be tested, and obtaining an assertion verification result of the interface to be tested. By automatically generating test cases and automatically generating target strategies, assertion verification is performed based on the target strategies, the automation degree of interface testing is improved, dependence on manual writing rules is reduced, the problem finding capability is improved, and finer assertion verification of the interface is realized.

Drawings

In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, a brief description will be given below of the drawings that are needed in the embodiments or the description of the prior art, it being obvious that the drawings in the following description are some embodiments of the present disclosure, and that other drawings may be obtained from these drawings without inventive effort to a person of ordinary skill in the art.

Fig. 1 is an exemplary diagram of an application scenario of an interface testing method according to an embodiment of the disclosure;

FIG. 2 is a flow chart of an interface testing method according to an embodiment of the disclosure;

FIG. 3 is a flowchart of an interface testing method according to another embodiment of the present disclosure;

FIG. 4 is a flowchart of an interface testing method according to another embodiment of the present disclosure;

FIG. 5 is a flowchart of an interface testing method according to another embodiment of the present disclosure;

FIG. 6 is a block diagram of an interface test apparatus provided by an embodiment of the present disclosure;

fig. 7 is a schematic hardware structure of an electronic device according to an embodiment of the disclosure.

Detailed Description

For the purposes of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure, and it is apparent that the described embodiments are some embodiments of the present disclosure, but not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without inventive effort, based on the embodiments in this disclosure are intended to be within the scope of this disclosure.

In the interface test process, the functional integrity, reliability, safety and other performance indexes of the service are verified by calling the API interface. For the service interface test, the high-accuracy automatic assertion capability of the interface response is always a difficult problem to be solved in the industry due to the influence of a plurality of business actual factors, wherein assertion is a process of judging whether response data corresponding to an interface request is in accordance with expectations, for example, the expected response data of a certain interface request is certain specific response data, and if the actual response data is in accordance with the specific response data, assertion is successful; for another example, the expected response data for a certain interface request is a failure response, if the actual response data is also a failure response, the assertion is successful, and if the actual response data is a successful response, the assertion fails.

In the conventional automated interface test practice, the assertion judgment is usually implemented by manually writing a verification rule, screening a field to be verified, and comparing the interfaces under the scene to realize interface assertion verification. The process relies on manual writing rules, is only suitable for specified scenes, is complex and cumbersome to configure, and often requires a tester of a business party to have higher programming capability to write and execute scripts to perform relevant verification.

In order to solve the above technical problems, after obtaining the original flow data of the interface to be tested, the embodiments of the present disclosure may automatically construct a test case and a target policy according to the original flow data, where the test case includes a test request; further, the test request can be sent to the server through the interface to be tested, and target response data fed back by the server is received; and according to the test request and the target response data, adopting the target strategy to carry out assertion verification on the interface to be tested, and obtaining an assertion verification result of the interface to be tested. Through the process, the automation degree of the interface test is improved, the dependence on manual writing rules is reduced, the problem finding capability is improved, and finer assertion verification of the interface is realized.

The interface testing method provided by the embodiment of the disclosure is suitable for an application scenario shown in fig. 1, and comprises a terminal 101 and a server 102, wherein the terminal 101 can acquire original flow data of an interface to be tested in the process of interacting with the server 102 through the interface to be tested, and automatically constructs a test case and a target strategy, wherein the test case comprises a test request; further, the terminal 101 may send the test request to the server 102 through the interface to be tested, and receive the target response data fed back by the server 102; the terminal 101 may perform assertion verification on the interface to be tested by adopting a target policy according to the test request and the target response data, so as to obtain an assertion verification result of the interface to be tested.

The interface test method of the present disclosure will be described in detail with reference to specific embodiments.

Referring to fig. 2, fig. 2 is a flowchart illustrating an interface testing method according to an embodiment of the disclosure. The method of the embodiment can be applied to terminal equipment or a server, and the interface testing method comprises the following steps:

s201, acquiring original flow data of an interface to be tested.

In this embodiment, for an interface to be tested, original flow data of the interface to be tested may be collected, where the original flow data is flow data recorded by the interface to be tested collected during a process that a user uses the interface to be tested, and the flow data includes a request sent to a server through the interface to be tested and corresponding response data returned by the server received by the interface to be tested.

S202, constructing a test case according to an original request and corresponding response data included in original flow data;

wherein the test case comprises a test request.

After the original flow data of the interface to be tested is obtained, a test case can be automatically constructed according to the original flow data, the test case comprises a request (called a test request in the embodiment) for testing the interface to be tested, namely the test request needs to be sent to a server through the interface to be tested, and after response data of the server for the test request is received, assertion verification is carried out to judge whether the response data accords with expectations.

Optionally, in this embodiment, the original traffic data may be divided into normal response original traffic data and abnormal response original traffic data, and may specifically be divided according to a response status code, for example, for an RPC (Remote Procedure Call ) request, a corresponding response status code (status code) is 0, and then the normal response is performed, and the response status code is other values, for example, 50001, 50002, and then the abnormal response is performed; for Http requests, the response status code is 200, then it is a normal response, and the response status code is other values, e.g., 400, 500, then it is an abnormal response.

Further, when the test case is constructed according to the original request and the corresponding response data included in the original flow data, the method specifically may include:

processing an original request in normal response original flow data to construct a first test case; and determining the original request in the abnormal response original flow data as a second test case.

In this embodiment, an original request in normal response original flow data is processed by using an example construction algorithm, and a first test case which is efficient and usable is constructed to form a first test case set, where the first test cases are used for expanding the original request in normal response original flow data, and increasing the number of test requests, for example, 100 original requests in normal response original flow data, the number of test requests in the first test case set obtained after expansion may be 200, the expansion process may be a combination of some original requests, and the like, so that an interface to be tested may be tested more comprehensively, and of course, the test requests of the first test cases may not necessarily all return normal response data.

The original requests in the abnormal response original flow data can cause the response data to be abnormal, the original requests in the abnormal response original flow data can be directly determined to be the second test case, and the original requests are subjected to flow playback, namely, the original requests are sent to the server through the interface to be tested again.

Of course, other test case construction methods may also be adopted in this embodiment, and will not be described herein.

S203, generating a target strategy according to the original request and the corresponding response data included in the original flow data;

the target strategy is used for representing the mapping relation between the original request and response data of the normal response original flow data in the original flow data and the mapping relation between the original request and response data of the request in the abnormal response original flow data.

In this embodiment, the original request and the corresponding response data included in the original traffic data may be automatically learned and analyzed, and a target policy for interface assertion may be automatically generated, where the target policy is used to characterize a mapping relationship between the original request and the response data of the normal response original traffic data in the original traffic data, and a mapping relationship between the original request and the response data of the request in the abnormal response original traffic data. It may be determined whether the mapping between the interface request and response data meets expectations according to the target policy.

In an alternative embodiment, the target policy may be an interface response data template, including an interface response data template of normal response data and an interface response data template of abnormal response data, where the interface response data template of normal response data and the interface response data template of abnormal response data respectively include a field information complete sub-template and a field information sub-template, and it may be possible to implement assertion verification of the interface by determining whether the structure of the response data meets the interface response data template.

In another alternative embodiment, the target policy may be an assertion model trained according to a mapping relationship between an original request and corresponding response data in the original flow data, and determine, according to the assertion model, whether the mapping relationship between the interface request and the response data meets an expectation, to implement assertion verification on the interface.

Of course, in this embodiment, other target policies for interface assertion may be automatically generated by automatically learning and analyzing the original request and the corresponding response data included in the original flow data, which will not be described herein.

Note that, in this embodiment, the execution sequence of S202 and S203 is not limited, and any sequence may be adopted.

S204, the test request is sent to a server through the interface to be tested, and target response data fed back by the server are received.

In this embodiment, each test request is sequentially sent to the server through the interface to be tested in the test execution stage, and the server feeds back the target response data corresponding to each test request according to the test request.

S205, according to the test request and the target response data, adopting the target strategy to carry out assertion verification on the interface to be tested, and obtaining an assertion verification result of the interface to be tested.

In this embodiment, whether the target response data corresponding to the test request meets the expectation or not may be automatically determined based on the generated target policy, and whether the mapping relationship between the interface request and the response data meets the expectation or not is determined, so as to implement assertion verification on the interface to be tested, and obtain an assertion verification result on the interface to be tested.

According to the interface testing method provided by the embodiment, the original flow data of the interface to be tested is obtained; constructing a test case according to an original request and corresponding response data included in the original flow data, wherein the test case includes a test request; generating a target strategy according to the original request and the corresponding response data included in the original flow data, wherein the target strategy is used for representing the mapping relation between the original request and the response data of the normal response original flow data in the original flow data and the mapping relation between the original request and the response data of the request in the abnormal response original flow data; triggering a test case to send a test request to a server through an interface to be tested, and receiving target response data fed back by the server; and according to the test request and the target response data, adopting a target strategy to carry out assertion verification on the interface to be tested, and obtaining an assertion verification result of the interface to be tested. By automatically generating test cases and automatically generating target strategies, assertion verification is performed based on the target strategies, the automation degree of interface testing is improved, dependence on manual writing rules is reduced, the problem finding capability is improved, and finer assertion verification of the interface is realized.

In an alternative embodiment, the generating the target policy in S203 according to the original request and the corresponding response data included in the original traffic data may specifically include:

generating an interface response data template according to the response data in the original flow data;

the interface response data templates comprise interface response data templates of normal response data and interface response data templates of abnormal response data, and the interface response data templates of the normal response data and the interface response data templates of the abnormal response data respectively comprise field information total sub-templates and character field information sub-templates which need to be filled.

In this embodiment, an interface response data template may be constructed from the original traffic data, and the interface response data template may define fields included in the response data, for example, which mandatory fields the response data needs to include, which optional fields the response data may also have, all possible fields (not exceeded) in the response data, and types of various fields, etc., to be used as a basis for assertion verification for the structure of the response data. The field information may be extracted for response data included in the original traffic data, respectively, and an interface response data template may be generated according to the extracted field information.

Optionally, the interface response data template includes an interface response data template of normal response data and an interface response data template of abnormal response data; the interface response data template of the normal response data is obtained by summarizing field information extracted from response data in the normal response original flow data, and the interface response data template of the abnormal response data is obtained by summarizing field information extracted from response data in the abnormal response original flow data.

As an optional embodiment, field information may be extracted from the response data included in the normal response original traffic data, and the union set may be obtained from the field information of each response data, so that the complete set of all possible fields in the normal response data may be determined, that is, the fields in any normal response data do not exceed the complete set of all possible fields; the intersection of the field information of each response data can determine the necessary filling fields in the normal response data, namely the fields in any normal response data must comprise the necessary filling fields; by adopting the same process as described above for the exception response raw traffic data, a complete set of all possible fields in the exception response data, as well as the mandatory fields in the exception response data, may be determined.

Based on the above embodiment, the performing, according to the test request and the target response data, assertion verification on the interface to be tested by using the target policy in S205 includes:

and determining the type of the target response data as normal response data or abnormal response data according to the response state code of the target response data, and carrying out assertion verification on the structure of the target response data according to an interface response data template corresponding to the type of the target response data.

In this embodiment, after the target response data corresponding to the test request is obtained, the structure of the target response data may be subjected to assertion verification according to the interface response data template, where assertion verification is to determine whether the target response data accords with expectations, that is, for some test requests, the target response data is expected to return to normal response data, determine whether the actually returned target response data meets the template of the normal response data, for some test requests, the target response data is expected to return to abnormal response data, and determine whether the actually returned target response data meets the template of the abnormal response data, especially whether there is a field missing and/or a field type error. The problem of the interface to be tested can be automatically discovered by carrying out assertion verification on the structure of the target response data based on the interface response data template, and in addition, in the embodiment, the structure assertion verification is carried out from the angles of the fields and the field types based on the interface response data template, so that finer assertion verification of the interface can be realized.

On the basis of any one of the above embodiments, the interface response data templates include an interface response data template of normal response data and an interface response data template of abnormal response data; the interface response data template of the normal response data is obtained by summarizing response data in the normal response original flow data, and the interface response data template of the abnormal response data is obtained by summarizing response data in the abnormal response original flow data.

Further, in this embodiment, if it is determined that the target response data is normal response data, the structure of the target response data is asserted and verified according to the interface response data template of the normal response data, if the structure of the target response data meets the interface response data template of the normal response data, the assertion and verification are successful, otherwise the assertion and verification are failed; in contrast, if the target response data is determined to be abnormal response data, the structure of the target response data is subjected to assertion verification according to the interface response data template of the abnormal response data, if the structure of the target response data meets the interface response data template of the abnormal response data, the assertion verification is successful, and otherwise, the assertion verification is failed.

On the basis of the embodiment, each interface response data template can comprise a field information full-set sub-template and a character field information sub-template which is necessary to be filled, wherein the field information full-set sub-template comprises all field information full sets, and the character field information full-set sub-template comprises character field information necessary to be filled and optional character field information; and the mandatory field information sub-template includes mandatory field information.

Wherein, optionally, when generating the interface response data template, the method specifically may include:

extracting field information from response data included in the normal response original flow data respectively, and merging the field information of each response data to generate a field information full-set sub-template of the normal response data; intersection is taken for the field information of each response data, and a sub-template of the character field information which is necessary to be filled for the normal response data is generated;

extracting field information from response data included in the abnormal response original flow data respectively, and merging the field information of each response data to generate a field information total-set sub-template of the abnormal response data; intersection is taken for the field information of each response data, and a sub-template of the character field information which is necessary to be filled for the abnormal response data is generated.

In this embodiment, taking the response data format as a JSON format as an example, for each response data included in the normal response original traffic data, a JSON Schema template file is extracted respectively, where JSON Schema is used to mark and verify the data in JSON format

In this embodiment, a union is taken for leaf nodes in each JSON Schema to obtain the union JSON Schema, which is used as a field information total-set sub-template of normal response data, including all possible fields in the normal response data and corresponding field types, such as character strings and numbers.

In this embodiment, an intersection is taken for leaf nodes in each JSON Schema, so as to obtain the intersection JSON Schema, which is used as a sub-template of the necessary field information of the normal response data, including the field which must appear in the normal response data, the corresponding field type, and the like, such as a character string and a number.

The same processing manner is adopted for the abnormal response original flow data, and is not repeated here.

On the basis of the foregoing embodiment, the asserting and verifying the structure of the target response data according to the corresponding interface response data template includes:

judging whether all fields in the target response data do not exceed the fields included in the target field information total-set sub-template corresponding to the types of the target response data, and the field types of all fields respectively meet the field types corresponding to the target field information total-set sub-template; and/or

And judging whether the target response data comprises all the necessary-to-be-filled characters in the target necessary-to-be-filled character information sub-template corresponding to the type of the target response data, and meeting the corresponding field type.

In this embodiment, if the target response data is normal response data, it may be determined whether all fields in the target response data do not exceed the fields included in the target field information full-set sub-template of the normal response data, and the field types of all fields in the target response data respectively satisfy the corresponding field types in the target field information full-set sub-template; whether the target response data comprises all the necessary-to-be-filled characters in the target necessary-to-be-filled character information sub-template of the normal response data can be judged, and the corresponding field types are met; if yes, determining that the structure assertion verification of the target response data is successful, namely that the target response data meets the expectations; if any of the above judgments is not satisfied, determining that the structural assertion verification of the target response data fails, that is, the target response data does not satisfy the expectations.

Similarly, if the target response data is abnormal response data, it may be determined whether all fields in the target response data do not exceed the fields included in the target field information total-set sub-template of the abnormal response data, and the field types of all fields in the target response data respectively satisfy the corresponding field types in the target field information total-set sub-template; whether the target response data comprises all the necessary-to-be-filled characters in the target necessary-to-be-filled character information sub-template of the abnormal response data can be judged, and the corresponding field types are met; if yes, determining that the structure assertion verification of the target response data is successful, namely that the target response data meets the expectations; if any of the above judgments is not satisfied, determining that the structural assertion verification of the target response data fails, that is, the target response data does not satisfy the expectations.

On the basis of any embodiment, because the types of the server side problems covered by the structural assertion verification of the response data are limited, the structural problems such as field missing, field return value type error and the like can be covered only, but more of the actually occurring server side accidents are caused by field content errors in the response data, and for the problems of the field content errors, the embodiment also provides content assertion verification based on natural language understanding and model learning capability.

In particular, the method comprises the steps of,

in the foregoing embodiment, generating the target policy according to the original request and the corresponding response data included in the original traffic data in S203 may further include:

constructing training data according to the mapping relation between the original request and the corresponding response data in the original flow data, and training an initial model according to the training data to obtain an assertion model;

in this embodiment, an assertion model may be obtained in advance, where the input of the model is a data pair formed by request and response data, and the output is an assertion verification result of the response data content, that is, a classification result for judging whether the response data content meets the expectation, and whether the assertion verification is successful or not.

Further, in the foregoing embodiment, according to the test request and the target response data in S205, performing assertion verification on the interface to be tested by using the target policy includes:

inputting the test request and the target response data into the assertion model, and performing assertion verification on the mapping relation between the test request and the target response data according to the assertion model.

In this embodiment, in the interface test stage, after target response data corresponding to a test request is obtained, the test request and the target response data are input into the assertion model, so as to obtain an assertion verification result of the content of the target response data.

Optionally, in order to improve the model prediction effect based on the above embodiment, different assertion models may be configured for different response status codes, for example, one assertion model may be configured for the response status codes of 0, 50001, and 50002, respectively.

On this basis, as shown in fig. 3, the inputting the test request and the target response data into an assertion model, and performing assertion verification on the content of the target response data according to the assertion model may include:

S301, acquiring a response state code of the target response data;

s302, selecting an assertion model corresponding to a response state code of the target response data;

s303, inputting the test request and the target response data into an assertion model corresponding to the response state code, and obtaining a classification result of successful or failed verification of mapping assertion between the test request and the target response data.

In the embodiment, through targeted training of the assertion model, the content assertion verification accuracy of the assertion model on the target response data of the specific response state code is improved.

On the basis of any of the foregoing embodiments, as shown in fig. 4, the constructing training data according to the mapping relationship between the original request and the corresponding response data in the original traffic data includes:

s401, constructing positive example training data and negative example training data corresponding to each response state code according to an original request and corresponding response data in original flow data;

s402, training an initial model corresponding to each response state code according to positive training data and negative training data corresponding to each response state code, and obtaining the assertion model corresponding to each response state code.

In this embodiment, after the original flow data of the interface to be tested is obtained, training data may be constructed based on the original flow data, and optionally, because different assertion models are respectively configured by different response status codes, training data may be separately constructed and training may be separately performed for each response status code.

For any response status code, the original request and the corresponding response data under the response status code can be extracted from the original flow data, and based on the original request and the corresponding response data, positive example training data and negative example training data corresponding to the response status code are obtained, and the specific construction mode can include:

acquiring response data and corresponding original requests of successful content assertion verification, and determining the response data and the corresponding original requests as positive training data; and/or

Acquiring response data of failed verification of content assertion and corresponding original request, and determining the response data as negative training data; and/or

And acquiring response data and corresponding original requests of successful content assertion verification, performing error injection on the response data, and determining the response data and the corresponding original requests after error injection as negative training data.

In this embodiment, response data of successful content assertion verification and a corresponding original request are obtained, that is, the response data meets expectations (assertion verification is 0 in two classification results), and a data pair < req, resp > of the original request and the response data is used as positive training data;

Obtaining response data of content assertion verification failure and corresponding original request, namely that the response data does not accord with expectations (assertion verification is divided into two types of results are 1), and taking a data pair < req, resp > of the original request and the response data as negative training data;

the two cases are original request and response data which are actually present, and in addition, some negative training data can be constructed, namely, error injection is carried out on the response data on the basis of the positive training data, namely, the data pair < req, fuzz resp > of the original request and the response data after error injection is used as the negative training data.

Through the process, sufficient training data are obtained, and an initial model corresponding to the response state code is trained to obtain the assertion model corresponding to the response state code.

Wherein, optionally, the training data can be converted into text information and abstracted into vectors, and used as input of a model for model training. The embodiment can be used for any natural language processing model, and the embodiment has higher requirement on model training timeliness, compared with the model, more and shallower neural network models have the accuracy comparable with that of a deep network, and the model classifying method has the advantage of extremely high classifying speed. For example, the assertion model may be fastText model, which is a word vector and text classification tool that is open in 2016 by FaceBook corporation, and combines natural language processing and machine learning concepts: 1) Characterizing the sentence using a bag of words and an n-gram bag; 2) Sharing information among categories by hidden tokens using subword (subword) information; 3) A softmax hierarchy is adopted, and the advantage of unbalanced distribution of categories is utilized to accelerate the operation process.

Optionally, in the subsequent application process, the flow data on the current interface line can be recorded continuously and regularly, a test set (development set) is constructed and expanded, and the assertion model of the current version is continuously and iteratively optimized.

The method for verifying the assertion of the content of the response data can effectively establish the mapping relation between the request and the response data, realize the general and high-accuracy understanding of the request and the response data, abstract the training data into text content by automatically acquiring the training data, and abstract the text content into vectors by processing a related model through natural language. Through light corpus processing model learning, high-efficiency and high-accuracy model construction is realized, and probability binary judgment is given to an actual test case execution response result, so that an interface is assisted to automatically verify high-accuracy assertion.

It should be noted that, in the foregoing embodiment, the verification of the assertion of the response data structure and the verification of the assertion of the response data structure may be selected according to actual requirements, for example, in an alternative embodiment, if the requirement is that only the assertion verification of the structure is performed, only the steps of the related embodiment of the assertion verification of the response data structure based on the interface response data template are executed; in another alternative embodiment, if the requirement is content assertion verification only, then only the steps of the relevant embodiments of verifying responsive data content assertion based on the assertion model are performed, without performing the steps of the relevant embodiments of verifying responsive data structure assertion.

In another alternative embodiment, if the requirement is to perform both structure assertion verification and content assertion verification, then the steps of the related embodiment for response data structure assertion verification and the steps of the related embodiment for response data content assertion verification need to be performed, as shown in fig. 5, include the following steps:

s501, acquiring original flow data of an interface to be tested;

s502, constructing a test case according to an original request and corresponding response data included in original flow data, wherein the test case comprises a test request;

s503, generating an interface response data template according to response data in the original flow data;

s504, constructing positive training data and negative training data corresponding to each response state code according to the original request and the corresponding response data in the original flow data;

s505, training an initial model corresponding to each response state code according to positive training data and negative training data corresponding to each response state code to obtain the assertion model corresponding to each response state code;

s506, the test request is sent to a server through the interface to be tested, and target response data fed back by the server are received;

S507, carrying out assertion verification on the structure of the target response data according to an interface response data template corresponding to the type of the target response data;

s508, inputting the test request and the target response data into the assertion model, and performing assertion verification on the mapping relation between the test request and the target response data according to the assertion model.

In addition, the execution sequence between S502 and S506 and S503 and between S504 and S505 is not limited in the present embodiment.

In this embodiment, the final assertion verification is determined to be successful if both assertion verification is successful, otherwise the final assertion verification is determined to be failed if either assertion verification is failed.

Corresponding to the interface testing method of the above embodiment, fig. 6 is a block diagram of the structure of the interface testing apparatus provided in the embodiment of the present disclosure. For ease of illustration, only portions relevant to embodiments of the present disclosure are shown. Referring to fig. 6, the interface test apparatus 600 includes: an acquisition unit 601, a test case construction unit 602, a template generation unit 603, a test unit 604, and an assertion unit 605.

The acquiring unit 601 is configured to acquire original flow data of an interface to be tested;

The test case construction unit 602 is configured to construct a test case according to an original request and corresponding response data included in the original flow data, where the test case includes a test request;

a policy generating unit 603, configured to generate a target policy according to the original request and the corresponding response data included in the original traffic data, where the target policy is used to characterize a mapping relationship between the original request and the response data of the normal response original traffic data in the original traffic data, and a mapping relationship between the original request and the response data of the request in the abnormal response original traffic data;

the test unit 604 is configured to trigger sending a test request to a server through the interface to be tested by using the test case, and receive target response data fed back by the server;

and the asserting unit 605 is configured to assert and verify the interface to be tested by adopting the target policy according to the test request and the target response data, and obtain an assertion and verification result of the interface to be tested.

In one or more embodiments of the present disclosure, the policy generating unit 603, when generating a target policy according to an original request and corresponding response data included in the original traffic data, is configured to:

Generating an interface response data template according to response data in the original flow data, wherein the interface response data template comprises an interface response data template of normal response data and an interface response data template of abnormal response data, and the interface response data template of the normal response data and the interface response data template of the abnormal response data respectively comprise a field information total-set sub-template and a word segment information sub-template which are needed to be filled;

the asserting unit 605 is configured to, when asserting and verifying the interface to be tested by using the target policy according to the test request and the target response data:

In one or more embodiments of the present disclosure, the policy generating unit 603, when generating an interface response data template according to response data in the original traffic data, is configured to:

respectively acquiring normal response original flow data and abnormal response original flow data in the original flow data according to the response state code;

In one or more embodiments of the present disclosure, when the asserting unit 605 performs asserting verification on the structure of the target response data according to the interface response data template corresponding to the type of the target response data, the asserting unit is configured to:

In one or more embodiments of the present disclosure, the policy generating unit 603 is configured to, when constructing training data according to a mapping relationship between an original request and corresponding response data in the original traffic data, and training an initial model according to the training data, obtain an assertion model, perform:

Respectively constructing positive example training data and negative example training data corresponding to each response state code according to the mapping relation between the original request corresponding to each response state code and the corresponding response data in the original flow data;

and training the initial model corresponding to each response state code according to the positive training data and the negative training data corresponding to each response state code to obtain the assertion model corresponding to each response state code.

In one or more embodiments of the present disclosure, when the assertion unit 605 inputs the test request and the target response data into the assertion model, performs assertion verification on a mapping relationship between the test request and the target response data according to the assertion model, the assertion unit is configured to:

and inputting the test request and the target response data into an assertion model corresponding to the response state code according to the response state code of the target response data, and acquiring a classification result of successful or failed verification of mapping assertion between the test request and the target response data according to the assertion model.

In one or more embodiments of the present disclosure, the policy generating unit 603 is configured, when constructing training data according to a mapping relationship between an original request and corresponding response data in the original traffic data, to:

Acquiring an original request corresponding to each response state code and corresponding response data from the original flow data;

for any response state code, response data of successful content assertion verification and corresponding original request are obtained, and positive training data is determined; and/or

In one or more embodiments of the present disclosure, the assertion model is a fastText model.

In one or more embodiments of the present disclosure, the test case construction unit 602, when constructing a test case according to the original request and the corresponding response data included in the original traffic data, is configured to:

processing an original request in normal response original flow data to construct a first test case;

and determining the original request in the abnormal response original flow data as a second test case.

The device provided in this embodiment may be used to execute the technical solution of the foregoing method embodiment, and its implementation principle and technical effects are similar, and this embodiment will not be described herein again.

Referring to fig. 7, there is shown a schematic diagram of an electronic device 700 suitable for use in implementing embodiments of the present disclosure, which electronic device 700 may be a terminal device or a server. The terminal device may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a personal digital assistant (Personal Digital Assistant, PDA for short), a tablet (Portable Android Device, PAD for short), a portable multimedia player (Portable Media Player, PMP for short), an in-vehicle terminal (e.g., an in-vehicle navigation terminal), and the like, and a fixed terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 7 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.

As shown in fig. 7, the electronic apparatus 700 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 701 that may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage device 708 into a random access Memory (Random Access Memory, RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.

In general, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 707 including, for example, a liquid crystal display (Liquid Crystal Display, LCD for short), a speaker, a vibrator, and the like; storage 708 including, for example, magnetic tape, hard disk, etc.; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 shows an electronic device 700 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.

In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication device 709, or installed from storage 708, or installed from ROM 702. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 701.

It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.

The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.

The computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above-described embodiments.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (Local Area Network, LAN for short) or a wide area network (Wide Area Network, WAN for short), or it may be connected to an external computer (e.g., connected via the internet using an internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The name of the unit does not in any way constitute a limitation of the unit itself, for example the first acquisition unit may also be described as "unit acquiring at least two internet protocol addresses". The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In a first aspect, according to one or more embodiments of the present disclosure, there is provided an interface testing method, including:

According to one or more embodiments of the present disclosure, the generating a target policy according to the original request and the corresponding response data included in the original traffic data includes:

And according to the test request and the target response data, adopting the target policy to perform assertion verification on the interface to be tested, including:

According to one or more embodiments of the present disclosure, the generating an interface response data template according to response data in the raw traffic data includes:

According to one or more embodiments of the present disclosure, the asserting and verifying the structure of the target response data according to the interface response data template corresponding to the type of the target response data includes:

According to one or more embodiments of the present disclosure, the constructing training data according to a mapping relationship between an original request and corresponding response data in the original flow data, and training an initial model according to the training data to obtain an assertion model includes:

According to one or more embodiments of the present disclosure, the inputting the test request and the target response data into the assertion model, and performing assertion verification on a mapping relationship between the test request and the target response data according to the assertion model includes:

According to one or more embodiments of the present disclosure, the constructing training data according to a mapping relationship between an original request and corresponding response data in the original traffic data includes:

According to one or more embodiments of the present disclosure, the assertion model is a fastText model.

According to one or more embodiments of the present disclosure, the constructing a test case according to original request and corresponding response data included in original traffic data includes:

In a second aspect, according to one or more embodiments of the present disclosure, there is provided an interface test apparatus comprising:

According to one or more embodiments of the present disclosure, the policy generation unit, when generating a target policy from an original request and corresponding response data included in the original traffic data, is configured to:

the asserting unit is configured to, when asserting and verifying the interface to be tested by adopting the target policy according to the test request and the target response data:

According to one or more embodiments of the present disclosure, the policy generation unit, when generating an interface response data template according to response data in the original traffic data, is configured to:

According to one or more embodiments of the present disclosure, when the asserting unit performs asserting verification on the structure of the target response data according to the interface response data template corresponding to the type of the target response data, the asserting unit is configured to:

According to one or more embodiments of the present disclosure, the policy generating unit is configured to, when constructing training data according to a mapping relationship between an original request and corresponding response data in the original traffic data, and training an initial model according to the training data, obtain an assertion model, perform:

According to one or more embodiments of the present disclosure, when the test request and the target response data are input into the assertion model, the third assertion unit is configured to, when performing assertion verification on a mapping relationship between the test request and the target response data according to the assertion model:

According to one or more embodiments of the present disclosure, the policy generation unit, when constructing training data according to a mapping relationship between an original request and corresponding response data in the original traffic data, is configured to:

According to one or more embodiments of the present disclosure, the test case construction sheet is configured to, when constructing a test case from original request and corresponding response data included in original traffic data:

In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device comprising: at least one processor and memory;

the memory stores computer-executable instructions;

In a fourth aspect, according to one or more embodiments of the present disclosure, there is provided a computer-readable storage medium having stored therein computer-executable instructions which, when executed by a processor, implement the interface test method as described above in the first aspect and the various possible designs of the first aspect.

In a fifth aspect, according to one or more embodiments of the present disclosure, there is provided a computer program product comprising computer-executable instructions which, when executed by a processor, implement the interface test method as described above in the first aspect and the various possible designs of the first aspect.

The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by persons skilled in the art that the scope of the disclosure referred to in this disclosure is not limited to the specific combinations of features described above, but also covers other embodiments which may be formed by any combination of features described above or equivalents thereof without departing from the spirit of the disclosure. Such as those described above, are mutually substituted with the technical features having similar functions disclosed in the present disclosure (but not limited thereto).

Moreover, although operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are example forms of implementing the claims.

Claims

1. An interface testing method, comprising:

2. The method of claim 1, wherein generating the target policy from the original request and corresponding response data included in the original traffic data comprises:

3. The method of claim 2, wherein generating an interface response data template from response data in the raw traffic data comprises:

4. A method according to claim 2 or 3, wherein said asserting verification of the structure of the target response data according to the interface response data template corresponding to the type of the target response data comprises:

5. The method according to claim 1 or 2, wherein said generating a target policy from original request and corresponding response data included in said original traffic data comprises:

6. The method of claim 5, wherein constructing training data according to a mapping relationship between an original request and corresponding response data in the original flow data, and training an initial model according to the training data, to obtain an assertion model, comprises:

7. The method of claim 6, wherein the inputting the test request and the target response data into the assertion model, and performing assertion verification on a mapping relationship between the test request and the target response data according to the assertion model, comprises:

8. The method of claim 5, wherein constructing training data from the mapping between the original request and the corresponding response data in the original traffic data comprises:

9. The method of claim 5, wherein the assertion model is a fastText model.

10. The method of claim 1, wherein constructing test cases from the original request and corresponding response data included in the original traffic data comprises:

11. An interface test apparatus, comprising:

12. An electronic device, comprising: at least one processor and memory;

The memory stores computer-executable instructions;

the at least one processor executing computer-executable instructions stored in the memory causes the at least one processor to perform the method of any one of claims 1-10.

13. A computer readable storage medium having stored therein computer executable instructions which, when executed by a processor, implement the method of any of claims 1-10.

14. A computer program product comprising computer-executable instructions which, when executed by a processor, implement the method of any of claims 1-10.