CN110941559B - Automatic test method and system - Google Patents
Automatic test method and system Download PDFInfo
- Publication number
- CN110941559B CN110941559B CN201911181263.6A CN201911181263A CN110941559B CN 110941559 B CN110941559 B CN 110941559B CN 201911181263 A CN201911181263 A CN 201911181263A CN 110941559 B CN110941559 B CN 110941559B
- Authority
- CN
- China
- Prior art keywords
- test
- database
- automatic
- flow
- advertisement
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
Abstract
The embodiment of the invention provides an automatic test method and an automatic test system, which are used for reducing misjudgment of an advertisement engine in an automatic test process. The automatic test system includes: an automatic test framework system and a database for test; before the automatic test, the test database is an empty database; the method comprises the following steps: loading test cases by the automatic test framework system; the test case comprises an initialization flow, a test flow and an expected response; the automatic test framework system fills configuration data and advertisement materials into a test database according to an initialization flow in the test case, and configures server parameters of the BS; the expected response is set for the configuration data, the advertising material, the server parameters, and the advertising request in the test flow; and the automatic test framework system sends an advertisement request to the BS according to the test flow in the test case, receives the response returned by the BS, compares the returned response with the expected response, and generates a test report.
Description
Technical Field
The invention relates to the technical field of automatic testing, in particular to an automatic testing method and an automatic testing system.
Background
The communication between the back-end service internal components, and the communication between the server and the client all have respective different interface definitions. When the interface implementation of a certain module does not meet the requirements of other modules, serious communication faults can occur. A complete interface test is required.
The existing interface test framework mainly aims at Web services, and can carry out coverage test on scenes of different URLs, different request parameters and different request methods. The same is true of advertisement engine (BS) interface testing.
When testing, the interface testing framework executes the test case, sends an http request to the BS service and obtains a response, and the interface testing framework compares whether the returned response is consistent with the expected response or not and obtains a test report.
However, the inventors have found that automatic testing of BS interfaces is quite specific, since most of the complex logic of the BS (such as filtering, ordering, etc. logic of advertisement material) is concentrated inside the BS service, determined by various configuration tables (which may be collectively referred to as configuration data) in the database, as well as server parameters.
The expected response is designed for specific configuration data and server parameters. If the configuration data and the service parameters of the database change, for the same http request, the BS may trigger to execute different logic, so as to return a response inconsistent with the expected response, and thus, even if the BS does not have an error, the BS may be misjudged due to the difference between the returned response and the expected response.
Disclosure of Invention
In view of this, the embodiments of the present invention provide an automatic test method and system to reduce erroneous judgment of an advertisement engine during an automatic test process.
In order to achieve the above object, the embodiment of the present invention provides the following technical solutions:
an automatic test method is used for testing an advertisement engine BS and is applied to an automatic test system; the automatic test system includes: an automatic test framework system and a database for test; before an automatic test is performed, the test database is an empty database;
the method comprises the following steps:
the automatic test framework system loads test cases; the test case comprises an initialization flow, a test flow and an expected response;
the automatic test framework system fills configuration data and advertisement materials into the test database according to the initialization flow in the test case, and configures server parameters of the BS; the expected response is set for the configuration data, the advertising material, the server parameters, and the advertising request in the test flow;
the automatic test framework system sends an advertisement request to the BS according to the test flow in the test case;
the automatic test framework system receives a response returned by the BS, compares the returned response with the expected response and generates a test report; the test report includes at least: and characterizing the success or failure information of the test case test.
Optionally, the method further comprises: and deleting the data in the database after the test case is executed.
Optionally, the method further comprises: and after the test case is executed, restoring the server parameters of the BS to default values.
Optionally, the test report further includes: the BS generates a log for the advertisement request.
An automatic test system for testing an advertisement engine BS, comprising: an automatic test framework system and a database for test; before an automatic test is performed, the test database is an empty database;
the automatic test framework system comprises:
the loading unit is used for loading the test cases; the test case comprises an initialization flow, a test flow and an expected response;
the database configuration unit is used for filling configuration data and advertisement materials into the test database according to the initialization flow in the test case;
the BS configuration unit is used for configuring server parameters of the BS according to the initialization flow in the test case; the expected response is set for the temporary configuration data, the advertising material, the server parameters, and the advertising request in the test flow;
a test unit for:
sending an advertisement request to the BS according to the test flow in the test case;
receiving a response returned by the BS, comparing the returned response with the expected response, and generating a test report; the test report includes at least: and characterizing the success or failure information of the test case test.
Optionally, the database configuration unit is further configured to: and deleting the data in the database after the test case is executed.
Optionally, the BS configuration unit is further configured to: and after the test case is executed, restoring the server parameters of the BS to default values.
Optionally, the test report further includes: the BS generates a log for the advertisement request.
It can be seen that in the embodiment of the present invention, the initialization process is performed before the test process is performed. In the initialization process, configuration data and advertisement materials are filled into the empty database, the environment of the database is configured into an expected environment, and meanwhile, server parameters of the BS are also configured. In this way, consistency of the database configuration data and the server parameters of the BS with the expected response can be ensured. When automatic test is carried out, the BS triggers and executes different logics to return different responses due to the change of configuration data and server parameters, so that erroneous judgment of the BS is reduced.
Drawings
FIG. 1 is an exemplary architecture of an automatic test system provided in an embodiment of the present invention;
FIG. 2 is an exemplary interaction flow of an automatic test method provided by an embodiment of the present invention;
FIG. 3 is another exemplary interaction flow of an automatic test method provided by an embodiment of the present invention;
FIG. 4 is a further exemplary interaction flow of the automatic test method provided by an embodiment of the present invention;
fig. 5 is an exemplary structure of an automatic test system according to an embodiment of the present invention.
Detailed Description
The embodiment of the invention provides an automatic test method and an automatic test system, which are used for reducing misjudgment of an advertisement engine in an automatic test process.
Fig. 1 illustrates an exemplary architecture of the automatic test system described above, including: an automatic test framework system (autotest) and a database for test (DB).
Wherein the DB is an empty database prior to performing the automated test.
Autotest may include a server and an automatic test framework (server side application) deployed on the server, the automatic test framework being implemented in Python language, and executing related test flow operations by launching services through command line tools. It can be deployed on jenkins persistence integration tool, periodically performing interface testing.
It should be noted that, the jenkins continuous integration tool is a framework for continuous integration, has a graphical interface, and can be conveniently operated repeatedly, so that the operation of logging on a server is not needed.
The tested advertisement engine service (BS) is mainly used for advertisement filtering, advertisement coarse ranking and the like.
The automatic test system and the advertisement engine service (BS) under test constitute a test environment.
In the test environment, the BS acquires data only from the database for test, and does not acquire data from other databases.
In addition, the test environment may further include:
and the pulling unit is used for pulling the advertisement materials and the configuration data of the database at regular time and 2 times a day, and writing the advertisement materials and the configuration data into the BS service through ft (a miniature service of the engine system, which mainly has the effect of transmitting text files from one machine to another machine).
And a real-time updating module: for acquiring the contents of the DB update in real time and writing into the BS service.
In actual operation, the advertisement material and configuration data are stored in the database by the advertisement delivery end (advertiser), and the server parameters of the BS are configured.
In this embodiment, after the test case or the test task/item is loaded by the automatic test framework system, in the initialization process of executing the test case, the operation of the advertisement delivery end is simulated, the advertisement material and the configuration data are filled into the database, and the server parameters of the BS are configured.
FIG. 2 illustrates exemplary interactive steps of an automatic test method performed based on the automatic test system described above, including:
s1: the automatic test framework system loads test cases.
The automated test framework system may execute multiple test cases under one test task or item in sequence.
As each test case is executed, the test case is loaded.
Each test case may include an initialization flow, a test flow, and an expected response.
The initialization procedure may include, among other things, the operations of filling the database with configuration data and advertising material, and configuring server parameters of the BS.
The test flow may include logic to send an http request and to validate an http response.
It should be noted that the test flow simulates a process that a user (client) requests an advertisement.
For example, when a user opens a client of a mobile phone shopping software, the client sends an http request to a server to request advertisement delivery if the page of the shopping software has an advertisement space.
S2: and the automatic test framework system fills configuration data and advertisement materials into the test database according to the initialization flow in the test case, and configures server parameters of the BS.
"filling" is understood here as "writing", "storing".
In the test case, the configuration data of the database may include, but is not limited to: advertisement delivery period, advertisement placement, bidding, advertisement priority, advertisement delivery status, budget, frequency control configuration, black and white list, etc.
The configuration data may be in the form of a configuration table.
The server parameters may include advertisement template configuration (advertisement picture size), black and white list.
Step S2 simulates the operation of the advertising end.
S3: the automatic test framework system sends an advertisement request (http request) to the BS according to the test flow in the test case.
The advertisement request may include a user identification, ad spot information, etc.
Step S3 simulates an operation in which the client requests an advertisement.
S4: the BS returns an http response.
S5: the automatic test framework system compares the returned response with the expected response in the test case to generate a test report.
Wherein the expected response in the test case is set for the configuration data, the advertisement material, the server parameters and the advertisement request in the test flow.
The test report includes at least: and information (test result) representing the success or failure of the test case test.
In addition, the test results may further include: the BS generates a log for the advertisement request.
It should be noted that the test report may include test results of a plurality of test cases.
It can be seen that in the embodiment of the present invention, the initialization process is performed before the test process is performed. In the initialization process, configuration data and advertisement materials are filled into the empty database, the environment of the database is configured into an expected environment, and meanwhile, server parameters of the BS are also configured. In this way, consistency of the database configuration data and the server parameters of the BS with the expected response can be ensured. When automatic test is carried out, the BS triggers and executes different logics to return different responses due to the change of configuration data and server parameters, so that erroneous judgment of the BS is reduced.
In other embodiments of the present invention, referring to fig. 3, after the test case is executed, the automatic test method may further include:
s6: the automated test framework system deletes the data in the database.
The database is emptied after the test is completed, so that the influence on the final execution effect due to different test case sequences can be avoided.
In other embodiments of the present invention, referring still to fig. 3, after the test case is executed, the automatic test method may further include:
s7, the automatic test framework system restores the server parameters of the BS to default values.
Referring to fig. 4, the following describes an automatic test method according to an embodiment of the present invention with a more specific example, which may include:
s41: the automated test framework system initializes the configuration table in the database to a default value.
It should be noted that there may be multiple configuration tables, the test case may be configured for one or more configuration tables, and other configuration tables may use default values.
S42: the automatic test framework system loads test cases.
The execution order of step S42 and step S41 is interchangeable.
The relevant description of the test case is referred to the foregoing description, and will not be repeated here.
S43: and the automatic test framework system fills configuration data and advertisement materials into the test database according to the initialization flow in the test case, and updates the server parameters of the BS.
Specifically, the server parameters are in a file form, and the automatic test framework system can log in the BS by using a ssh related module (the ssh module is a code function related to a remote login service and is mainly used for logging in a server, downloading logs in a connected mode and checking the logs), so that the server parameters are updated.
The automatic test framework system may perform "inject configuration data and advertisement materials into the database for test" and "configure server parameters of BS" using two modules, respectively.
Refer to the aforementioned step S2 for other related descriptions, and are not repeated here.
S44: the BS acquires configuration data in the database for test.
The advertising material and configuration data may be pulled from the database by the aforementioned pulling unit 2 times per day, written to the BS service by ft.
In addition, updated contents of the DB may be acquired in real time through the previous real-time update module and written into the BS service.
S45: the automatic test framework system sends an advertisement request (http request) to the BS according to the test flow in the test case.
The advertisement request may include a user identification, ad spot information, etc.
Step S45 simulates an operation in which the client requests an advertisement.
S46: the BS returns an http response.
S47: the automated test framework system compares the returned responses to the expected responses in the test cases.
S48: the automatic test framework system acquires the log of the BS and matches the expected log.
The BS writes text files into the logs generated by the advertisement requests, and the automatic test framework system reads the text files to perform relevant text matching to verify whether the text files accord with expectations.
S49: and the automatic test framework system generates a test report according to the comparison result.
The test report may be an HTML test report.
Test cases that succeed in execution, as well as test cases that fail in execution, may be counted in the test report.
If the log comparison is consistent and the http response comparison is consistent, the test case is successfully executed, otherwise, the test case fails.
For failed test cases, a corresponding log can be attached to the test report so as to facilitate positioning.
S410: the automated test framework system deletes the data in the database.
An automatic test system is described below. Referring to fig. 5, exemplary includes:
the loading unit 1 is used for loading test cases;
the test case comprises an initialization flow, a test flow and an expected response;
the database configuration unit 2 is used for filling configuration data and advertisement materials into the database for test according to the initialization flow in the test case;
the BS configuration unit 3 is used for configuring server parameters of the BS according to the initialization flow in the test case; the expected response is set for the ad request in the test flow, the ad material, server parameters, and temporary configuration data;
a test unit 4 for:
according to the test flow in the test case, sending an advertisement request to the BS;
receiving a response returned by the BS, and comparing the returned response with an expected response to generate a test report; the test report includes at least: and information representing success or failure of test case test.
The specific description is referred to the foregoing description, and will not be repeated here.
In addition, the test unit 4 may also obtain a log of the advertisement request from the BS and compare the log with the expected log, and the test report may further include the log generated by the BS for the advertisement request. The specific description is referred to the foregoing description, and will not be repeated here.
In other embodiments of the present invention, the database configuration unit 2 is further configured to: after the test cases are executed, deleting the data in the database.
The specific description is referred to the foregoing description, and will not be repeated here.
The BS configuration unit 3 described above is also configured to: after the test case is executed, the server parameters of the BS are restored to the default values.
The specific description is referred to the foregoing description, and will not be repeated here.
In the present specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant points refer to the description of the method section.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software unit executed by a processor, or in a combination of the two. The software elements may be disposed in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. An automatic test method for testing an advertisement engine BS, which is characterized by being applied to an automatic test system; the automatic test system includes: an automatic test framework system and a database for test; before an automatic test is performed, the test database is an empty database;
the method comprises the following steps:
the automatic test framework system loads test cases; the test case comprises an initialization flow, a test flow and an expected response;
the automatic test framework system fills configuration data and advertisement materials into the test database according to the initialization flow in the test case, and configures server parameters of the BS; the expected response is set for the configuration data, the advertising material, the server parameters, and the advertising request in the test flow;
the automatic test framework system sends an advertisement request to the BS according to the test flow in the test case;
the automatic test framework system receives a response returned by the BS, compares the returned response with the expected response and generates a test report; the test report includes at least: and characterizing the success or failure information of the test case test.
2. The method as recited in claim 1, further comprising: and deleting the data in the database after the test case is executed.
3. The method as recited in claim 2, further comprising: and after the test case is executed, restoring the server parameters of the BS to default values.
4. The method of claim 3, wherein the test report further comprises: the BS generates a log for the advertisement request.
5. An automatic test system for testing an advertisement engine BS, comprising: an automatic test framework system and a database for test; before an automatic test is performed, the test database is an empty database;
the automatic test framework system comprises:
the loading unit is used for loading the test cases; the test case comprises an initialization flow, a test flow and an expected response;
the database configuration unit is used for filling configuration data and advertisement materials into the test database according to the initialization flow in the test case;
the BS configuration unit is used for configuring server parameters of the BS according to the initialization flow in the test case; the expected response is set for the configuration data, the advertising material, the server parameters, and the advertising request in the test flow;
a test unit for:
sending an advertisement request to the BS according to the test flow in the test case;
receiving a response returned by the BS, comparing the returned response with the expected response, and generating a test report; the test report includes at least: and characterizing the success or failure information of the test case test.
6. The system of claim 5, wherein the database configuration unit is further to: and deleting the data in the database after the test case is executed.
7. The system of claim 6, wherein the BS configuration unit is further to: and after the test case is executed, restoring the server parameters of the BS to default values.
8. The system of claim 7, wherein the test report further comprises: the BS generates a log for the advertisement request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911181263.6A CN110941559B (en) | 2019-11-27 | 2019-11-27 | Automatic test method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911181263.6A CN110941559B (en) | 2019-11-27 | 2019-11-27 | Automatic test method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110941559A CN110941559A (en) | 2020-03-31 |
| CN110941559B true CN110941559B (en) | 2023-10-03 |
Family
ID=69908607
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911181263.6A Active CN110941559B (en) | 2019-11-27 | 2019-11-27 | Automatic test method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110941559B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112925725B (en) * | 2021-04-09 | 2024-03-15 | 网易（杭州）网络有限公司 | Data testing method and device, readable storage medium and electronic equipment |
| CN113485919B (en) * | 2021-06-30 | 2024-01-26 | 山东有人物联网股份有限公司 | Test method, test device, and computer-readable storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012135748A2 (en) * | 2011-03-31 | 2012-10-04 | Google Inc. | Integrated mobile/server applications |
| CN110162468A (en) * | 2019-04-26 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of test method, device and computer readable storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110088011A1 (en) * | 2009-10-14 | 2011-04-14 | Vermeg Sarl | Automated Enterprise Software Development |
| US20140073420A1 (en) * | 2012-09-07 | 2014-03-13 | Downing Matthew | System and method for optimizing user value in an online environment |
-
2019
- 2019-11-27 CN CN201911181263.6A patent/CN110941559B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2012135748A2 (en) * | 2011-03-31 | 2012-10-04 | Google Inc. | Integrated mobile/server applications |
| CN110162468A (en) * | 2019-04-26 | 2019-08-23 | 腾讯科技（深圳）有限公司 | A kind of test method, device and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 叶永鑫 ; 丁晓明 ; .实时嵌入式产品的测试系统设计.单片机与嵌入式系统应用.2010,(07),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110941559A (en) | 2020-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7900216B2 (en) | Creating a modified service request and processing the results of its execution | |
| CN108347356B (en) | Multi-protocol unit testing method and device, electronic equipment and storage medium | |
| CN106844217A (en) | Control to applying bury method and device, readable storage medium storing program for executing a little | |
| CN110941559B (en) | Automatic test method and system | |
| US10303463B2 (en) | Terminal management framework and terminal management method | |
| CN109165170B (en) | Method and system for automatic request test | |
| CN103984573A (en) | Installation method and device of application program | |
| CN109344053B (en) | Interface coverage test method, system, computer device and storage medium | |
| CN111290936A (en) | Interface testing method and device | |
| CN107896244A (en) | A kind of distribution method of version file, client and server | |
| CN110659202A (en) | Client automatic testing method and device | |
| US11573780B2 (en) | Automated generation of status chains for software updates | |
| CN107302476A (en) | A kind of automated testing method and system for testing asynchronous interactive system | |
| CN102572896A (en) | Upgrading method and upgrading device for wireless communication system | |
| CN110833693B (en) | Game running method and device, storage medium and electronic device | |
| CN110196800A (en) | Distributed transaction test method, apparatus and system | |
| CN107948942A (en) | A kind of short message service automated testing method and device | |
| CN113791765A (en) | Resource arranging method, device and equipment of cloud service and storage medium | |
| CN109062786A (en) | Operation method, device and the terminal of test case | |
| CN112799782A (en) | Model generation system, method, electronic device, and storage medium | |
| CN109739704A (en) | A kind of interface test method, server-side and computer readable storage medium | |
| CN116257438A (en) | Updating method of interface test case and related equipment | |
| CN106293842A (en) | A kind of software installation method and device | |
| CN105991700A (en) | Voice data processing method, cloud server system and terminal equipment | |
| US9053084B1 (en) | Self-service testing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |