CN111026668B - Retry method of test case, retry device of test case and storage medium - Google Patents
Retry method of test case, retry device of test case and storage medium Download PDFInfo
- Publication number
- CN111026668B CN111026668B CN201911270840.9A CN201911270840A CN111026668B CN 111026668 B CN111026668 B CN 111026668B CN 201911270840 A CN201911270840 A CN 201911270840A CN 111026668 B CN111026668 B CN 111026668B
- Authority
- CN
- China
- Prior art keywords
- retry
- test case
- preset
- mode
- test
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Retry When Errors Occur (AREA)
Abstract
The invention discloses a retry method of a test case, which comprises the following steps: when detecting that the test case fails to execute, acquiring a preset retry mode, wherein the preset retry mode comprises at least one of a TestNg retry mode, a current process retry mode and a restart process retry mode; and re-executing the test case according to the preset retry mode. The invention also discloses a retry device of the test case and a computer readable storage medium. The invention improves the stability of the test case with failed retry execution.
Description
Technical Field
The present invention relates to the field of software testing technologies, and in particular, to a test case retry method, a test case retry device, and a computer readable storage medium.
Background
At present, since the automatic test cases are usually not very stable, the execution test fails occasionally due to the environment or the robustness of the test cases, and then a function of failing automatic retry is needed, so that the test cases can automatically initiate retry under the failure condition. However, the automatic retry process of the current test case often causes retry failure due to system reasons, and the automatic retry stability is poor.
The foregoing is provided merely for the purpose of facilitating understanding of the technical solutions of the present invention and is not intended to represent an admission that the foregoing is prior art.
Disclosure of Invention
The invention mainly aims to provide a retry method of a test case, a retry device of the test case and a computer readable storage medium, which improve the stability of the test case in which the retry execution fails.
In order to achieve the above object, the present invention provides a retry method of a test case, the retry method of the test case including the steps of:
when detecting that the test case fails to execute, acquiring a preset retry mode, wherein the preset retry mode comprises at least one of a TestNg retry mode, a current process retry mode and a restart process retry mode;
and re-executing the test case according to the preset retry mode.
Optionally, the step of performing the retry of the execution of the test case according to the preset retry mode includes:
when the preset retry mode is the TestNg retry mode, executing the test case which fails currently again each time the test case is detected to fail;
when the preset retry mode is the current process retry mode, after all test cases are executed, acquiring the test cases with failed execution as target test cases, and executing the target test cases one by one again;
and restarting the test process when the preset retry mode is the restart process retry mode, and re-executing the test case based on the restarted test process.
Optionally, after the step of acquiring the preset retry mode when the failure of executing the test case is detected, the method further includes:
when at least two preset retry modes are obtained, the priority corresponding to the preset retry modes is obtained;
and re-executing the test case by adopting the preset retry mode with the highest priority.
Optionally, after the step of re-executing the test case by adopting the preset retry mode with the highest priority, the method further includes:
and when the currently executed preset retry mode fails to retry, executing the test case again by adopting the preset retry mode of the next-level priority.
Optionally, after the step of re-executing the test case by adopting the preset retry mode with the highest priority, the method further includes:
and when the number of times of the currently executed preset retry mode is larger than the preset number of times, judging that the currently executed preset retry mode fails to retry.
Optionally, the retry method of the test case further includes:
providing a selection interface comprising all preset retry modes;
and when the preset retry mode is acquired, acquiring the preset retry mode selected based on the selection interface.
In order to achieve the above object, the present invention further provides a retry apparatus for a test case, the retry apparatus for a test case including:
the retry device of the test case comprises a memory, a processor and a retry program of the test case stored in the memory and capable of running on the processor, wherein the retry program of the test case realizes the steps of the retry method of the test case when being executed by the processor.
In order to achieve the above object, the present invention further provides a computer readable storage medium, where a retry program of a test case is stored, where the retry program of the test case, when executed by a processor, implements the steps of the retry method of the test case.
The invention provides a retry method of a test case, a retry device of the test case and a computer readable storage medium, when detecting that the test case fails to execute, a preset retry mode is obtained, wherein the preset retry mode comprises at least one of a TestNg retry mode, a current process retry mode and a restart process retry mode; and re-executing the test case according to the preset retry mode. Thus, stability in retrying a failed test case is improved.
Drawings
FIG. 1 is a schematic diagram of a hardware operating environment of an embodiment terminal according to an embodiment of the present invention;
FIG. 2 is a flowchart of a first embodiment of a retry method of a test case according to the present invention;
FIG. 3 is a flowchart illustrating a retry method according to a second embodiment of the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The invention provides a retry method of a test case, which improves the stability of the test case in which the retry execution fails.
As shown in fig. 1, fig. 1 is a schematic diagram of a hardware running environment of an embodiment terminal according to an embodiment of the present invention;
the terminal of the embodiment of the invention can be a retry device of the test case, and the retry device of the test case can be a computer.
As shown in fig. 1, the terminal may include: a processor 1001, such as a CPU central processing unit (central processing unit), a memory 1002, and a communication bus 1003. The communication bus 1003 is used to implement connection communication between the constituent elements in the terminal. The memory 1002 may be a high-speed RAM Random Access Memory (RAM) or a stable memory (non-volatile memory), such as a disk memory. The memory 1002 may alternatively be a storage device separate from the processor 1001 described above.
It will be appreciated by those skilled in the art that the configuration of the terminal shown in fig. 1 is not limiting of the terminal of the embodiments of the present invention, and may include more or less components than those illustrated, or may combine certain components, or may be arranged in different components.
As shown in fig. 1, a retry program of a test case may be included in a memory 1002 as a kind of computer storage medium.
In the terminal shown in fig. 1, the processor 1001 may be configured to call a retry program of a test case stored in the memory 1002, and perform the following operations:
when detecting that the test case fails to execute, acquiring a preset retry mode, wherein the preset retry mode comprises at least one of a TestNg retry mode, a current process retry mode and a restart process retry mode;
and re-executing the test case according to the preset retry mode.
Further, the processor 1001 may call a retry program of the test case stored in the memory 1002, and further perform the following operations:
when the preset retry mode is the TestNg retry mode, executing the test case which fails currently again each time the test case is detected to fail;
when the preset retry mode is the current process retry mode, after all test cases are executed, acquiring the test cases with failed execution as target test cases, and executing the target test cases one by one again;
and restarting the test process when the preset retry mode is the restart process retry mode, and re-executing the test case based on the restarted test process.
Further, the processor 1001 may call a retry program of the test case stored in the memory 1002, and further perform the following operations:
when at least two preset retry modes are obtained, the priority corresponding to the preset retry modes is obtained;
and re-executing the test case by adopting the preset retry mode with the highest priority.
Further, the processor 1001 may call a retry program of the test case stored in the memory 1002, and further perform the following operations:
and when the currently executed preset retry mode fails to retry, executing the test case again by adopting the preset retry mode of the next-level priority.
Further, the processor 1001 may call a retry program of the test case stored in the memory 1002, and further perform the following operations:
and when the number of times of the currently executed preset retry mode is larger than the preset number of times, judging that the currently executed preset retry mode fails to retry.
Further, the processor 1001 may call a retry program of the test case stored in the memory 1002, and further perform the following operations:
providing a selection interface comprising all preset retry modes;
and when the preset retry mode is acquired, acquiring the preset retry mode selected based on the selection interface.
Referring to fig. 2, in an embodiment, the retry method of the test case includes:
step S10, when the failure of executing the test case is detected, a preset retry mode is obtained, wherein the preset retry mode comprises at least one of a TestNg retry mode, a current process retry mode and a restart process retry mode.
In this embodiment, the embodiment terminal may be a retry apparatus for a test case, and the retry apparatus for a test case may be specifically a computer.
Alternatively, the test case currently executed by the terminal may be an interface for a test system, or may be a test server. It should be noted that, a Test Case refers to a description of a Test task performed on a specific software product, which embodies a Test scheme, a method, a technique and a policy. The content of the method comprises a test target, a test environment, input data, a test step, an expected result, a test script and the like, and finally a document is formed. Briefly, a test case is a set of test inputs, execution conditions, and expected results tailored for a particular goal to verify that a particular software requirement is met.
Optionally, the terminal is preset with a plurality of preset retry modes, and when detecting that the test case fails to execute, at least one preset retry mode is obtained from the terminal to retry the test case that fails to execute.
Optionally, the preset retry mode includes at least one of a TestNg retry mode, a current process retry mode, and a restart process retry mode.
The TestNg retry mode is implemented based on an open-source TestNg framework, and when a single test case is identified, the test case is retried immediately. The current process retry mode is to uniformly acquire the test cases with execution failure based on the current test thread for retry after all the test cases are executed. The restarting process retrying mode is to restart the test thread after all the test cases are executed, and then uniformly acquire the test cases with failed execution based on the restarted test thread for retrying.
Optionally, the preset retry mode configured by the terminal may be configured by an engineer in advance by acquiring at least one preset test mode from the TestNg retry mode, the current process retry mode, and the restart process retry mode.
Optionally, the preset retry mode configured by the terminal may be set autonomously by the user according to the own needs. The terminal is provided with a selection interface of preset retry modes, wherein the selection interface comprises all the preset retry modes, and a user can select the preset retry modes based on the selection interface and select at least one preset retry mode. When the terminal executes the step of acquiring the preset retry mode, the terminal only needs to acquire the preset retry mode selected by the user based on the selection interface.
And step S20, re-executing the test case according to the preset retry mode.
Optionally, when the preset retry mode obtained by the terminal is a TestNg retry mode, whenever the execution failure of the test case is detected, the currently failed test case is re-executed, that is, the retry mode is to retry immediately after the execution failure of one test case in the test process.
Optionally, the terminal configures retry variables of the TestNg retry mode in a test set consisting of a plurality of test cases: maxretries=number of retries to set the number of retries. When adopting the TestNg retry mode, logic calculates whether the retry is needed currently according to the current retry times and the set retry times after the test case fails, if the retry times are not reached, the retry method is used for returning the true value to indicate the retry, otherwise, the false value is returned to indicate that the retry is not performed.
Note that when maxRetries is set to 0 or a corresponding value is not set, the test ng retry mode is disabled by default.
In this way, table parameterization can be realized by adopting a TestNg retry mode, and each row of data retry is supported.
Optionally, when the preset retry mode acquired by the terminal is the current process retry mode, after all test cases are executed, acquiring the test cases with failed execution as target test cases, and executing the target test cases one by one in the current test process.
When the terminal detects that the test cases with the execution failure exist in the current test case set after all the test cases in one test case set are executed, the terminal takes the failed test cases as target test cases and executes retry on the target test cases one by one based on the current test process by adopting an API (Application Programming Interface, application program interface) execution mode of directly adjusting the test ng. In the retry mode, the retry is executed by calling testng.run during the retry, wherein the target test case needing to be retried is from a TestResultHolder and is obtained from a failure test case list cached in the previous test case execution process.
Optionally, the terminal configures retry variables of the current process retry mode in a test set consisting of a plurality of test cases: subtereies = number of retries to uniformly set the number of retries for a single target test case. When each target test case is retried, after the execution fails, the terminal detects whether the current retry number is greater than the retry number, if so, the target test case is abandoned and the next test case is retried, and if not, the target test case is retried continuously.
It should be noted that, when the suite retries is set to 0 value or the corresponding value is not set, the current process retry mode is disabled by default.
Thus, the retry of the failed test case by using the current process retry method has a higher retry success rate than the testNg retry method (because it is the same process, it is not a clean retry, such as a static variable, etc., the code of the before/after suite is re-executed, but spring does not repeat initialization).
Optionally, when the preset retry mode acquired by the terminal is a restart process retry mode, restarting the test process, and re-executing the failed test case based on the restarted test process. Restarting the process is controlled by the vtp-agent program, and each retry restarts the test process. The method comprises the following specific steps:
(1) Judging whether the current execution retry number is smaller than a retryCount value (retry number), and whether a failed test case exists, and executing retry if both the test cases are satisfied;
(2) Recursion retry, the dorunntask is a recursion method, and each time the local end is debugged in retry, the current test process can jump out of the recursion cycle only if the retry condition in the step (1) is not satisfied;
(3) When the retry is executed for the last time (the retry for the last time or the retry of all test cases is successful), the vtp-agent program will combine the retry result with the original result, and the logic of the combination is: traversing the retry result list, finding out the corresponding test case result in the original result according to the unique ID of the test case result in the original result list, merging the test case retry result into the test case result, and covering the test case result recorded when the execution of the original test case fails.
In this way, the retry of the failed test case is performed by adopting a restarting process retry mode, the initialization test process is re-executed, the clean execution is performed, and the retry success rate is higher.
In an embodiment, when detecting that the test case fails to execute, acquiring a preset retry mode, where the preset retry mode includes at least one of a TestNg retry mode, a current process retry mode, and a restart process retry mode; and re-executing the test case according to the preset retry mode. Thus, stability in retrying a failed test case is improved.
In a second embodiment, as shown in fig. 3, based on the embodiment shown in fig. 2, after the step of obtaining the preset retry mode when the failure of executing the test case is detected, the method further includes:
step S30, when at least two preset retry modes are obtained, the priority corresponding to the preset retry modes is obtained.
And S40, re-executing the test case by adopting the preset retry mode with the highest priority.
In this embodiment, the terminal is configured with at least two preset retry modes.
Optionally, the terminal is configured with two preset retry modes, namely a TestNg retry mode and a current process retry mode.
Optionally, the terminal is configured with two preset retry modes, namely a TestNg retry mode and a restart process retry mode.
Optionally, the terminal is configured with two preset retry modes, namely a current process retry mode and a restart process retry mode.
Optionally, the terminal is configured with three preset retry modes, namely a TestNg retry mode, a current process retry mode and a restart process retry mode.
Optionally, different types of preset retry modes are preconfigured with corresponding priorities, when the terminal acquires at least two preset retry modes, the terminal acquires the priorities corresponding to each preset retry mode respectively, and then the test case with the failed previous execution is re-executed by adopting the preset retry mode with the highest priority. The priority of the TestNg retry mode is higher than the priority of the current process retry mode, and the priority of the current process retry mode is higher than the priority of the restarting process retry mode.
Optionally, when the terminal detects that the retry of the currently executed preset retry mode fails, acquiring a preset retry mode of the next level of priority, that is, acquiring a preset retry mode with the highest priority except for the currently executed preset retry mode, and re-executing the test case which is still not executed successfully by adopting the acquired preset retry mode.
In this way, by setting the priority of the preset retry mode, when the currently executed preset retry mode fails, the preset retry mode of the next priority is acquired to perform the retry of the test case of the execution failure.
Optionally, each preset retry mode is preset with a corresponding executed retry number, when the terminal detects that the retry number of the currently executed preset retry mode is greater than the preset number, it is determined that the currently executed preset retry mode fails to retry, and then the preset retry mode of the next level priority is acquired to retry the test case of the execution failure.
It should be noted that, the specific value of the number of retries executed by each preset retry method may be set according to the actual situation, which is not limited in this embodiment.
For example, when the currently acquired preset retry mode is a TestNg retry mode, after the retry of the TestNg retry mode fails, the terminal acquires the retry of the failed test case in the current process retry mode of the next priority, and after the retry of the current process retry mode fails, the terminal acquires the retry of the failed test case in the restart process retry mode of the next priority.
Therefore, by setting at least two preset retry modes and setting corresponding retry times, the effects of improving the execution stability of the test case and meeting the requirements of users on different preset retry modes are achieved.
In addition, the invention also provides a retry device of the test case, which comprises a memory, a processor and a retry program of the test case stored on the memory and capable of running on the processor, wherein the steps of the retry method of the test case described in the above embodiment are realized when the processor executes the retry program of the test case.
In addition, the invention also provides a computer readable storage medium, which comprises a retry program of the test case, wherein the retry program of the test case realizes the steps of the retry method of the test case according to the embodiment.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of embodiments, it will be clear to a person skilled in the art that the above embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course also by means of hardware, but in many cases the former is an alternative embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) as described above, comprising instructions for causing a terminal device (which may be a television, a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The foregoing description of the invention and the accompanying drawings is only illustrative, and is not intended to limit the scope of the invention, but rather is to be construed in any way in view of the following description, the accompanying drawings and the like.
Claims (7)
1. The retry method of the test case is characterized by comprising the following steps:
when detecting that the test case fails to execute, acquiring a preset retry mode, wherein the preset retry mode comprises at least one of a TestNg retry mode, a current process retry mode and a restart process retry mode;
when at least two preset retry modes are obtained, the priority corresponding to the preset retry modes is obtained;
and re-executing the test case by adopting a preset retry mode with the highest priority.
2. The retry method of claim 1, wherein the step of re-executing the test case in the predetermined retry mode with the highest priority comprises:
when the preset retry mode is the TestNg retry mode, executing the test case which fails currently again each time the test case is detected to fail;
when the preset retry mode is the current process retry mode, after all test cases are executed, acquiring the test cases with failed execution as target test cases, and executing the target test cases one by one again;
and restarting the test process when the preset retry mode is the restart process retry mode, and re-executing the test case based on the restarted test process.
3. The method for retrying a test case according to claim 1, wherein after the step of re-executing the test case by using a preset retry method with a highest priority, the method further comprises:
and when the currently executed preset retry mode fails to retry, executing the test case again by adopting the preset retry mode of the next-level priority.
4. The method for retrying a test case according to claim 1, wherein after the step of re-executing the test case by using a preset retry method with a highest priority, the method further comprises:
and when the number of times of the currently executed preset retry mode is larger than the preset number of times, judging that the currently executed preset retry mode fails to retry.
5. The test case retry method of claim 1, wherein said test case retry method further comprises:
providing a selection interface comprising all preset retry modes;
and when the preset retry mode is acquired, acquiring the preset retry mode selected based on the selection interface.
6. A test case retry apparatus comprising a memory, a processor, and a test case retry program stored on the memory and executable on the processor, the test case retry program implementing the test case retry method steps of any one of claims 1 to 5 when executed by the processor.
7. A computer readable storage medium, wherein a retry program of a test case is stored on the computer readable storage medium, and the retry program of the test case, when executed by a processor, implements the steps of the retry method of the test case according to any one of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911270840.9A CN111026668B (en) | 2019-12-10 | 2019-12-10 | Retry method of test case, retry device of test case and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911270840.9A CN111026668B (en) | 2019-12-10 | 2019-12-10 | Retry method of test case, retry device of test case and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111026668A CN111026668A (en) | 2020-04-17 |
| CN111026668B true CN111026668B (en) | 2023-10-24 |
Family
ID=70208203
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911270840.9A Active CN111026668B (en) | 2019-12-10 | 2019-12-10 | Retry method of test case, retry device of test case and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111026668B (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103500142A (en) * | 2013-10-12 | 2014-01-08 | 南京大学 | Method for testing multiple target test case priorities facing dynamic Web application |
| CN108287789A (en) * | 2017-12-29 | 2018-07-17 | 五八有限公司 | Batch-automated test method, device and electronic equipment |
| CN108459961A (en) * | 2017-12-29 | 2018-08-28 | 微梦创科网络科技（中国）有限公司 | The method, client and server of examination are resurveyed after a kind of failure of testing case |
-
2019
- 2019-12-10 CN CN201911270840.9A patent/CN111026668B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103500142A (en) * | 2013-10-12 | 2014-01-08 | 南京大学 | Method for testing multiple target test case priorities facing dynamic Web application |
| CN108287789A (en) * | 2017-12-29 | 2018-07-17 | 五八有限公司 | Batch-automated test method, device and electronic equipment |
| CN108459961A (en) * | 2017-12-29 | 2018-08-28 | 微梦创科网络科技（中国）有限公司 | The method, client and server of examination are resurveyed after a kind of failure of testing case |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111026668A (en) | 2020-04-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7908521B2 (en) | Process reflection | |
| CN102866930B (en) | Blue screen restorative procedure, Apparatus and system | |
| CN108804215B (en) | Task processing method and device and electronic equipment | |
| CN105302722B (en) | CTS automatic testing method and device | |
| CN110795128B (en) | Program bug repairing method and device, storage medium and server | |
| CN108121543B (en) | Software code compiling processing method and device | |
| US9430221B2 (en) | Device and method for updating firmware of a rackmount server system | |
| WO2018006702A1 (en) | Abnormality processing method, apparatus and system in automation test | |
| CN102855160A (en) | Method, device and system for software repair | |
| CN103019872B (en) | Browser restorative procedure and device | |
| CA3129985A1 (en) | Abnormal operation environment restoration method and device, computer equipment and storage medium | |
| CN110597545A (en) | Hot patch intelligent upgrading method and system based on OTA component | |
| CN111124509B (en) | Server starting method and device | |
| CN109960658B (en) | Application programming interface testing method and device, electronic equipment and storage medium | |
| KR102705465B1 (en) | Methods for handling abnormalities, terminal devices and storage media | |
| US7882399B2 (en) | Intelligent job functionality | |
| CN111026668B (en) | Retry method of test case, retry device of test case and storage medium | |
| US9389948B2 (en) | Method and system for fixing loopholes | |
| US20230315620A1 (en) | System and Method for Diagnosing a Computing Device in Safe Mode | |
| CN102999555A (en) | Webpage exception handling method and device | |
| CN104699554B (en) | Software repair, apparatus and system | |
| CN110442493B (en) | Automatic service management system and method | |
| CN109284137B (en) | Hypervisor-based QNX operating system starting method and device | |
| CN107423087A (en) | A kind of application program updating method that server can monitor | |
| CN108595292B (en) | System optimization method, mobile terminal and computer storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |