WO2018006702A1

WO2018006702A1 - Abnormality processing method, apparatus and system in automation test

Info

Publication number: WO2018006702A1
Application number: PCT/CN2017/088937
Authority: WO
Inventors: 郇昌波
Original assignee: 中兴通讯股份有限公司
Priority date: 2016-07-05
Filing date: 2017-06-19
Publication date: 2018-01-11
Also published as: CN107577591A

Abstract

An abnormality protection method, apparatus and system in an automation test. The method comprises: during the process of testing a device to be tested according to a test script, if a connection with the device to be tested is abnormally interrupted, according to the interruption situation, determining a configuration command needing to be executed after the device to be tested recovers to a pre-set original state; and after the device to be tested is re-connected, controlling the device to be tested so that same recovers to the original state and issuing the configuration command needing to be executed to the device to be tested.

Description

Exception handling method, device and system in automated testing

Technical field

The present disclosure relates to the field of automated testing, for example, to an anomaly processing method, apparatus, and system in an automated test.

Background technique

Automated testing has gradually penetrated into many aspects of software testing and has irreplaceable advantages. Automated testing uses simple scripts to achieve a large number of repetitive operations, which can reduce test time to a certain extent, save test manpower, and test results are accurate. Freeing test human resources through automated testing enables testers to focus on manual testing, improve test efficiency, shorten product development cycles, and increase R&D efficiency.

However, during the automated test process, the transmission control protocol TCP (Transmission Control Protocol (TCP) connection between the automated test system and the device under test may be abnormal due to the performance of the device under test, the stability of the test environment, and other unstable factors. Interrupted. Such a situation not only causes the running script to fail, but may even affect subsequent automated tests.

Summary of the invention

A method, device and system for exception processing in an automated test, which avoids an abnormal interruption of a TCP connection between an automated test system and a device under test in the related art, causes the running script to fail, and may even affect subsequent automation Test the phenomenon.

An exception handling method in automated testing, including:

In the process of performing an automated test on a device under test according to a test script, if the connection with the device under test is abnormally interrupted, determining, according to the abnormal interruption, that the device under test returns to a preset original state. The configuration command that the device under test needs to execute;

After the device is reconnected, the device under test is restored to the original state, and the configuration command to be executed is sent to the device under test.

Optionally, the determining, according to the abnormal interruption condition, that the device under test returns to a preset original The configuration commands that need to be executed by the device under test after the status include:

Determining, by the device under test, the original state that was restored to the last time before the abnormal interrupt occurs, and determining that the configuration command that the device under test needs to execute after the device is restored to the original state is the configuration command. .

Optionally, the determining, according to the abnormal interruption, that the device under test needs to be executed after the device under test returns to a preset original state, includes:

After the connection with the device under test is abnormally interrupted, obtaining a configuration command that fails to be delivered in the test script;

Determining, according to the configuration command that fails to be delivered, a configuration command that the device under test needs to perform after the device under test returns to the original state.

Optionally, in the process of automatically testing a device under test according to a test script, the method includes:

Detecting whether a connection to the device under test is abnormally interrupted;

When the connection with the device under test is abnormally interrupted, a configuration command that fails to be delivered in the test script is obtained.

Optionally, in the process of testing a device under test according to a test script, the method includes:

A configuration command sent to the device under test according to the test script is recorded.

An anomaly processing device for automated testing, comprising:

Determining a module, in the process of performing an automatic test on a device under test according to a test script, if an abnormal interruption occurs in a connection with the device under test, determining that the device under test is restored according to the abnormal interruption a configuration command that the device under test needs to execute after the preset original state;

After the control module is configured to reconnect with the device under test, the device is controlled to be restored to the original state, and the configuration command to be executed is sent to the device under test.

Optionally, the determining module includes:

a first determining unit, configured to determine that the device under test is last before an abnormal interruption occurs The original state is restored to the next time, and the configuration command that the device under test needs to execute after the device is restored to the original state is determined.

Optionally, the determining module includes:

Obtaining a configuration command that is sent in the test script after the connection to the device under test is abnormally interrupted;

The first determining unit is configured to determine, according to the configuration command that the delivery fails, a configuration command that the device under test needs to perform after the device under test returns to the original state.

Optionally, the device further includes:

The recording module is configured to record a configuration command sent to the device under test according to the test script during an automated test of a device under test according to a test script.

An automated test system comprising: an exception handling device in any of the above automated tests.

A computer readable storage medium storing computer executable instructions arranged to perform the above method.

An automated test system that includes:

At least one processor;

a memory communicatively coupled to the at least one processor; wherein

The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the method described above.

In the above method for handling exceptions in an automated test, in the process of automatically testing a device under test according to a test script, if the connection with the device under test is abnormally interrupted, the device under test is determined according to the abnormal interruption. After the device is restored to the original state, the device to be tested needs to be executed. After the device is reconnected, the device under test is restored to the original state, and the configuration command to be executed is sent to the device under test. After reconnecting with the device under test, the control device is restored to the preset original state, and the configuration command after the determined original state is executed, the interrupted configuration command can be quickly restored, so that the subsequent automated test can be smoothly performed. It avoids the failure of the test script to run, ensures the continuity of the automated test, and improves the system's fault tolerance. In this method, after reconnecting with the device, it is not necessary to re-test from the initial configuration command, which saves the time for restoring the configuration and improves the efficiency of the automated test. The above technical solution solves related technologies An abnormal interruption in the TCP connection between the automated test system and the device under test causes the running script to fail, and may even affect the subsequent automated test execution.

DRAWINGS

1 is a flow chart of a method for exception handling in an automated test in an embodiment;

2 is a schematic structural diagram of an apparatus for abnormal processing in an automated test in an embodiment;

3 is a schematic diagram showing the hardware structure of an automated test system in an embodiment.

detailed description

In order to make the technical solutions clearer, the technical solutions will be described in detail below with reference to the accompanying drawings and embodiments.

The system fails to send a configuration command to the device when the connection between the system (automated test system) and the device (the device under test) is abnormally interrupted. In this case, the configuration command that is not sent to the device fails. Configuration command. After the system reconnects to the abnormally interrupted device, if the device continues to execute the failed configuration command, the configuration mode of the current test device may not match the configuration command delivered by the system. The configuration command fails. The subsequent automatic test fails. . Therefore, restoring the configuration mode of the currently tested device can cause the device to continue the automated test after an abnormal interruption.

In the method of the embodiment, after the system is reconnected with the device by default, the device can be restored to the original state, and the configuration command after the original state is executed, so that the device continues to perform automatic testing to ensure automation. Test the continuity of the device configuration.

As shown in FIG. 1, the abnormal processing method of the automated test in this embodiment includes the following steps.

In step 101, in the process of testing a device under test according to a test script, if an abnormal interruption occurs in the connection with the device under test, it is determined that the device under test is restored to the preset according to the abnormal interruption. The configuration command that the device under test needs to execute after the original state.

The original state may be the state of the device (the device under test) customized in advance through the automated test platform. For example, the original state may be set as the initial state after the device is reconnected with the system or the state in which the device enters the basic configuration mode after reconnecting with the system. .

The automated testing process may include multiple sets of different configuration commands, and the tested device may restore the original state after executing a complete set of configuration commands.

After the device is restored to the original state, it is determined that the configuration command to be executed by the device under test can be configured before the device is restored to the original state. Re-execute the interrupted configuration command to quickly break away from the abort.

After the device is reconnected with the device under test, the device under test is restored to the original state, and the configuration command to be executed is sent to the device under test.

After the device is controlled to return to the original state, the device can execute the configuration command issued by the system after the original state, and the device executes the configuration command that fails, and can continue to perform the automatic test to ensure the continuity of the automated test.

The automated test platform can have the ability to restore the original state, which can easily control the tested device to return to the original state.

The method for automatically testing the abnormality protection of the embodiment can restore the preset original state by the control device after reconnecting with the device, and execute the configuration command after the original state, and execute the interrupt configuration command, so that the subsequent automation The test can be carried out smoothly, avoiding the failure of the test script, ensuring the continuity of the automated test and improving the fault tolerance of the system. After the system is reconnected with the device, it can be re-tested without the initial configuration command in the automated test, saving the time for restoring the configuration, improving the efficiency of the automated test, and solving the related technology between the automated test system and the device under test. When an abnormal connection occurs in the TCP connection, the script that is being executed fails to run, and may even affect the subsequent automated test execution.

Optionally, the foregoing step 101 includes:

Determining the original state that the device under test is restored to before the abnormal interruption, and determining that the configuration command that the device under test needs to execute after the last restoration to the original state is the configuration command.

At this time, by determining the original state of the device before the interruption, and the configuration command that needs to be executed after the last time, the device can be reconnected with the device, and the device can execute a set of configuration commands before the abnormal interrupt occurs, and the interrupted service is resumed. And relative to the process of reconfiguring from the very beginning of automated testing, It saves time in restoring configuration and improves the efficiency of automated testing.

Optionally, the steps of step 101 above include:

Determining, according to the configuration command that fails to be delivered, a configuration command that needs to be executed after the device under test returns to the original state.

At this time, when the connection between the system and the device is abnormally interrupted, the system fails to send the configuration command to the device. The configuration command is the configuration command that fails to be delivered. The system sends a configuration command to abort. The configuration command that fails to be sent can determine the original state that the device is restored to before the abnormal interrupt occurs. After the device is restored to the original state, the configuration command to be executed is determined to be reconnected with the device under test. The configuration commands that the device under test needs to execute.

Optionally, in the process of performing an automated test on a device under test according to a test script, the method further includes:

It can monitor the connection between the automated test system and the device under test in real time, and perform exception handling in case of abnormal interruption. Obtain the configuration command that fails to be sent in the test script. The system can accurately determine the configuration commands that need to be executed after the device under test returns to the original state to determine the configuration commands to be executed by the device after the interrupt occurs to ensure the continuity of the automated test.

When the connection with the device under test is abnormally interrupted, the device under test that has an abnormal interruption can also be obtained, and the device under test can be accurately found and the connection with the device under test can be restored.

Optionally, in the process of testing a device under test according to a test script, the method further includes:

The data is supported by the configuration command sent to the device by the test script to obtain the configuration command that fails to be delivered and the configuration command that needs to be executed after the device is restored to the original state. The automation test is guaranteed to proceed smoothly.

You can save the configuration commands sent by the test script to the storage database for easy searching and calling.

An application flow of the method of this embodiment is illustrated below.

It is assumed that the preset original state is the global configuration (Config) state of the device, which may be a state in which the system executes a configuration terminal (confgure terminal) command after connecting the device. According to a test script, a series of configurations can be performed on the device under test, including interface configuration, device loopback address loopback configuration, and intermediate system to intermediate system (ISIS) protocol configuration. When configuring the ISIS protocol, you can enter the Border Gateway Protocol (BGP) configuration mode, enter the protocol family mode of Internet Protocol Version 6, IPv6, and then execute the route re-release command. Redistribute isis-1-2) Redistribute ISIS routes.

In the method of the embodiment, during the test, whether the connection with the device under test is abnormally interrupted, and the configuration command sent to the device under test is recorded in the storage database.

Assume that the connection to the device under test is abnormally interrupted when the ISIS protocol is configured. The system cannot send a configuration command to the device under test and the configuration command is aborted. For example, the configuration command that the system obtains fails to be redistribute isis-1-2, and obtains the device under test that has an abnormal interrupt.

According to the configuration command redistribute isis-1-2 that fails to execute, the original state of the last recovery is retrieved before the abnormal interrupt is retrieved in the storage database, and the configuration commands to be executed after the original state are sequentially acquired. Optionally, the retrieved configuration command to be restored to the original state is to enter the Border Gateway Protocol (BGP) configuration mode, enter the IPv6 protocol family mode, and then execute the redistribute isis-1-2 command. Redistribute ISIS routes.

Optionally, after reconnecting with the device under test that is abnormally interrupted, the device under test is restored to the global configuration Config state, and the configuration command to be executed after the original state is restored is sent to the device under test, so that the device under test enters. In the BGP configuration mode, the IPv6 protocol family mode is entered. The redistribute isis-1-2 command is used to re-issue the ISIS route. The automated test can be performed to ensure the continuity of the automated test.

Compared with sequentially performing the foregoing series of related configurations, the method of the embodiment, after the control device returns to the global configuration state, can execute only the configuration command for configuring the ISIS protocol, saving the configuration time and improving the efficiency of the automatic test. .

In summary, the method for abnormal processing in the automatic test of the embodiment is restored to a preset original state by the control device after reconnecting with the device, and the configuration command after the original state is executed, so that the subsequent automatic test can be smoothly performed. This is done to avoid the failure of the test script to run, to ensure the continuity of the automated test, and to improve the system's fault tolerance. After the method is reconnected with the device, the test may not be re-tested from the initial configuration command, which saves the time for restoring the configuration, improves the efficiency of the automated test, and solves the problem between the automated test system and the device under test in the related art. An abnormal interruption in the TCP connection causes the running script to fail, and even affects the subsequent automated test execution.

As shown in FIG. 2, the embodiment provides an apparatus for abnormal processing in an automated test, comprising: a determining module and a control module.

The determining module is configured to determine, in the process of testing a device under test according to a test script, if the connection with the device under test is abnormally interrupted, determining that the device under test is restored to the preset original state according to the abnormal interruption. The configuration command that the device under test needs to execute after the status.

After the control module is configured to reconnect with the device under test, the device under test is controlled to return to the original state, and the configuration command to be executed is sent to the device under test.

The apparatus for abnormal processing in the automatic test of the embodiment is restored to a preset original state by the control device after being reconnected with the device, and the configuration command after the original state is executed, so that the subsequent automatic test can be smoothly performed to avoid The test script failed to run, ensuring the continuity of automated testing and improving the system's fault tolerance. Moreover, the method does not need to re-test from the initial configuration command after reconnecting with the device, which saves the time for restoring the configuration, improves the efficiency of the automated test, and solves the problem between the automated test system and the device under test in the related art. An abnormal interruption in the TCP connection causes the running script to fail, and even affects the subsequent automated test execution.

Optionally, the determining module includes: a first determining unit.

The first determining unit is configured to determine an original state that the device under test is restored to before the abnormal interrupt occurs, and determine that the device under test is restored after being restored to the original state for the last time. The configuration commands that the test device needs to execute.

Optionally, the determining module includes: an obtaining unit and a second determining unit.

After the acquisition unit is set to be abnormally interrupted with the connection between the device under test, the configuration command that fails to be delivered in the test script is obtained.

The second determining unit is configured to determine, according to the configuration command that the delivery fails, a configuration command that needs to be executed by the device under test after the device under test returns to the original state.

Optionally, the device further includes: a detection module and an acquisition module.

The detection module is configured to detect whether an abnormal interruption has occurred in connection with the device under test.

The obtaining module is configured to acquire a configuration command that fails to be delivered in the test script when an abnormality occurs in the connection with the device under test.

Optionally, the device further includes: a recording module.

The recording module is configured to record a configuration command issued to the device under test according to the test script during an automated test of a device under test according to a test script.

The device for abnormal processing in the automatic test of the embodiment is restored to a preset original state by the control device after being reconnected with the device, and the device under test executes the configuration command after the original state, so that the subsequent automatic test can be smoothly performed. It avoids the failure of the test script to run, ensures the continuity of the automated test, and improves the system's fault tolerance. In this method, after reconnecting with the device, it is not necessary to re-test from the initial configuration command, which improves the efficiency of the automated test, and solves the abnormal interruption of the TCP connection between the automated test system and the device under test in the related art. The script being executed failed to run, and even affected the issue of subsequent automated test execution.

The abnormality processing device in the automated test is a device corresponding to the abnormality processing method in the above-described automated test, wherein all the implementations in the above method embodiments are applicable to the embodiment of the device.

The abnormality processing device in the automated test of the above embodiment can be applied to an automated test system. The embodiment provides an automated test system comprising: an exception handling device in an automated test as described in the above embodiments. Wherein, the implementation embodiments of the exception handling device in the above automated test are all applicable to the embodiment of the automated test system.

The embodiment provides a computer readable storage medium storing computer executable instructions. The computer executable instructions are arranged to perform the method of any of the above embodiments.

This embodiment provides a hardware structure diagram of an automated test system. Referring to Figure 3, the automated test system includes:

At least one processor 30, which is exemplified by a processor 30 in FIG. 3; a memory 31; and a communication interface 32 and a bus 33. The processor 30, the memory 31 and the communication interface 32 can complete communication with each other via the bus 33. Communication interface 32 can transmit data. The processor 30 can call the logic instructions in the memory 31 to perform the methods in the above embodiments.

In addition, the logic instructions in the memory 31 described above may be implemented in the form of a software functional unit and sold or used as a stand-alone product, and may be stored in a computer readable storage medium.

The memory 31 is a computer readable storage medium and can be used to store software programs, computer executable programs, and program instructions or modules corresponding to the methods in the above embodiments. The processor 30 executes the functional application and the data processing by executing a software program, an instruction or a module stored in the memory 31, that is, the method in the above embodiment is implemented.

The memory 31 may include a storage program area and an storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to usage of the terminal device, and the like. Further, the memory 31 may include a high speed random access memory, and may also include a nonvolatile memory.

The above technical solution may be embodied in the form of a software product stored in a storage medium, including one or more instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to execute All or part of the steps of the method described in the above embodiments. The foregoing storage medium may be a non-transitory storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. A medium that can store program code, or a transitory storage medium.

Industrial applicability

The method, device and system for abnormal processing in automated testing improve the efficiency of automated testing, ensure the continuity of automated testing, and improve the fault tolerance of the system.

Claims

An exception handling method in automated testing, including:

In the process of performing an automated test on a device under test according to a test script, if the connection with the device under test is abnormally interrupted, determining, according to the abnormal interruption, that the device under test returns to a preset original state. The configuration command that the device under test needs to execute;

After the device is reconnected, the device under test is restored to the original state, and the configuration command to be executed is sent to the device under test.
The method of claim 1, wherein the determining, according to the abnormal interruption condition, that the device under test needs to be executed after the device under test returns to a preset original state comprises:

Determining, by the device under test, the original state that was restored to the last time before the abnormal interrupt occurs, and determining that the configuration command that the device under test needs to execute after the device is restored to the original state is the configuration command. .
The method of claim 1, wherein the determining, according to the abnormality interruption, that the device under test needs to be executed after the device under test returns to a preset original state comprises:

After the connection with the device under test is abnormally interrupted, obtaining a configuration command that fails to be delivered in the test script;

Determining, according to the configuration command that fails to be delivered, a configuration command that the device under test needs to perform after the device under test returns to the original state.
The method according to claim 3, in the process of automatically testing a device under test according to a test script, the method comprising:

Detecting whether a connection to the device under test is abnormally interrupted;

Obtaining the test script in the middle of an abnormal interruption in the connection with the device under test A failed configuration command.
The method according to claim 1, wherein in the process of testing a device under test according to a test script, the method comprises:

A configuration command sent to the device under test according to the test script is recorded.
An anomaly processing device for automated testing, comprising:

Determining a module, in the process of performing an automatic test on a device under test according to a test script, if an abnormal interruption occurs in a connection with the device under test, determining that the device under test is restored according to the abnormal interruption a configuration command that the device under test needs to execute after the preset original state;

After the control module is configured to reconnect with the device under test, the device is controlled to be restored to the original state, and the configuration command to be executed is sent to the device under test.
The apparatus of claim 6 wherein said determining module comprises:

a first determining unit, configured to determine an original state that is restored to the device under test before the abnormal interrupt occurs, and determine that the device under test needs to be executed after the device is restored to the original state for the last time. Configuration commands.
The apparatus of claim 6 wherein said determining module comprises:

Obtaining a configuration command that is sent in the test script after the connection to the device under test is abnormally interrupted;

The first determining unit is configured to determine, according to the configuration command that the delivery fails, a configuration command that the device under test needs to perform after the device under test returns to the original state.
The apparatus of claim 6 further comprising:

Recording module, set to automate testing of a device under test according to a test script The configuration command sent to the device under test according to the test script is recorded.
An automated test system comprising: an abnormality processing device in an automated test according to any one of claims 6-9.
A computer readable storage medium storing computer executable instructions arranged to perform the method of any of claims 1-5.