CN116303041A - Distributed testing method and system - Google Patents

Abstract

The present invention provides a distributed test method and system. The method includes: building a cloud test client on the test device, and configuring and initializing the required environment; creating a test strategy at the cloud test system; The terminal sends the test strategy to the corresponding test equipment and starts the test task; after the test task ends, it summarizes the test results of each test equipment and sends a summary report. On the one hand, the present invention can manage and utilize scattered terminal devices to the greatest extent; on the other hand, it can distribute test cases to terminal devices of different test devices for testing according to strategies, and after the test tasks of each test device are completed, test Summarize the results and send a summary report, thereby significantly shortening the test time and improving test efficiency; saving product testing and development time.

Description

Distributed testing method and system

technical field

The invention relates to the field of computer technology, in particular to a distributed testing method and system.

Background technique

XTS (Executable Test Suite, executable test suite) test is for CTS (Compatibility TestSuite, compatibility test suite) / VTS (Vendor Test Suite, Vendor interface test suite) / STS (SecurityTest Suite, security test suite) and other test services Collectively, when the test cases of each test suite pass the test, the latest version can be applied for release.

However, when there is only a single terminal device on the test device, the XTS test will consume more time when running tests on modules with many use cases. Testers must wait for the test result file to be generated before proceeding to the next step. This process reduces improved test efficiency. To improve test efficiency, at present, terminal equipment can only be collected to the same test equipment for testing using a distributed test method, but it is very inconvenient when terminal equipment is in short supply and scattered. Therefore, there is an urgent need for a distributed test solution or technology that can utilize terminal devices scattered in different test devices online.

Contents of the invention

The distributed testing method and system provided by the present invention, on the one hand, can manage and utilize scattered terminal equipment to the greatest extent; After the test task of a test equipment is completed, the test results are summarized and sent to a summary report, thereby significantly shortening the test time, improving test efficiency, and saving product testing and development time.

In a first aspect, the present invention provides a distributed testing method, the method comprising:

Built-in cloud test client on the test equipment, and configure and initialize the required environment;

Create a test strategy on the cloud test system side;

The test strategy is sent to the corresponding test equipment by the cloud test system end and the test task is started;

After the test task is completed, the test results of each test equipment are summarized and a summary report is sent.

Optionally, the built-in cloud test client on the test device, and configuring and initializing the required environment includes:

Built-in cloud test client on the test device, and configure the required environment;

Execute the cloud measurement client startup script and select the configuration file;

After the cloud test client is started, the cloud test system side detects the test equipment and allocates and manages the test equipment.

Optionally, said creating a test strategy at the cloud testing system end includes:

The cloud test system side creates a corresponding set of test strategies according to the terminal device information and/or test information;

The test strategy set is sent to the corresponding test equipment as a strategy parameter by the cloud test system end;

Wherein, the set of test strategies includes at least one test strategy, and each test strategy corresponds to the terminal device information and/or test information.

The cloud test system end creates a test strategy set according to terminal equipment information and/or test information including:

When at least two terminal devices are connected to the test device, at least two terminal devices are used for distributed testing at the same time;

And/or, when the cloud test system is connected to at least two test devices, at least two test devices are used to perform distributed testing at the same time.

Optionally, the test information includes: the full path of the test package, and/or the name of the module to be tested, and/or the scope of the test cases of the module to be tested, and/or the full path containing the historical test results of the module to be tested. path, and/or network information of the laboratory where the test equipment is located.

Optionally, the sending the test strategy to the corresponding test equipment by the cloud test system and starting the test task includes:

The test equipment executes the cloud test adaptation script, and analyzes the corresponding policy parameters sent by the cloud test system;

The cloud test adaptation script parses the use case information of the specified module; and splits the use cases within the range into a plan file according to the scope of the module test cases to be tested in the policy parameters, and saves them in the directory corresponding to the test package path;

Execute the described planning document using the toolkit and conduct tests for the stated range of use cases;

After the toolkit test is completed, the cloud test client uploads the test results to the cloud test server.

Optionally, the policy parameters include: a path of a test package on the test device, and/or a path of a historical result file containing a specified test module.

In a second aspect, the present invention provides a distributed testing system, the system comprising:

The configuration unit is used to build the cloud test client on the test device, and configure and initialize the required environment;

A strategy creation unit is used to create a test strategy on the cloud test system side;

A test start unit, configured to send the test strategy to the corresponding test equipment through the cloud test system and start the test task;

The summary unit is used to summarize the test results of each test equipment after the test task is completed, and send a summary report.

Optionally, the configuration unit includes:

The environment configuration subunit is used to build the cloud test client on the test device and configure the required environment;

Select a subunit to execute the cloud measurement client startup script and select a configuration file;

The allocation management subunit is used to detect the test equipment through the cloud test system after starting the cloud test client and perform allocation management on the test equipment.

Optionally, the policy creation unit includes:

Create a subunit, which is used to create a corresponding test strategy set according to terminal device information and/or test information through the cloud test system; wherein, the test strategy set includes at least one test strategy, and each test strategy is related to the terminal device information, and/or test information;

The sending subunit is configured to send the set of test strategies as strategy parameters to the corresponding test equipment through the cloud test system.

Optionally, the test initiation unit includes:

The policy analysis subunit is used to execute the cloud test adaptation script through the test equipment, and analyze the corresponding policy parameters sent by the cloud test system;

The use case analysis subunit is used to analyze the use case information of the specified module through the cloud test adaptation script; and split the use cases within the range into a plan file according to the test case range of the module to be tested in the policy parameters, and save it in the test Under the directory corresponding to the package path;

a test subunit for executing said plan file using a toolkit and testing a range of use cases;

The upload subunit is used to upload the test results to the cloud test server through the cloud test client after the toolkit test is completed.

The distributed test method and system provided by the present invention, on the one hand, can manage and utilize scattered terminal equipment to the greatest extent, flexibly utilize terminal equipment of different test equipment, fully utilize all terminal equipment resources, and greatly save test costs. On the other hand, test cases can be distributed to terminal devices of different test devices for testing according to strategies, and after the test task of each test device is completed, the test results can be summarized and a summary report can be sent, especially for a large number of tests The test module of the use case, the present invention can obviously shorten test time, improve test efficiency; greatly save product test and development time.

Description of drawings

Fig. 1 is the flowchart of the distributed test method of an embodiment of the present invention;

Fig. 2 is the flow chart of another embodiment of the distributed test method of the present invention;

Fig. 3 is a schematic structural diagram of a distributed test system according to an embodiment of the present invention;

FIG. 4 is a schematic structural diagram of a distributed testing system according to another embodiment of the present invention.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is only some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

An embodiment of the present invention provides a distributed testing method, as shown in Figure 1, the method includes:

S11. Build a cloud test client on the test device, and configure and initialize the required environment;

S12. Create a test strategy on the cloud test system side;

S13. The cloud test system sends the test strategy to the corresponding test equipment and starts the test task;

S14. After the test task is completed, the test results of each test device are summarized, and a summary report is sent.

The distributed test method provided by the embodiment of the present invention can manage and utilize scattered terminal equipment to the greatest extent, flexibly utilize terminal equipment of different test equipment, fully utilize all terminal equipment resources, and greatly save test costs. On the other hand, test cases can be distributed to terminal devices of different test devices for testing according to strategies, and after the test task of each test device is completed, the test results can be summarized and a summary report can be sent, especially for a large number of tests The test module of the use case, the present invention can obviously shorten the test time, improve the efficiency of the test, and greatly save the product test and development time.

For example, when the module to be tested (such as XTS test) needs to use a large number of test cases, and there are multiple available test equipment (may not be within a geographical space range), you can use the built-in cloud test client on the test equipment end, and configure and initialize the environment required by the module to be tested; then the cloud test system end allocates and manages multiple test devices according to test strategies; for example, to send a test strategy to the first test device, the first test device The strategy starts the test task, and completes the test case in the range of test 1-S ₁ , sends the test strategy to the first test device, and the first test device starts the test task according to the test strategy, and completes the test in the range of S ₁₊₁ -S ₂ Test cases within...until all the test cases of the module to be tested are completed. Then, the first test equipment, the second test equipment...send the test results to the cloud side system, and the cloud side system summarizes all the test results and sends a complete test report.

In addition, when the test device is connected to multiple terminal devices of the same type, it can simultaneously control multiple terminal devices of the _same type for testing. Complete the test cases in the range of N ₁₊₁ -N ₂ ....

As shown in Figure 2, optionally, the built-in cloud test client on the test device, and configuring and initializing the required environment includes:

S111, build a cloud test client on the test device, and configure the required environment;

S112. Execute the startup script of the cloud test client, and select the configuration file;

S113. After starting the cloud test client, the cloud test system side detects the test equipment and allocates and manages the test equipment.

Specifically, the present invention builds a cloud test client in advance on the device to be tested, and configures the required environment of the relevant modules, such as jdk, python, flashing tool environment, etc. Execute the cloud test client startup script, and select the XTS special configuration file (cts-and-vts.xml) for the configuration file. After starting the cloud test client, the cloud test system can detect the terminal devices on the device to be tested and manage and allocate these devices. In addition, the XTS test environment needs to be prepared on the device to be tested.

Taking the CTS test module as an example, if you want to test Android 13 devices, you need to place the CTS test package corresponding to this version on the device to be tested. If the user wants to test a specific module, it is also necessary to prepare an xml file (test_result.xml) containing the historical test results of the module on the device to be tested.

Optionally, after step S113, it also includes:

S114. Determine whether the environment configuration is successful; if yes, execute step S12; if not, execute step S111.

Optionally, said creating a test strategy at the cloud testing system end includes:

S121. The cloud test system side creates a corresponding test strategy set according to the terminal device information and/or test information;

S122. The cloud test system side sends the test policy set as a policy parameter to the corresponding test equipment;

Wherein, the set of test strategies includes at least one test strategy, and each test strategy corresponds to the terminal device information and/or test information.

The cloud test system end creates a test strategy according to terminal equipment information and/or test information including:

When at least two terminal devices are connected to the test device, at least two terminal devices are used for distributed testing at the same time;

And/or, when the cloud test system is connected to at least two test devices, at least two test devices are used to perform distributed testing at the same time.

Optionally, the test information includes: the full path of the test package, and/or the name of the module to be tested, and/or the scope of the test cases of the module to be tested, and/or the full path containing the historical test results of the module to be tested. path, and/or network information of the laboratory where the test equipment is located.

Specifically, after step S11, the cloud test system end can manage all terminal devices on the device to be tested, and can create a policy set through the cloud test system end. The policy set is composed of a series of policies (at least two), and the object of each policy is a terminal device on a device to be tested. The significance of creating a strategy lies in that the method can automatically or manually formulate differentiated configurations for the XTS testing process of different devices under test, and deliver these configurations to the devices under test in the form of policy parameters.

In each strategy, by inputting part of the information of the XTS test package on the device to be tested, such as the full path of the XTS test package on the test device, the name of the module to be tested, the scope of the test case of the module to be tested, and the history of the module The full path where the test result (test_result.xml) is located, and the network information (such as WIFI information, Internet information, etc.) of the laboratory where the test equipment is located.

For example: when at least two terminal devices of the same type are connected to the test device, the test strategy created by the method can be automatically or manually selected to use at least two terminal devices to perform distributed testing on the module to be tested at the same time; for example, testers can pass Select the distributed test option (DistributedTest) in the strategy to perform distributed testing;

When the module to be tested is a media-related module, the test strategy created by the method can automatically transfer and input the full path of the media file on the device to be tested, or manually fill in the media file in the strategy on the device to be tested full path.

When it is necessary to retest the untested items and test failed items in the test process after the first test is completed, the test strategy created by the method can enable retest items in the test strategy and fill in the number of retests.

When the final test log and test results need to be uploaded after the entire test is completed, the test strategy created by the method can automatically input or manually fill in the relevant information of the target FTP server to be uploaded in the test strategy, including server address, transmission port ID, user name, user password, save path in the server.

Optionally, the sending the test strategy to the corresponding test equipment by the cloud test system and starting the test task includes:

S131. The test device executes the cloud test adaptation script, and analyzes the corresponding policy parameters sent by the cloud test system;

For example, the method executes the XTS cloud test adaptation script (start.py) to analyze the corresponding policy parameters issued by the cloud test system, including but not limited to the XTS test package path on the test device, the path containing the specified test module Historical result XML file (test_result.xml) path, etc.

S132. The cloud test adaptation script parses the use case information of the specified module; and splits the use cases within the scope into a plan file according to the scope of the module test cases to be tested in the policy parameters, and saves them in the directory corresponding to the test package path;

For example, the method of the present invention can analyze the use case information of the specified module from test_result.xml through the XTS cloud test adaptation script, and split the use cases in this range according to the use case range parsed in step S131 to generate a plan XML separately The file is saved in the android-xts/subplans/ directory of the XTS test package path.

S133. Use the toolkit to execute the plan file, and test the range of use cases;

For example, the method of the present invention can use the XTS toolkit to execute the plan XML file generated in step S132, and at this time only the use cases in the specified range are tested.

S134. After the toolkit test is completed, the cloud test client uploads the test result to the cloud test server.

Optionally, the parameters include: a path of a test package on the test device, and/or a path of a historical result file containing a specified test module.

S14. After the test task is completed, the test results of each test device are summarized, and a summary report is sent.

Specifically, in the method of the present invention, after all the test strategies in the test task have been executed in the corresponding test equipment, the cloud test client can also upload the test results to the cloud test server, and the cloud test server performs all test strategies on the results Analyze and summarize, and send the analyzed data to relevant personnel or equipment in the form of reports. Wherein, the form of the report may be a table or a document format.

The embodiment of the present invention also provides a distributed test system, as shown in Figure 3, the system includes:

The configuration unit 11 is used to build the cloud test client on the test equipment, and configure and initialize the required environment;

A strategy creation unit 12, used to create a test strategy at the cloud test system end;

The test start unit 13 is used to send the test strategy to the corresponding test equipment and start the test task through the cloud test system end;

The summarizing unit 14 is used for summarizing the test results of each test equipment and sending a summary report after the test task is finished.

The distributed test system provided by the embodiment of the present invention, on the one hand, the system can manage and utilize scattered terminal devices to a great extent, flexibly utilize terminal devices of different test devices, fully utilize all terminal device resources, and greatly save test costs. On the other hand, it is possible to utilize the strategy creation unit 12 to distribute the use cases of the test to the terminal equipment of different test equipment for testing according to the test strategy, and after the test task of each test equipment is finished, the test results are summarized by the summary unit 14, and Sending summary reports, especially for test modules with a large number of test cases, the present invention can significantly shorten test time and improve test efficiency; greatly save product test and development time.

As shown in Figure 4, optionally, the configuration unit 11 includes:

The environment configuration subunit 111 is used to build the cloud test client on the test equipment and configure the required environment;

The selection subunit 112 is used to execute the cloud measurement client startup script and select the configuration file;

The allocation management subunit 113 is configured to detect the test equipment through the cloud test system after starting the cloud test client and perform allocation management on the test equipment.

Optionally, the configuration unit 11 also includes:

The judging subunit 114 is configured to judge whether the environment configuration is successful.

Optionally, the policy creation unit 12 includes:

Creating a subunit 121 for creating a corresponding test strategy set according to terminal device information and/or test information through the cloud test system; wherein, the test strategy set includes at least one test strategy, and each test strategy is related to the terminal Corresponding to equipment information and/or test information;

The sending subunit 122 is configured to send the set of test strategies as strategy parameters to the corresponding test equipment through the cloud test system.

Optionally, the test initiation unit 13 includes:

The policy analysis subunit 131 is used to execute the cloud test adaptation script through the test equipment, and analyze the corresponding policy parameters sent by the cloud test system end;

The use case analysis subunit 132 is used to analyze the use case information of the specified module through the cloud test adaptation script; and according to the scope of the module test case to be tested in the policy parameter, the use case in the scope is split into a plan file, and stored in Under the corresponding directory of the test package path;

A test subunit 133, configured to use a toolkit to execute the plan file and test the range of use cases;

The upload subunit 134 is configured to upload the test results to the cloud test server through the cloud test client after the toolkit test is completed.

The test equipment can be a general-purpose or special-purpose computer. Examples: personal computers, server computers, handheld or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, including A distributed computing environment for any of the above systems or devices, etc.

The mobile device may be a smart phone, or a digital camera, or a digital video camera, or a tablet computer, or MP4, or a nuclear magnetic resonance imaging device (MRI), or an electronic computer tomography imaging device (CT).

Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium. The program During execution, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM) and the like.

The above is only a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present invention. All should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (11)

1. A distributed testing method, characterized in that the method comprises: Built-in cloud test client on the test equipment, and configure and initialize the required environment; Create a test strategy on the cloud test system side; The test strategy is sent to the corresponding test equipment by the cloud test system end and the test task is started; After the test task is completed, the test results of each test equipment are summarized and a summary report is sent. 2. The method according to claim 1, wherein the built-in cloud test client on the test device, and configuring and initializing the required environment comprises: Built-in cloud test client on the test device, and configure the required environment; Execute the cloud measurement client startup script and select the configuration file; After starting the cloud test client, the cloud test system side detects the test equipment and allocates and manages the test equipment. 3. The method according to claim 2, wherein said creating a test strategy at the cloud testing system end comprises: The cloud test system side creates a corresponding set of test strategies according to the terminal device information and/or test information; The test strategy set is sent to the corresponding test equipment as a strategy parameter by the cloud test system end; Wherein, the set of test strategies includes at least one test strategy, and each test strategy corresponds to the terminal device information and/or test information. 4. The method according to claim 3, wherein said cloud testing system end creates a test strategy set according to terminal device information and/or test information including: When at least two terminal devices are connected to the test device, at least two terminal devices are used for distributed testing at the same time; And/or, when the cloud test system is connected to at least two test devices, at least two test devices are used to perform distributed testing at the same time. 5. The method according to claim 3, wherein the test information includes: the full path of the test package, and/or the name of the module to be tested, and/or the scope of the test case of the module to be tested, and/or It includes the full path where the historical test results of the module to be tested are located, and/or the network information of the laboratory where the test equipment is located. 6. The method according to any one of claims 1-5, wherein the sending the test strategy to the corresponding test equipment by the cloud test system and starting the test task comprises: The test equipment executes the cloud test adaptation script, and analyzes the corresponding policy parameters sent by the cloud test system; The cloud test adaptation script parses the use case information of the specified module; and splits the use cases within the range into a plan file according to the scope of the module test cases to be tested in the policy parameters, and saves them in the directory corresponding to the test package path; Execute the described planning document using the toolkit and conduct tests for the stated range of use cases; After the toolkit test is completed, the cloud test client uploads the test results to the cloud test server. 7. The method according to claim 6, wherein the policy parameters include: a test package path on the test device, and/or a path of a historical result file containing a specified test module. 8. A distributed testing system, characterized in that the system comprises: The configuration unit is used to build the cloud test client on the test device, and configure and initialize the required environment; A strategy creation unit is used to create a test strategy on the cloud test system side; A test start unit, configured to send the test strategy to the corresponding test equipment through the cloud test system and start the test task; The summary unit is used for summarizing the test results of each test equipment and sending a summary report after the test task ends. 9. The system according to claim 8, wherein the configuration unit comprises: The environment configuration subunit is used to build the cloud test client on the test device and configure the required environment; Select a subunit to execute the cloud measurement client startup script and select a configuration file; The allocation management subunit is used to detect the test equipment through the cloud test system and perform allocation management on the test equipment after the cloud test client is started. 10. The system according to claim 8 or 9, wherein the policy creation unit comprises: Create a subunit, which is used to create a corresponding test strategy set according to terminal device information and/or test information through the cloud test system; wherein, the test strategy set includes at least one test strategy, and each test strategy is related to the terminal device information, and/or test information; The sending subunit is configured to send the set of test strategies as strategy parameters to the corresponding test equipment through the cloud test system. 11. The system according to any one of claims 8-10, wherein the test initiation unit comprises: The policy analysis subunit is used to execute the cloud test adaptation script through the test equipment, and analyze the corresponding policy parameters sent by the cloud test system; The use case analysis subunit is used to analyze the use case information of the specified module through the cloud test adaptation script; and split the use cases within the range into a plan file according to the test case range of the module to be tested in the policy parameters, and save it in the test Under the directory corresponding to the package path; a test subunit for executing said plan file using a toolkit and testing a range of use cases; The upload subunit is used to upload the test results to the cloud test server through the cloud test client after the toolkit test is completed.

Images