CN111459798A

CN111459798A - Test method, test device, test medium, and electronic apparatus

Info

Publication number: CN111459798A
Application number: CN202010138639.1A
Authority: CN
Inventors: 郑如刚
Original assignee: OneConnect Financial Technology Co Ltd Shanghai
Current assignee: OneConnect Smart Technology Co Ltd; OneConnect Financial Technology Co Ltd Shanghai
Priority date: 2020-03-03
Filing date: 2020-03-03
Publication date: 2020-07-28

Abstract

The application relates to a test method and a related device, belonging to the technical field of computers, wherein the method comprises the following steps: acquiring configuration information of test cases in a test case set for a target test task; extracting test case characteristic information from the configuration information of the test case; inputting the test case characteristic information into a pre-trained grouping model to obtain a plurality of test case groups; concurrently executing test cases in the test case group capable of being in a parallel type, accessing a thread lock to lock the test cases in operation during concurrent execution, and releasing the thread lock when the operation is finished; and when the execution of the test cases in the test case group capable of being paralleled is finished, obtaining a concurrent test result. The application can improve the reliability and stability of the test.

Description

Test method, test device, test medium, and electronic apparatus

Technical Field

The present application relates to the field of computer technologies, and in particular, to a test method, an apparatus, a medium, and an electronic device.

Background

At present, some false alarm rates can occur in the execution process of the test script, the false alarm rates are generally divided into two types, one type is an ineffectiveness factor, namely, the conditions that the test script cannot normally run under the conditions of network breaking, power failure, system restarting or no startup are called environment false alarm, the probability of the occurrence of the errors is very small, the second type is logic false alarm, namely, the verification result is inconsistent with the expected result due to the logic error occurring in the verification process, the errors are often caused by not considering perfection when executing a test case, the multithread control is unreliable, the logic error is easy to occur, when the test script is controlled, the next logic test is triggered after the previous test is not completed, so that the false alarm can bring serious stability influence on the test, therefore, the problems of low test reliability and test stability exist in the test process at present.

Disclosure of Invention

The purpose of this application is to provide a test scheme, and then promote test reliability and stability to a certain extent at least.

According to an aspect of the present application, there is provided a test method including:

acquiring configuration information of test cases in a test case set for a target test task;

extracting test case characteristic information from the configuration information of the test cases, wherein the test case characteristic information is the key attribute characteristic of the execution strategy of each test case;

inputting the test case characteristic information into a pre-trained grouping model to obtain a plurality of test case groups;

concurrently executing test cases in the test case group capable of being in a parallel type, accessing a thread lock to lock the test cases in operation during concurrent execution, and releasing the thread lock when the operation is finished;

and when the execution of the test cases in the test case group capable of being in the parallel type is finished, obtaining a concurrent test result.

In an exemplary embodiment of the present application, the training method of the grouping model includes:

collecting a test case characteristic information sample set, wherein each sample in the sample set is calibrated with a grouping label of a corresponding test case in advance;

inputting the samples in the sample set into a grouping model, and outputting a predicted grouping label of the samples;

when the predicted grouping label is inconsistent with a grouping label calibrated in advance for the sample, adjusting the coefficient of a grouping model until the predicted grouping label is consistent with the grouping label calibrated in advance for the sample;

and when the grouping model aims at the sample set, the predicted grouping labels of all the samples are consistent with the grouping labels calibrated in advance for the samples, and the training is finished.

In an exemplary embodiment of the present application, the concurrently executing test cases in a test case group of a parallelizable type, and accessing a thread lock to lock the test cases in execution while concurrently executing, and releasing the thread lock when execution is finished includes:

starting concurrent execution of test cases in the test case group capable of being in a parallel type, and accessing a first thread lock to lock a first test case for starting operation;

when the first test case started to run finishes running, releasing the first thread lock, and accessing a second thread lock to lock a second test case started to run;

and when the second test case started to run finishes running, releasing the second thread lock, and sequentially and alternately accessing and releasing the subsequent thread locks to execute the subsequent test cases started to run until the test cases in the test case group capable of being parallel are tested.

In an exemplary embodiment of the present application, the extracting test case feature information from the configuration information of the test case includes:

and extracting test case attribute information corresponding to the preset attribute keywords from the configuration information of the test cases based on the preset attribute keywords to obtain the test case characteristic information.

In an exemplary embodiment of the present application, the extracting, based on a preset attribute keyword, test case attribute information corresponding to the preset attribute keyword from configuration information of the test case to obtain the test case characteristic information includes:

acquiring preset attribute keywords corresponding to the case type of each test case, wherein the preset attribute keywords comprise general attribute keywords and special attribute keywords, the general attribute keywords correspond to all the test cases, and the special attribute keywords correspond to each case type;

and extracting test case attribute information from the configuration information of each test case according to the special attribute key words and the general attribute key words corresponding to each test case to obtain the test case characteristic information.

In an exemplary embodiment of the present application, the plurality of test case groups further include a test case group whose test cases need to be serialized, and the method further includes:

serially executing the test cases in the test case group to be serially executed until all the test cases are executed;

when the execution of the test cases in the test case group needing to be serialized is finished, obtaining a serial test result;

and taking the concurrent test result and the serial test result as a total test result.

In an exemplary embodiment of the present application, the test case groups to be serialized are arranged in a serial order to form a group, and the serially executing the test cases in the test case groups to be serialized includes:

and executing the test cases in the test case group to be serialized in series according to the serial sequence of the test cases in the test case group to be serialized.

According to an aspect of the present application, there is provided a test apparatus, comprising:

the acquisition module is used for acquiring the configuration information of the test cases in the test case set for the target test task;

the extraction module is used for extracting test case characteristic information from the configuration information of the test cases, wherein the test case characteristic information is the key attribute characteristic of the execution strategy of each test case;

the grouping module is used for inputting the test case characteristic information into a grouping model trained in advance to obtain a plurality of test case groups;

the concurrent module is used for concurrently executing the test cases in the test case group which can be in a parallel type, accessing the thread lock to lock the test cases which are in operation when concurrently executing, and releasing the thread lock when the operation is finished;

and the end concurrency module is used for obtaining a concurrency test result when the execution of the test cases in the test case group capable of being in the parallel type is ended.

According to an aspect of the application, there is provided a computer readable storage medium having stored thereon program instructions, characterized in that the program instructions, when executed by a processor, implement the method of any of the above.

According to an aspect of the present application, there is provided an electronic device, comprising:

a processor; and

a memory for storing program instructions for the processor; wherein the processor is configured to perform any of the methods described above via execution of the program instructions.

The application provides a test method and a related device, firstly, obtaining configuration information of test cases in a test case set for a target test task; the relevant information describing the test case configured for the target test task can be acquired. Then, extracting test case characteristic information from the configuration information of the test cases, wherein the test case characteristic information is the key attribute characteristic of the execution strategy of each test case; by extracting the key attribute characteristics of the execution strategy of each test case, the potential relation among the cases and the key characteristics of the cases based on the case execution strategy can be obtained. Further, inputting the characteristic information of the test cases into a pre-trained grouping model to obtain a plurality of test case groups; the test case group with the test cases capable of being parallel can be analyzed based on the grouping model according to the test case characteristic information, and then the high-efficiency concurrent execution of the test cases can be carried out. Then, concurrently executing the test cases in the test case group which can be in parallel, accessing a thread lock to lock the test cases in operation during concurrent execution, and releasing the thread lock when the operation of the test cases in operation is finished; therefore, the thread lock is accessed to lock the thread when each test case is executed in sequence, the thread lock is released when the test case is finished, and the next thread lock is accessed until the test is finished, so that the high concurrency process is ensured, and all tests are performed orderly and reliably. And finally, when the execution of the test cases in the test case group capable of being parallel is finished, a concurrent test result is obtained, and the test reliability and stability are effectively improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the application, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.

Fig. 1 schematically shows a flow chart of a test method.

Fig. 2 schematically shows an application scenario example of a testing method.

Fig. 3 schematically shows a flow chart of a method of concurrent execution.

Fig. 4 schematically shows a block diagram of a test apparatus.

Fig. 5 schematically shows an example block diagram of an electronic device for implementing the above-described testing method.

Fig. 6 schematically illustrates a computer-readable storage medium for implementing the above-described testing method.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the application. One skilled in the relevant art will recognize, however, that the subject matter of the present application can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present application.

Furthermore, the drawings are merely schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

In the embodiment of the present application, a test method is first provided, and the test method may be run on a server, or may also be run on a server cluster or a cloud server, and of course, a person skilled in the art may also run the method of the present application on other platforms according to a requirement, which is not particularly limited in this exemplary embodiment. Referring to fig. 1, the test method may include the steps of:

step S110, acquiring configuration information of test cases in a test case set for a target test task;

step S120, extracting test case characteristic information from the configuration information of the test cases, wherein the test case characteristic information is the key attribute characteristic of the execution strategy of each test case;

step S130, inputting the characteristic information of the test cases into a pre-trained grouping model to obtain a plurality of test case groups;

step S140, concurrently executing the test cases in the test case group capable of being in parallel type, accessing a thread lock to lock the test cases in concurrent execution, and releasing the thread lock when the execution is finished;

and S150, obtaining a concurrent test result when the execution of the test cases in the test case group capable of being in the parallel type is finished.

Firstly, acquiring configuration information of test cases in a test case set for a target test task; the relevant information describing the test case configured for the target test task can be acquired. Then, extracting test case characteristic information from the configuration information of the test cases, wherein the test case characteristic information is the key attribute characteristic of the execution strategy of each test case; by extracting the key attribute characteristics of the execution strategy of each test case, the potential relation among the cases and the key characteristics of the cases based on the case execution strategy can be obtained. Further, inputting the characteristic information of the test cases into a pre-trained grouping model to obtain a plurality of test case groups; the test case group with the test cases capable of being parallel can be analyzed based on the grouping model according to the test case characteristic information, and then the high-efficiency concurrent execution of the test cases can be carried out. Then, concurrently executing the test cases in the test case group which can be in parallel, accessing a thread lock to lock the test cases in operation during concurrent execution, and releasing the thread lock when the operation of the test cases in operation is finished; therefore, the thread lock is accessed to lock the thread when each test case is executed in sequence, the thread lock is released when the test case is finished, and the next thread lock is accessed until the test is finished, so that the high concurrency process is ensured, and all tests are performed orderly and reliably. And finally, when the execution of the test cases in the test case group capable of being parallel is finished, a concurrent test result is obtained, and the test reliability and stability are effectively improved.

Hereinafter, each step in the above-described concurrent testing method in the present exemplary embodiment will be explained and explained in detail with reference to the drawings.

In step S110, configuration information of test cases in the test case set for the target test task is acquired.

In the embodiment of the present example, referring to fig. 2, the server 201 uses the configuration information of the test case in the test case set of the target test task from the user terminal 202. In this way, in the subsequent steps, the server 201 can make a test slave strategy and perform a test according to the configuration information of the test case. It is understood that, among other things, the server 201 may be any device with processing capability, such as a computer, a microprocessor, etc., and is not particularly limited herein; the user terminal 202 may be any terminal having an information entry function.

A Test Case (Test Case) may be a set of Test inputs, execution conditions, and expected results tailored for a particular target to Test a certain program path or verify whether a certain specific requirement is met.

The configuration information of the test case is case-related information configured for each test case, and may include elements of each test case: title of test case, test thought, preset conditions, test steps, expected output, name of software or project, version (internal version number) of software or project, dependency relationship between the test case and other test cases, and the like.

Generally, a test task includes a test case set of multiple test cases, for example, a set of 1000 test cases, where the test cases may partially have a logical call relationship (for example, when a method a runs, a variable has a value that is a dynamically generated value of a method B.

The test case set corresponding to the task identifier of the target test task and the configuration information of each test case can be obtained through the task identifier of the target test task. The configuration information of the test case of the target test task can be configured in advance by a developer.

Therefore, the relevant information describing the test case configured for the target test task can be acquired, and further, the test strategy can be analyzed and formulated based on the configuration information in the subsequent steps.

In step S120, test case feature information is extracted from the configuration information of the test case, where the test case feature information is a key attribute feature of an execution policy of each test case.

In the embodiment of the present example, the key attribute feature of the execution policy of each test case, that is, the test case feature information, may be key information reflecting the execution policy in the test case configuration information, and these key information may be information corresponding to a plurality of key elements. For example, the key elements may be test ideas, preset conditions, test steps, expected outputs, dependency relationships between the test case and other test cases, and the like, the key attribute features may be attribute information corresponding to the key attributes, the information corresponding to the key elements may be directly used as case feature information by directly presetting the key elements, and the key attribute features of the execution strategy of each test case may be completely included.

The content in each test case element can be configured in the configuration information in advance according to the attribute and attribute information contrast relationship, and further the key attribute characteristics in each test case element can be determined through the preset attribute key words.

By extracting the key attribute characteristics of the execution strategy of each test case, the potential relation among the cases reflecting the case execution strategy and the key characteristics of the cases can be obtained.

In one embodiment, the extracting test case feature information from the configuration information of the test case includes:

The preset attribute key may be a test element name (for example, a test idea, a preset condition, a test step, an expected output, a dependency relationship, and the like) in a plurality of test case configuration information of a predetermined setting. The content attribute in the content of each test element may also be, for example, for a test concept element, the content may be: and (3) after the abnormal condition 1 is full of storage space, the abnormal condition 2 is used for running a program occupying space, a test purpose is realized, whether the terminal is abnormal under the condition of storage overflow is tested, and the content attribute keywords in the content of the test thought elements are the abnormal condition 1, the abnormal condition 2, the test purpose and the like. Furthermore, test case attribute information corresponding to the preset attribute keywords can be extracted from the configuration information of the test cases according to the preset attribute keywords, so that test case characteristic information can be obtained.

In one embodiment, the extracting, based on a preset attribute keyword, test case attribute information corresponding to the preset attribute keyword from configuration information of the test case to obtain the test case characteristic information includes:

The case type of each test case, for example: functional testing, usability testing, installability testing, exception testing, stress testing, GUI testing, compatibility testing, safety testing, reliability testing, and the like.

The test cases have different case types, the configuration information such as the test thought is adaptive to the case types, and the characteristic information of each test case can be accurately acquired by acquiring the preset attribute keywords corresponding to the case type of each test case.

The preset attribute keywords include general attribute keywords and special attribute keywords, the general attribute keywords correspond to all test cases, that is, the general attribute keywords are applicable to all test case configuration information, and keywords corresponding to general attributes of the test cases, such as test purposes, can be obtained. The special attribute key corresponds to the configuration information of each case type and corresponds to the type of the test case, for example, for the test case of abnormal test, the special attribute key may be abnormal condition 1, abnormal condition 2, etc. Therefore, the characteristic information of the test case can be completely acquired through the general attribute keywords and the special attribute keywords.

In step S130, the test case feature information is input into a pre-trained grouping model to obtain a plurality of test case groups.

In the embodiment of the present example, the pre-trained grouping model is a machine learning model obtained by training based on test case feature information of a large number of test cases as input and a group label to which each test case belongs as output, and the test cases are grouped by label calibration. In one embodiment, the number of test cases in each test case group is less than a predetermined threshold.

The group of tags can comprise a total tag which can be a parallel tag and a serial tag; the parallel tags can comprise sub-tags such as a parallel tag 1 and a parallel tag 2, and the serial tags can comprise sub-tags such as a serial tag 1 and a serial tag 2.

The test cases pointed by the parallelizable labels can be parallelized, each sub-label of the parallelizable labels can correspond to a plurality of test cases, namely each sub-label can correspond to one test case group, and the use cases in each parallelizable test case group are executed together in a concurrence mode, based on the potential relation among the use cases of the use case execution strategy of each use case and the key characteristics of the use cases, the use cases are grouped more reasonably, the test efficiency of the test cases is effectively improved during execution, and for example, the robust test can be carried out while the function test is carried out.

Typically, an efficient test script requires thousands of test cases to be completed in an hour, with a test case taking 1-2 minutes to execute 1000 cases, or more than one day if 1000 (1-2) cases are executed in serial fashion. Furthermore, in a parallel manner: 1000 test cases are operated within 1 hour, and 16 test cases are required to be completed by one-minute parallel tasks, so that the time can be effectively saved, and the efficiency is improved.

Meanwhile, for some special scenes, for example, the test case 2 can be operated after the test case 1 is finished, so that the test mode can be only used in a serial mode. By grouping, for example, dynamically dividing into 100 groups in 1000 pieces, test cases in groups that can be executed in parallel in the 100 groups in the subsequent step can be executed in parallel, and test cases that need to be executed in series can be executed in series.

Based on the grouping model, the test case components can be accurately and reasonably divided into a plurality of groups by utilizing the characteristic information of the test case, wherein the test case groups comprise a plurality of test case groups which can be parallel, so that the concurrent test in each test case group which can be parallel in the subsequent step can be realized, the high efficiency, reliability and stability of the test case execution in the subsequent step are ensured, and meanwhile, the reliability of the thread lock addition in the concurrent execution in the subsequent step (the thread lock addition in the concurrent execution of the test case in the test case group which can be parallel) is ensured through the reasonable grouping of the model.

In one embodiment, the training method of the grouping model includes:

The method comprises the steps that characteristic information samples of test cases are concentrated, each sample is calibrated by an expert according to combination experience among the test cases, then the characteristic information samples of the test cases are input into a grouping model, the training model outputs corresponding samples to achieve calibrated labels, and the labels can be represented by character strings or digital codes and the like. And when the grouping model aims at the sample set, the predicted grouping labels of all the samples are consistent with the grouping labels calibrated in advance for the samples, and the training is finished. Therefore, analysis output grouping labels of potential features of various cases reflected by feature information of the test cases can be trained.

In step S140, the test cases in the test case group of the parallelizable type are executed concurrently, and the thread lock is accessed to lock the test cases in execution while being executed concurrently, and the thread lock is released when the execution is finished.

In the embodiment of the example, the concurrent execution of the test case group is that multiple test cases are concurrently executed by multiple threads, so that the test speed is effectively improved.

When high concurrency testing is performed out-of-order and in parallel, only a single environment, say, testing of one scenario, can be tested. The multi-scenario test cannot be performed, because the high concurrency is used for the unordered arrangement of the threads, that is, which thread occupies the resources and then is mounted, so that deadlock with a certain probability is caused, and the problem of false alarm occurs. The unordered threads are arranged in the thread pool, dynamic grouping is carried out based on the grouping model, a plurality of test case groups capable of being parallel are obtained, parallel execution of the test cases is carried out, the test cases in operation are further locked through the thread lock, the thread lock can be released after the execution is finished, the thread lock is released when the execution of each test case is finished, the next lock is connected until the concurrent test of the test case groups is finished, and the orderliness of the test cases in the concurrent process is effectively guaranteed. Therefore, the resources can be released even after each test case thread is finished, and the problems of misinformation and the like can not be caused.

In one embodiment, referring to fig. 3, the concurrently executing test cases in a test case group of a parallelizable type, and accessing a thread lock to lock the test cases in execution concurrently, and releasing the thread lock when execution is finished, includes:

step S310, starting concurrent execution of test cases in the test case group with the parallel type, and accessing a first thread lock to lock a first test case which is started to run;

step S320, when the first test case for starting operation finishes operation, releasing the first thread lock, and accessing a second thread lock to lock a second test case for starting operation;

and step S330, when the second test case started to run finishes running, releasing the second thread lock, and sequentially and alternately accessing and releasing the subsequent thread locks to execute the subsequent test cases started to run until the test cases in the test case group capable of being parallel are tested.

For example, under the condition that a group of 7 test cases are not serial, a thread lock is used for locking the running test case, and then the second thread lock is continuously accessed through the process of releasing the lock after execution is completed, so that the situation that a false alarm bug is finally generated due to mutual exclusion between the test case and the use case is avoided, the test time can be saved in the execution process, and the test quality can be ensured. Particularly for the test case with very complex business logic, the authenticity and the test efficiency of the test result can be effectively ensured. Similarly, for test cases without business logic, parallel tests can be performed.

In step S150, when the test case in the test case group of the parallelizable type is executed, a concurrent test result is obtained.

In the embodiment of the present example, when the test case group capable of being parallel can cover all test cases, and when the execution of the test cases in the test case group capable of being parallel is finished, the concurrent test result, that is, the final test result corresponding to the target test task is obtained. Therefore, the concurrent test result is obtained when the execution of the test cases in the test case group capable of being parallel is finished, and the test result of the test task can be efficiently, accurately and stably obtained.

If the model grouping result includes other test case groups besides the test case group which can be in parallel, for example, the test case group needs to be in series, and the concurrent test result is a partial result of the test task. Therefore, when the execution of the test cases in the test case group capable of being parallel is finished, the concurrent test result is obtained, so that the part of the test result of the test task can be obtained efficiently, accurately and stably to the greatest extent, and the efficiency, accuracy and stability of the test can be further ensured on the whole.

In one embodiment, the plurality of test case groups further include a test case group whose test cases need to be serialized, and the method further includes:

The test case to which the serial label points is the result of the previous test case which needs to be used by the next test case after the test case is executed. Each sub-tag needing the serial tag can correspond to a plurality of test cases, namely each sub-tag can correspond to one test case group, and the use cases in each test case group are executed in a serial mode during execution. In an example, the test cases in a test case group that need to be serialized do not need to be serialized absolutely, and a group of multiple test cases that can effectively improve test stability when the test cases are executed serially can be reflected by the potential relationship between the use cases of the use case execution strategy of each use case and the key characteristics of the use cases.

And taking the concurrent test result and the serial test result as a total test result, so that the test can be completed completely and reliably to obtain a test result.

In one embodiment, the test case groups to be serialized are arranged in a group according to a serial order, and the serial execution of the test cases in the test case groups to be serialized includes:

Through the grouping model, the grouping labels and the label serial numbers are added to the test cases needing to be serialized according to the calling relation and other characteristics of the test cases, so that the serial sequence of the test case groups can be obtained, and the test case groups needing to be serialized are further arranged into groups according to the serial sequence. When the test cases in the test case group needing to be serialized are executed serially according to the serial sequence of the test cases in the test case group needing to be serialized, the serial accuracy can be effectively ensured.

The application also provides a testing device. Referring to fig. 4, the concurrency testing device includes:

the obtaining module 410 is configured to obtain configuration information of test cases in a test case set for a target test task;

the extraction module 420 is configured to extract test case feature information from the configuration information of the test case, where the test case feature information is a key attribute feature of an execution policy of each test case;

the grouping module 430 is configured to input the test case feature information into a grouping model trained in advance, so as to obtain a plurality of test case groups;

the concurrency module 440 is configured to concurrently execute test cases in the test case group of the parallelizable type, access the thread lock to lock the test cases in execution during concurrent execution, and release the thread lock when the execution is finished;

the end concurrency module 450 is configured to obtain a concurrent test result when the execution of the test case in the test case group of the parallelizable type is ended.

The specific details of each module in the testing apparatus have been described in detail in the corresponding concurrent testing method, and therefore, the details are not described herein again.

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the application. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Moreover, although the steps of the methods herein are depicted in the drawings in a particular order, this does not require or imply that the steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present application.

In an exemplary embodiment of the present application, there is also provided an electronic device capable of implementing the above method.

As will be appreciated by one skilled in the art, aspects of the present application may be embodied as a system, method or program product. Accordingly, various aspects of the present application may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

An electronic device 500 according to this embodiment of the present application is described below with reference to fig. 5. The electronic device 500 shown in fig. 5 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.

As shown in fig. 5, the electronic device 500 is embodied in the form of a general purpose computing device. The components of the electronic device 500 may include, but are not limited to: the at least one processing unit 510, the at least one memory unit 520, and a bus 530 that couples various system components including the memory unit 520 and the processing unit 510.

Wherein the storage unit stores program code that is executable by the processing unit 510 to cause the processing unit 510 to perform steps according to various exemplary embodiments of the present application as described in the above section "exemplary methods" of the present specification. For example, the processing unit 510 may perform the steps as shown in fig. 1.

The memory unit 520 may include a readable medium in the form of a volatile memory unit, such as a random access memory unit (RAM)5201 and/or a cache memory unit 5202, and may further include a read only memory unit (ROM) 5203.

Storage unit 520 may also include a program/utility 5204 having a set (at least one) of program modules 5205, such program modules 5205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 530 may be one or more of any of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

Electronic device 500 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and may also communicate with one or more devices that enable a client to interact with electronic device 500, and/or with any device (e.g., router, modem, etc.) that enables electronic device 500 to communicate with one or more other computing devices.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiments of the present application.

In an exemplary embodiment of the present application, referring to fig. 6, there is also provided a computer readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, various aspects of the present application may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the present application described in the "exemplary methods" section above of this specification, when the program product is run on the terminal device.

Referring to fig. 6, a program product 600 for implementing the above method according to an embodiment of the present application is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including AN object oriented programming language such as Java, C + +, or the like, as well as conventional procedural programming languages, such as the "C" language or similar programming languages.

Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the present application, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.

Claims

1. A method of testing, comprising:

and when the execution of the test cases in the test case group capable of being paralleled is finished, obtaining a concurrent test result.

2. The method of claim 1, wherein the training method of the packet model comprises:

3. The method of claim 1, wherein concurrently executing test cases in a group of test cases of a parallelizable type and accessing a thread lock to lock test cases in execution concurrently and releasing the thread lock at the end of execution comprises:

4. The method according to claim 1, wherein the extracting test case feature information from the configuration information of the test case comprises:

5. The method according to claim 1, wherein the extracting test case attribute information corresponding to the preset attribute keyword from the configuration information of the test case based on the preset attribute keyword to obtain the test case feature information comprises:

6. The method of claim 1, wherein the plurality of test case groups further includes a test case group whose test cases need to be serialized, and the method further comprises:

7. The method of claim 6, wherein the test case groups to be serialized are arranged in a serial order into groups, and the serially executing the test cases in the test case groups to be serialized comprises:

8. A test apparatus, comprising:

and the end concurrency module is used for obtaining a concurrency test result when the execution of the test cases in the test case group capable of being paralleled is ended.

9. A computer readable storage medium having stored thereon program instructions, characterized in that the program instructions, when executed by a processor, implement the method of any of claims 1-7.

10. An electronic device, comprising:

a processor; and

a memory for storing program instructions for the processor; wherein the processor is configured to perform the method of any of claims 1-7 via execution of the program instructions.