CN112905491B

CN112905491B - Software test effectiveness analysis method and device

Info

Publication number: CN112905491B
Application number: CN202110352732.7A
Authority: CN
Inventors: 熊小庆; 黄震人; 陈巧燕; 丁计来
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2021-03-31
Filing date: 2021-03-31
Publication date: 2024-02-27
Anticipated expiration: 2041-03-31
Also published as: CN112905491A

Abstract

The application provides a software test effectiveness analysis method and device, which can be used in the financial field, wherein the method comprises the following steps: determining a program segment to be tested by software and a corresponding variation test factor in the program to be tested according to a modification record of the program to be tested; carrying out concurrent mutation testing on the program segments to be tested by utilizing the mutation testing factors according to the mutation testing groups to which the program segments belong; and determining the validity of the software test according to the mutation test result of each program segment to be tested. The method and the device can select the corresponding mutation test factors for the program segments to be tested by the software, and conduct concurrent mutation test on the program segments to be tested by the software by utilizing the mutation test factors, so that the effectiveness of the software test is finally determined.

Description

Software test effectiveness analysis method and device

Technical Field

The invention relates to the field of software testing, and can be applied to the field of finance, in particular to a method and a device for analyzing the effectiveness of software testing.

Background

Software testing plays a vital role in ensuring the quality of software development, and various methods for analyzing the effectiveness of software testing exist in the industry, wherein mutation testing is one of the methods. The principle of analyzing the software test effectiveness by using the mutation test is to inject various mutation test factors into the source code, then run a unit test script, and analyze the software test effectiveness by evaluating the capability of the unit test script to detect the mutation test factors. If the unit test script which is executed successfully fails to be executed after the variant test factors are injected, the unit test script can intercept the variant test factors successfully, namely the variant test factors are killed successfully, so that the software test script is indicated to have effectiveness; if the test script fails to run after the variant test factor is injected, the variant test factor survives because the variant test factor is not intercepted, and the software test script is invalid or partially invalid. The analytical index of the effectiveness of the software test is the number of killed variant test factors/total variant test factors.

Although the mutation test is a powerful tool for analyzing the effectiveness of the software test, the problems of large resource cost, long execution time and the like required by calculation are highlighted under the large background of agile development advocated by the current software development, so that the current mutation test is difficult to widely apply in the software development process.

Disclosure of Invention

Aiming at the problems in the prior art, the application provides a software test effectiveness analysis method and device, which can select corresponding variation test factors for program segments to be tested by software, and perform concurrent variation test on the program segments to be tested by the software by using the variation test factors, so as to finally determine the effectiveness of the software test.

In order to solve the technical problems, the application provides the following technical scheme:

in a first aspect, the present application provides a software test effectiveness analysis method, including:

determining a program segment to be tested by software and a corresponding variation test factor in the program to be tested according to a modification record of the program to be tested;

carrying out concurrent mutation testing on the program segments to be tested by utilizing the mutation testing factors according to the mutation testing groups to which the program segments belong;

and determining the validity of the software test according to the mutation test result of each program segment to be tested.

Further, determining a program segment to be tested by the software and a corresponding mutation test factor in the program to be tested according to the modification record of the program to be tested, including:

determining the program segment to be tested by the software according to the modification record;

determining the instruction type, the calling times and the calling depth according to the program segment to be tested by the software;

and determining the variation test factor corresponding to the program segment to be tested by the software according to the instruction type, the calling times and the calling depth.

Further, performing concurrent mutation testing on the program segment to be tested by using the mutation testing factor according to the mutation testing group to which the program segment belongs, including:

performing software testing on the program segment to be tested by a pre-acquired test script to obtain a software testing result and a coverage rate of the software testing corresponding to the program segment to be tested;

determining a program segment to be subjected to mutation test according to the software test result and the coverage rate of the software test;

loading the mutation test factors to corresponding program segments to be subjected to mutation test;

and carrying out concurrent mutation testing on the program segments to be subjected to mutation testing after loading the mutation testing factors according to the mutation testing groups to which the program segments belong by using the test script.

Further, the determining the validity of the software test according to the mutation test result of each program segment to be tested includes:

determining the effective rate of the software test according to the number of killed variant test factors and the total number of the variant test factors;

and determining the validity of the software test according to a preset valid threshold and the valid rate.

Further, after determining the validity of the software test according to the mutation test result of each program segment to be tested, the method further comprises:

determining a failure type according to the instruction type of the variation test factor which is not killed and the coverage rate; the instruction type at least comprises an operator type, a code structure, a variable type and a function call type; the failure type at least comprises insufficient assertion and insufficient branch coverage;

determining a failure level according to a preset program keyword and the instruction type;

and determining the failure reason of the software test according to the failure type and the failure level.

In a second aspect, the present application provides a software test effectiveness analysis apparatus, comprising:

the program segment factor determining unit is used for determining a program segment to be tested by software in the program to be tested and a corresponding variation test factor according to the modification record of the program to be tested;

The mutation test unit is used for carrying out concurrent mutation test on the program section to be tested by utilizing the mutation test factors according to the mutation test groups to which the program section belongs;

and the validity determining unit is used for determining the validity of the software test according to the mutation test result of each program segment to be tested.

Further, the program segment factor determining unit includes:

the program segment determining module is used for determining the program segment to be tested by the software according to the modification record;

the instruction number depth determining module is used for determining instruction types, calling numbers and calling depths according to the program segments to be tested by the software;

and the variation factor determining module is used for determining the variation test factor corresponding to the program segment to be tested by the software according to the instruction type, the calling times and the calling depth.

Further, the mutation test unit includes:

the test result coverage rate determining module is used for carrying out software testing on the program segment to be tested by the pre-acquired test script to obtain a software test result and a coverage rate of the software test corresponding to the program segment to be tested;

The program segment determining module is used for determining a program segment to be subjected to mutation test according to the software test result and the coverage rate of the software test;

the factor loading module is used for loading the mutation test factors to the corresponding program segments to be subjected to mutation test;

and the mutation test module is used for carrying out concurrent mutation test on the program segments to be subjected to mutation test after the mutation test factors are loaded according to the mutation test groups to which the program segments belong by utilizing the test script.

Further, the validity determination unit includes:

an effective rate determining module, configured to determine an effective rate of the software test according to the number of killed variant test factors and the total number of the variant test factors;

and the validity determining module is used for determining the validity of the software test according to a preset validity threshold and the validity.

Further, the software test effectiveness analysis device further comprises:

the failure type determining module is used for determining the failure type according to the instruction type of the variation test factor which is not killed and the coverage rate; the instruction type at least comprises an operator type, a code structure, a variable type and a function call type; the failure type at least comprises insufficient assertion and insufficient branch coverage;

The failure level determining module is used for determining the failure level according to preset program keywords and the instruction type;

and the failure reason determining module is used for determining the failure reason of the software test according to the failure type and the failure level.

In a third aspect, the present application provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the software test effectiveness analysis method when executing the program.

In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the software test effectiveness analysis method.

Aiming at the problems in the prior art, the software test effectiveness analysis method and device provided by the application can select the program segments to be tested and the corresponding variation test factors thereof, then execute concurrent variation tests on the program segments to be tested by using the selected variation test factors according to different groups of the program segments to be tested, accurately daemon the program segments to be tested, and continuously improve the adaptation degree of the variation test factors based on user feedback, thereby avoiding wasting execution time and calculation capability due to injection of a large number of universal variation test factors, greatly improving the execution efficiency of the variation test, and shortening the execution time of the variation test.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of a software test effectiveness analysis method according to an embodiment of the present application;

FIG. 2 is a flowchart of determining a program segment to be tested by software and a corresponding variance test factor according to an embodiment of the present application;

FIG. 3 is a flow chart of concurrent mutation testing according to an embodiment of the present application;

FIG. 4 is a flow chart of determining the validity of a software test in an embodiment of the present application;

FIG. 5 is a second flowchart of a software test effectiveness analysis method according to an embodiment of the present application;

FIG. 6 is a block diagram of a software test effectiveness analysis device according to an embodiment of the present application;

FIG. 7 is a block diagram of a program segment factor determination unit in an embodiment of the present application;

FIG. 8 is a block diagram of a variation test unit according to an embodiment of the present application;

Fig. 9 is a block diagram of a validity determining unit in the embodiment of the present application;

FIG. 10 is a second block diagram of a software test effectiveness analysis device according to an embodiment of the present application;

fig. 11 is a schematic structural diagram of an electronic device in an embodiment of the present application.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The software test effectiveness analysis method and device relate to the field of information safety, can be used in the financial field and any field except the financial field, and are not limited in application field.

Referring to fig. 1, in order to select a corresponding mutation test factor for a program segment to be tested by software, and perform concurrent mutation test on the program segment to be tested by software by using the mutation test factor, the validity of the software test is finally determined, and the application provides a software test validity analysis method, which comprises the following steps:

S101: determining a program segment to be tested by the software and a corresponding variation test factor in the program to be tested according to the modification record of the program to be tested;

it can be understood that in the embodiment of the present application, the program to be tested needs to be first subjected to a software test, and then a mutation test is performed on a program segment passing the software test, so as to determine the effectiveness of the software test.

In order to determine the program segment to be tested by the software, the embodiment of the application can query the modification record of the program to be tested. The modification record is actually a program modification record, the program modification record is derived from Git, which is a widely used code hosting and developing platform, and can record a file list submitted by a program developer each time and code change details specific to a line, and all change programs, change methods and change types submitted at present can be analyzed through a Hash value submitted by an incoming program, such as adding, deleting, and the like. By querying the Git it is possible to know which program segments have been modified, so that only software testing is required for these modified program segments.

When the pipeline execution reaches the mutation test execution stage, the embodiment of the application scans the code and the method (function) of the modification according to the program modification record recorded on the Git, and searches the corresponding test script.

When the program is executed for the first time, the test script may take the full amount of test script.

After determining the program segment to be tested by the software, the suitable mutation test factors can be selected according to the characteristics, and the specific selection method is described in S201-S203.

S102: carrying out concurrent mutation test on the program segments to be tested by using the mutation test factors according to the mutation test groups to which the program segments belong;

it can be understood that, since the mutation test in the embodiment of the present application is performed concurrently, it is necessary to group a plurality of program segments to be tested by software, so that each group can perform the mutation test at the same time, thereby improving the test efficiency. The mutation test group can be determined according to the characteristics of the program segment, and the details of the mutation test group are described in the description of S304.

S103: and determining the validity of the software test according to the mutation test result of each program segment to be tested.

It will be appreciated that after the mutation test is completed, the validity of the software test may be determined. The software test for failure can also analyze its cause of failure for subsequent improvement. Wherein, the mutation test result at least comprises: the variant factor is successfully killed and the variant factor is not successfully killed.

As can be seen from the above description, the software test effectiveness analysis method provided by the present application can select a program segment to be tested by software and a corresponding variation test factor thereof, then execute concurrent variation test on the program segment to be tested by software according to different groups of the program segment to be tested by the selected variation test factor, and accurately daemon the program segment to be tested, and can continuously promote the adaptation degree of the variation test factor based on user feedback, thereby avoiding wasting execution time and calculation capability due to injection of a large number of universal variation test factors, greatly improving the execution efficiency of the variation test, and shortening the execution time of the variation test.

Referring to fig. 2, determining a program segment to be tested by software and a corresponding mutation test factor in the program to be tested according to a modification record of the program to be tested includes:

s201: determining a program segment to be tested by the software according to the modification record;

s202: determining the characteristics of the program segment to be tested by the software according to the program segment to be tested by the software, wherein the characteristics comprise instruction types, calling times, calling depth and the like;

s203: and determining a mutation test factor corresponding to the program segment to be tested by the software according to the program segment characteristics.

It will be appreciated that in embodiments of the present application, a program developer may submit modified program code on Git. In the embodiment of the application, only the modified program code is subjected to the software test and the mutation test, so that the program segment to be subjected to the software test can be determined according to the modification record. These program segments to be tested by the software may also be referred to as change programs.

The change program can be classified into a core program and a general program according to the number of times a method (function) in the source code is called and the depth of the call. The kernel refers to a program that has a large number of times a method (function) is called and has a large depth of call; the general program is a program having a small number of times a method (function) is called and a shallow depth.

According to the program classification, the corresponding variation test factors can be selected according to the program instruction types. For a core program, a plurality of variant test factors can be selected to obtain a more comprehensive test effect; for general procedures, only the less critical, higher-level, small number of variant test factors may be selected for more targeted, efficient testing.

Aiming at a change program, the embodiment of the application adopts an abstract syntax tree (Abstract Syntax Tree, AST) analysis method to determine the number of times and depth of the method (function) to be called, and simultaneously sets the threshold value of the number of times and the depth of the call in advance; when at least one core method exists in the change program, the change program can be identified as a core program.

The judging process of the core method is as follows: when the number of times of being called is larger than a preset calling number threshold value or the maximum calling depth is larger than a preset calling depth threshold value, the method (function) is a core method.

isKeyMethod＝invokeTime>INVOKE_TIME_THRESHOLD||max(invokeDepth)>INVOKE_DEPTH_THRESHOLD

On this basis, a variant test factor can be determined.

The selection of the mutation test factors may be considered from two aspects, program level and source code characteristics. The kernel program selects more variant test factors, which may include removing various variant test factors such as calling variant test factors, conditional access variant test factors, incremental modification variant test factors, and the like. A typical procedure may select a small number of variance test factors, such as boundary value variance test factors, that are more critical and have a higher level.

From the above description, the software testing effectiveness analysis method provided by the present application can determine the program segment to be tested by the software and the corresponding mutation test factor according to the modification record of the program to be tested.

Referring to fig. 3, concurrent mutation testing is performed on a program segment to be tested by using a mutation test factor according to a mutation test group to which the program segment belongs, including:

s301: performing software testing on a program segment to be tested by a pre-acquired test script to obtain a software testing result and a coverage rate of the software testing corresponding to the program segment to be tested;

it will be appreciated that the program segment to be tested by software has been determined in the previous step, and that the program segment to be tested by software can be tested by using the test script, and the software test is performed by using a conventional method. After the software test is completed, a software test result and a coverage rate of the software test can be obtained. The software test result refers to whether the program segment to be tested by the software passes the software test. The coverage rate of the software test refers to whether a test script corresponds to a program segment to be tested by the software, because not every program segment to be tested by the software corresponds to the test script, the program segment to be tested by the software corresponding to the test script cannot complete the software test, that is, is not covered by the software test.

Specifically, a unit test framework is found in source codes of a change program, test cases in the change program and a test script are used as factors to be executed at the time to be written into a configuration file, an execution software test is called, and an execution result and coverage rate data are collected.

S302: determining a program segment to be subjected to mutation test according to a software test result and the coverage rate of the software test;

it is understood that mutation testing in embodiments of the present application is performed only for program segments that successfully pass software testing.

It should be noted that if the software test is not passed, the problem of the program itself is solved, and the program code needs to be modified, so that the mutation test is not required; meanwhile, if the test script is not covered by the software test, the subsequent mutation test cannot be executed, because the mutation test factor needs to be injected as a carrier to execute the mutation test.

S303: loading a mutation test factor to a corresponding program segment to be subjected to mutation test;

it should be noted that, the loading step is also called an injecting step, and is mainly to replace the target instruction with a mutated instruction. The injection process of the variation factor employs ASM (byte code enhancement) technique to insert the variation factor into the source code.

S304: and carrying out concurrent mutation testing on the program segments to be subjected to mutation testing according to the mutation testing groups to which the program segments belong by using the testing script.

It can be appreciated that in order to improve the efficiency of mutation testing, the embodiments of the present application employ a concurrent execution mechanism. For concurrent execution, the programs to be tested need to be grouped, for example, the characteristics of the codes corresponding to the programs to be tested can be analyzed, the codes with multiple variation points and multiple instructions are distributed to the execution units with high configuration and low load, and the codes with low resource requirements are distributed to the execution units with low configuration and high load. Wherein, the state of the execution unit may include, but is not limited to: an execution machine occupation state, a CPU load state, a memory load state and the like. The characteristics of the code may include, but are not limited to: instruction number, instruction type, etc.

After grouping is completed, the concurrent mutation test can be performed on the program segments to be subjected to mutation test after the mutation test factors are loaded according to the mutation test groups to which the program segments belong by using the test script.

From the above description, the software test effectiveness analysis method provided by the application can use the mutation test factors to perform concurrent mutation test on the program segments to be tested according to the mutation test groups to which the program segments belong.

Referring to fig. 4, determining the validity of the software test according to the mutation test result of each program segment to be tested includes:

s401: determining the effective rate of the software test according to the number of the killed variant test factors and the total number of the variant test factors;

s402: and determining the validity of the software test according to a preset valid threshold and valid efficiency.

It should be noted that the above process can be understood as follows: if the test script fails to test in the mutation test, the software test script is effective, because the test script successfully discovers the mutation; otherwise, the software test script fails or partially fails. In addition, the present application does not specifically limit the preset effective rate threshold, and may be set according to actual conditions.

From the above description, the software test effectiveness analysis method provided by the application can determine the effectiveness of the software test according to the mutation test result of each program segment to be tested.

Referring to fig. 5, after determining the validity of the software test according to the mutation test result of each program segment to be tested, the method further includes:

s501: determining a failure type according to the instruction type of the variation test factor which is not killed and the coverage rate; the instruction type at least comprises an operator type, a code structure, a variable type and a function call type; the failure type at least comprises insufficient assertion and insufficient branch coverage;

it can be appreciated that the embodiment of the application can automatically analyze and classify the execution result of the mutation test by adopting a combination of instruction analysis and coverage rate analysis. For variant test factors that are not "killed," the cause may be specifically analyzed as insufficient assertion or branch coverage, etc., based on the instruction type of the variant factor and the coverage data of the software test. For each question, classification may be based on a classification rule base according to the variance test factor and keywords in the code. The rule base is initially initialized manually, and can be updated later according to feedback of a developer, and the updating method is described in the step S503.

S502: determining a failure level according to a preset program keyword and an instruction type, if the return value is not asserted as a serious problem, and if the exception type is not asserted as a general problem;

it will be appreciated that during execution of the mutation test, the pipeline records detailed mutation execution logs, including execution steps, time consuming steps, load conditions of the execution machine during execution, and the like. In the step, the logs are collected and analyzed to obtain the data such as the load of the execution machine, the execution time and the like in the execution process of different variation test factors. This data may be used as a basis for the mutation test groupings before the next mutation test is performed.

S503: and determining the failure reason of the software test according to the failure type and the failure level.

From the above description, the software test effectiveness analysis method provided by the application can determine the failure reason of the software test.

The test effectiveness is calculated firstly, the calculation mode is that the killed variation number/total variation number is used as an access control value submitted to a construction pipeline, and submitted codes can be refused to be combined to a main branch when the effectiveness is lower than a threshold value (such as 80%), so that the quality of unit test is controlled. Meanwhile, the undeleted variation details are pushed to a code developer, and the developer manually analyzes whether the effectiveness and grade assessment of the problems are reasonable or not and feeds back the problems. Based on the feedback of the developer, the classification rule base of the mutation factor used in step S301 can be expanded, so that the mutation result can be conveniently and effectively analyzed later.

It should be noted that, in an embodiment, step S101 may reduce the code range of the mutation test to a single number, and reduce the mutation test factor of the mutation to a minimum range; step S102 can realize concurrent scheduling during mutation testing, so that the execution time of a single mutation test is shortened from an hour level to a minute level, the mutation test can be integrated into a pipeline for execution, the effectiveness of a test case is continuously detected, and the function of guarding the code quality is truly played; step S103 may collect the results of mutation execution, and continuously update the mutation test factors according to the feedback of the developer, so that the mutation test factors most suitable for each code are more applied to the mutation test of each code, thereby playing a greater role.

Based on the same inventive concept, the embodiments of the present application also provide a software test effectiveness analysis device, which may be used to implement the method described in the above embodiments, as described in the following embodiments. Because the principle of the software test effectiveness analysis device for solving the problem is similar to that of the software test effectiveness analysis method, the implementation of the software test effectiveness analysis device can be referred to the implementation of the determination method based on the software performance benchmark, and the repetition is omitted. As used below, the term "unit" or "module" may be a combination of software and/or hardware that implements the intended function. While the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

Referring to fig. 6, in order to select a corresponding mutation test factor for a program segment to be tested by software, and perform concurrent mutation test on the program segment to be tested by software by using the mutation test factor, the validity of the software test is finally determined, the application provides a software test validity analysis device, which comprises:

the program segment factor determining unit 601 is configured to determine a program segment to be tested by software and a corresponding mutation test factor in the program to be tested according to a modification record of the program to be tested;

a mutation testing unit 602, configured to perform concurrent mutation testing on the program segment to be tested by using the mutation testing factor according to a mutation testing group to which the program segment belongs;

and the validity determining unit 603 is configured to determine validity of the software test according to a mutation test result of each program segment to be tested.

Referring to fig. 7, the program segment factor determining unit 601 includes:

a program segment determining module 701, configured to determine the program segment to be tested by the software according to the modification record;

the instruction number depth determining module 702 is configured to determine an instruction type, a call number and a call depth according to the program segment to be tested by the software;

And a mutation factor determining module 703, configured to determine the mutation test factor corresponding to the program segment to be tested by the software according to the instruction type, the call number and the call depth.

Referring to fig. 8, the mutation testing unit 602 includes:

the test result coverage rate determining module 801 is configured to perform a software test on the program segment to be tested by using a test script obtained in advance, so as to obtain a software test result and a coverage rate of the software test corresponding to the program segment to be tested;

a program segment determining module 802, configured to determine a program segment to be tested in a mutation manner according to the software test result and the coverage rate of the software test;

a factor loading module 803, configured to load the mutation test factor into a corresponding program segment to be tested in a mutation manner;

and the mutation test module 804 is configured to perform concurrent mutation test on the program segment to be subjected to mutation test after the mutation test factor is loaded according to the mutation test group to which the program segment belongs by using the test script.

Referring to fig. 9, the validity determining unit 603 includes:

an effective rate determining module 901, configured to determine an effective rate of the software test according to the number of killed variant test factors and the total number of the variant test factors;

The validity determining module 902 is configured to determine validity of the software test according to a preset validity threshold and the validity.

Referring to fig. 10, the software test effectiveness analysis device further includes:

a failure type determining module 1001, configured to determine a failure type according to an instruction type of the variation test factor that is not killed and the coverage rate; the instruction type at least comprises an operator type, a code structure, a variable type and a function call type; the failure type at least comprises insufficient assertion and insufficient branch coverage;

the failure level determining module 1002 is configured to determine a failure level according to a preset program keyword and the instruction type;

and a failure reason determining module 1003, configured to determine a failure reason of the software test according to the failure type and the failure level.

In order to select a corresponding mutation test factor for a program segment to be tested by software and perform concurrent mutation test on the program segment to be tested by using the mutation test factor and finally determine the validity of the software test, the application provides an embodiment of an electronic device for implementing all or part of the content in the method for analyzing the validity of the software test, wherein the electronic device specifically comprises the following contents:

A Processor (Processor), a Memory (Memory), a communication interface (Communications Interface), and a bus; the processor, the memory and the communication interface complete communication with each other through the bus; the communication interface is used for realizing information transmission between the software test effectiveness analysis device and related equipment such as a core service system, a user terminal, a related database 104 and the like; the logic controller may be a desktop computer, a tablet computer, a mobile terminal, etc., and the embodiment is not limited thereto. In this embodiment, the logic controller may refer to an embodiment of the software test effectiveness analysis method and an embodiment of the software test effectiveness analysis device, and the contents thereof are incorporated herein, and the repetition is omitted.

It is understood that the user terminal may include a smart phone, a tablet electronic device, a network set top box, a portable computer, a desktop computer, a Personal Digital Assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Wherein, intelligent wearing equipment can include intelligent glasses, intelligent wrist-watch, intelligent bracelet etc..

In practical applications, part of the software test effectiveness analysis method may be executed on the electronic device side as described above, or all operations may be completed in the client device. Specifically, the selection may be made according to the processing capability of the client device, and restrictions of the use scenario of the user. The present application is not limited in this regard. If all operations are performed in the client device, the client device may further include a processor.

The client device may have a communication module (i.e. a communication unit) and may be connected to a remote server in a communication manner, so as to implement data transmission with the server. The server may include a server on the side of the task scheduling center, and in other implementations may include a server on an intermediate platform, such as a server on a third party server platform having a communication link with the task scheduling center server. The server may include a single computer device, a server cluster formed by a plurality of servers, or a server structure of a distributed device.

Fig. 11 is a schematic block diagram of a system configuration of an electronic device 9600 of an embodiment of the present application. As shown in fig. 11, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this fig. 11 is exemplary; other types of structures may also be used in addition to or in place of the structures to implement telecommunications functions or other functions.

In one embodiment, the software test effectiveness analysis method functionality may be integrated into the central processor 9100. The central processor 9100 may be configured to perform the following control:

S101: determining a program segment to be tested by software and a corresponding variation test factor in the program to be tested according to a modification record of the program to be tested;

s102: carrying out concurrent mutation testing on the program segments to be tested by utilizing the mutation testing factors according to the mutation testing groups to which the program segments belong;

As can be seen from the above description, the software test effectiveness analysis method and apparatus provided by the present application can select a program segment to be tested by software and a corresponding variation test factor thereof, then execute concurrent variation test on the program segment to be tested by the selected variation test factor according to different groupings of the program segment to be tested by software, and accurately daemon the program segment to be tested by software.

In another embodiment, the software test effectiveness analysis device may be configured separately from the cpu 9100, for example, the data composite transmission device software test effectiveness analysis device may be configured as a chip connected to the cpu 9100, and the function of the software test effectiveness analysis method is implemented by the control of the cpu.

As shown in fig. 11, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 need not include all of the components shown in fig. 11; in addition, the electronic device 9600 may further include components not shown in fig. 11, and reference may be made to the related art.

As shown in fig. 11, the central processor 9100, sometimes referred to as a controller or operational control, may include a microprocessor or other processor device and/or logic device, which central processor 9100 receives inputs and controls the operation of the various components of the electronic device 9600.

The memory 9140 may be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information about failure may be stored, and a program for executing the information may be stored. And the central processor 9100 can execute the program stored in the memory 9140 to realize information storage or processing, and the like.

The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, but not limited to, an LCD display.

The memory 9140 may be a solid state memory such as Read Only Memory (ROM), random Access Memory (RAM), SIM card, etc. But also a memory which holds information even when powered down, can be selectively erased and provided with further data, an example of which is sometimes referred to as EPROM or the like. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 storing application programs and function programs or a flow for executing operations of the electronic device 9600 by the central processor 9100.

The memory 9140 may also include a data store 9143, the data store 9143 for storing data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers of the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, address book applications, etc.).

The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide an input signal and receive an output signal, which may be the same as in the case of a conventional mobile communication terminal.

Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless lan module, may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and to receive audio input from the microphone 9132 to implement usual telecommunications functions. The audio processor 9130 can include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100 so that sound can be recorded locally through the microphone 9132 and sound stored locally can be played through the speaker 9131.

The embodiments of the present application further provide a computer-readable storage medium capable of implementing all the steps in the software test effectiveness analysis method in which the execution subject is a server or a client in the above embodiments, and the computer-readable storage medium stores thereon a computer program that when executed by a processor implements all the steps in the software test effectiveness analysis method in which the execution subject is a server or a client in the above embodiments, for example, the processor implements the following steps when executing the computer program:

It will be apparent to those skilled in the art that embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The principles and embodiments of the present invention have been described in detail with reference to specific examples, which are provided to facilitate understanding of the method and core ideas of the present invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims

1. A method for analyzing the effectiveness of a software test, comprising:

Determining the validity of the software test according to the mutation test result of each program segment to be tested;

the method for determining the program segment to be tested by the software and the corresponding variation test factors in the program to be tested according to the modification record of the program to be tested comprises the following steps:

2. The method according to claim 1, wherein the step of performing concurrent mutation testing on the program segment to be tested by using the mutation test factor according to the mutation test group to which the program segment belongs includes:

3. The method for analyzing the validity of a software test according to claim 1, wherein determining the validity of the software test according to the mutation test result of each program segment to be tested comprises:

4. The software test effectiveness analysis method of claim 2, further comprising, after determining the effectiveness of the software test based on the mutation test result of each of the program segments to be software tested:

5. A software test effectiveness analysis device, comprising:

the validity determining unit is used for determining the validity of the software test according to the mutation test result of each program segment to be tested;

wherein the program segment factor determining unit includes:

6. The software testing effectiveness analysis device of claim 5, wherein the mutation testing unit includes:

7. The software testing effectiveness analysis device of claim 5, wherein the effectiveness determination unit includes:

8. The software testing effectiveness analysis device of claim 6, further comprising:

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the software test effectiveness analysis method of any one of claims 1 to 4 when the program is executed by the processor.

10. A computer readable storage medium having stored thereon a computer program, characterized in that the computer program, when executed by a processor, implements the steps of the software test effectiveness analysis method of any one of claims 1 to 4.