CN112363928B - Processing method and device of test case, processor and electronic device - Google Patents
Processing method and device of test case, processor and electronic device Download PDFInfo
- Publication number
- CN112363928B CN112363928B CN202011247876.8A CN202011247876A CN112363928B CN 112363928 B CN112363928 B CN 112363928B CN 202011247876 A CN202011247876 A CN 202011247876A CN 112363928 B CN112363928 B CN 112363928B
- Authority
- CN
- China
- Prior art keywords
- test
- test case
- cases
- probability
- type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Debugging And Monitoring (AREA)
Abstract
The invention discloses a processing method and device of a test case, a processor and an electronic device. The method comprises the following steps: acquiring currently acquired test data, wherein the test data comprises: the test method comprises the steps of a plurality of first test cases and test results corresponding to each first test case in the plurality of first test cases, wherein the test results comprise: code blocks of the software to be tested hit by each first test case in the execution process and at least one type of loopholes detected by each first test case in the execution process; calculating actual measurement probability of each first test case based on the test data; constructing a second test case, and acquiring the prediction probability of the second test case based on the actual measurement probability; and determining whether to add the second test case into the test data by using a comparison result of the predicted probability and the actually measured probability. The invention solves the technical problems that the test case generated by simply relying on manual test experience in the related technology needs to consume high labor cost, thereby being difficult to meet the agile development requirement.
Description
Technical Field
The present invention relates to the field of computers, and in particular, to a method and apparatus for processing a test case, a processor, and an electronic device.
Background
At present, under the background of rapid iteration and agile development, the test result is adversely affected whether the white box test is excessively depended or the black box test is excessively depended. Each successful test activity relies on a priori understanding of the object under test and generates test cases based thereon. However, the test cases generated by relying on manual test experience only require high labor cost, and thus are difficult to meet the requirements of agile development. When performing new characteristic iteration, how to design a brand new and effective test case for testing, and how to select a divergence reference in the existing test case when performing divergence test are all technical problems to be solved.
In the process of developing a debugger, a debugger can be used to record a part of code executed by the software to be tested in the process of executing the software. However, there is a lack of corresponding solutions for the test links.
In view of the above problems, no effective solution has been proposed at present.
Disclosure of Invention
At least some embodiments of the present invention provide a method, an apparatus, a processor, and an electronic device for processing a test case, so as to at least solve the technical problem that the test case generated by simply relying on manual test experience in the related art needs to consume high labor cost, thereby being difficult to meet the agile development requirement.
According to one embodiment of the present invention, there is provided a method for processing a test case, including:
acquiring currently acquired test data, wherein the test data comprises: the test method comprises the steps of a plurality of first test cases and test results corresponding to each first test case in the plurality of first test cases, wherein the test results comprise: code blocks of the software to be tested hit by each first test case in the execution process and at least one type of loopholes detected by each first test case in the execution process; calculating actual measurement probability of each first test case based on the test data; constructing a second test case, and acquiring the prediction probability of the second test case based on the actual measurement probability; and determining whether to add the second test case into the test data by using a comparison result of the predicted probability and the actually measured probability.
Optionally, the method further comprises: and performing block processing on codes in the software to be tested to obtain a plurality of code blocks.
Optionally, calculating the measured probability of each first test case based on the test data includes: acquiring first hit frequencies of each code block in the plurality of code blocks in the plurality of first test cases; acquiring the occurrence frequency of each code block in the plurality of code blocks in a first test case in which each type of vulnerability in at least one type of vulnerability is detected; acquiring the number ratio of the first test cases with each type of loopholes in at least one type of loopholes relative to a plurality of first test cases; based on the first hit frequency, the occurrence frequency and the number ratio, calculating the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block, and obtaining the actual measurement probability of each first test case.
Optionally, obtaining the first hit frequency of each code block in the plurality of code blocks in the plurality of first test cases includes: acquiring first hit times of each code block in a plurality of first test cases; acquiring second hit times of the code blocks in the first test cases; and calculating the ratio of the first hit times to the second hit times to obtain the first hit frequency.
Optionally, obtaining the frequency of occurrence of each code block of the plurality of code blocks in the first test case in which each type of vulnerability of the at least one type of vulnerability is detected includes: screening first test cases in which each type of vulnerability is detected from a plurality of first test cases respectively; and counting the occurrence frequency of each code block in the first test case obtained by screening.
Optionally, obtaining the number ratio of the first test cases in which each type of vulnerability occurs in the at least one type of vulnerability to the plurality of first test cases includes: acquiring the number of first cases of first test cases of each type of loopholes in at least one type of loopholes; acquiring the number of second cases of the plurality of first test cases; and calculating the ratio of the number of the first cases to the number of the second cases to obtain the number ratio.
Optionally, obtaining the predicted probability of the second test case based on the measured probability includes: acquiring a second hit frequency of each code block in the plurality of code blocks in a second test case; acquiring the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block; and calculating the prediction probability of the second test case for each type of loopholes based on the second hit frequency and the conditional probability.
Optionally, determining whether to add the second test case to the test data using the comparison of the predicted probability and the measured probability includes: setting a judging condition based on the number of the current cases of the plurality of first test cases; when the comparison result meets the judgment condition, determining to add the second test case into the test data; and when the comparison result does not meet the judgment condition, carrying out iterative processing on the second test case until the comparison result meets the judgment condition.
According to one embodiment of the present invention, there is also provided a processing device for a test case, including:
the acquisition module is used for acquiring the currently acquired test data, wherein the test data comprises: the test method comprises the steps of a plurality of first test cases and test results corresponding to each first test case in the plurality of first test cases, wherein the test results comprise: code blocks of the software to be tested hit by each first test case in the execution process and at least one type of loopholes detected by each first test case in the execution process; the calculation module is used for calculating actual measurement probability of each first test case based on the test data; the processing module is used for constructing a second test case and acquiring the prediction probability of the second test case based on the actual measurement probability; and the comparison module is used for determining whether to add the second test case into the test data by using the comparison result of the prediction probability and the actual measurement probability.
Optionally, the apparatus further includes: the dividing module is used for carrying out block processing on codes in the software to be detected to obtain a plurality of code blocks.
Optionally, the computing module is configured to obtain a first hit frequency of each code block in the plurality of first test cases; acquiring the occurrence frequency of each code block in the plurality of code blocks in a first test case in which each type of vulnerability in at least one type of vulnerability is detected; acquiring the number ratio of the first test cases with each type of loopholes in at least one type of loopholes relative to a plurality of first test cases; based on the first hit frequency, the occurrence frequency and the number ratio, calculating the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block, and obtaining the actual measurement probability of each first test case.
Optionally, the computing module is configured to obtain a first hit number of each code block in the plurality of first test cases; acquiring second hit times of the code blocks in the first test cases; and calculating the ratio of the first hit times to the second hit times to obtain the first hit frequency.
Optionally, the computing module is configured to screen out first test cases that detect each type of vulnerability from the plurality of first test cases respectively; and counting the occurrence frequency of each code block in the first test case obtained by screening.
Optionally, the computing module is configured to obtain a first number of cases of the first test case where each type of the at least one type of the vulnerabilities occurs; acquiring the number of second cases of the plurality of first test cases; and calculating the ratio of the number of the first cases to the number of the second cases to obtain the number ratio.
Optionally, the processing module is configured to obtain a second hit frequency of each code block in the plurality of code blocks in the second test case; acquiring the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block; and calculating the prediction probability of the second test case for each type of loopholes based on the second hit frequency and the conditional probability.
Optionally, the comparison module is used for setting a judging condition based on the number of the current cases of the plurality of first test cases; when the comparison result meets the judgment condition, determining to add the second test case into the test data; and when the comparison result does not meet the judgment condition, carrying out iterative processing on the second test case until the comparison result meets the judgment condition.
According to one embodiment of the present invention, there is also provided a nonvolatile storage medium in which a computer program is stored, wherein the computer program is configured to execute the processing method of the test case in any one of the above-mentioned aspects when running.
According to an embodiment of the present invention, there is further provided a processor for running a program, where the program is configured to execute the processing method of the test case in any one of the above-mentioned claims when running.
According to one embodiment of the present invention, there is also provided an electronic device including a memory in which a computer program is stored, and a processor configured to run the computer program to perform the method of processing the test case in any one of the above.
In at least some embodiments of the present invention, the test data acquired at present is adopted, the test data includes a plurality of first test cases and test results corresponding to each first test case in the plurality of first test cases, the test results include code blocks of software to be tested hit by each first test case in the execution process and at least one type of bug detected by each first test case in the execution process, the actual measurement probability of each first test case is obtained through test data calculation, the prediction probability of the second test case is obtained by constructing the second test case and based on the actual measurement probability, and whether the second test case is added into the test data is determined by using the comparison result of the prediction probability and the actual measurement probability, so that the prediction probability of the new test case for detecting different types of bugs is obtained by using the test data acquired, whether the new test case can be added into the test data is judged, the purpose of continuously updating the test cases included in the test data is achieved, the dependence of the new test case on the artificial test experience is reduced, the test result is obtained through calculation, the test probability of the new test case on the artificial test experience is obtained, the test case is difficult to be met through the artificial test, the problem is solved by the high-cost due to the fact that the test is required to the artificial test is required by the test, and the development of the technology is difficult to meet the requirements of the high-cost technology.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a flow chart of a method of processing test cases according to one embodiment of the application;
FIG. 2 is a diagram illustrating a mapping relationship between a plurality of test cases and different types of vulnerabilities according to an alternative embodiment of the present application;
FIG. 3 is a schematic diagram of predicting the probability of occurrence of a particular type of vulnerability in accordance with an alternative embodiment of the present application;
FIG. 4 is a block diagram of a processing device for a test case according to one embodiment of the application;
FIG. 5 is a block diagram of a test case processing apparatus according to an alternative embodiment of the present application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
According to one embodiment of the present invention, there is provided an embodiment of a method of processing test cases, it being noted that the steps illustrated in the flowchart of the figures may be performed in a computer system, such as a set of computer executable instructions, and that although a logical order is illustrated in the flowchart, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.
The method embodiments may be performed in a mobile terminal, a computer terminal, or similar computing device. Taking the mobile terminal as an example, the mobile terminal can be a terminal device such as a smart phone (such as an Android mobile phone, an iOS mobile phone, etc.), a tablet computer, a palm computer, a mobile internet device (Mobile Internet Devices, abbreviated as MID), a PAD, etc. A mobile terminal may include one or more processors (which may include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processor (GPU), a Digital Signal Processing (DSP) chip, a Microprocessor (MCU), a programmable logic device (FPGA), a neural Network Processor (NPU), a Tensor Processor (TPU), an Artificial Intelligence (AI) type processor, etc.) and a memory for storing data. Optionally, the mobile terminal may further include a transmission device, an input/output device, and a display device for a communication function. It will be appreciated by those of ordinary skill in the art that the foregoing structural descriptions are merely illustrative and are not intended to limit the structure of the mobile terminal. For example, the mobile terminal may also include more or fewer components than the above structural description, or have a different configuration than the above structural description.
The memory may be used to store a computer program, for example, a software program of an application software and a module, for example, a computer program corresponding to a processing method of a test case in an embodiment of the present invention, and the processor executes various functional applications and data processing by running the computer program stored in the memory, that is, implements the processing method of the test case. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid state memory. In some examples, the memory may further include memory remotely located with respect to the processor, the remote memory being connectable to the mobile terminal through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device includes a network adapter (Network Interface Controller, simply referred to as NIC) that can connect to other network devices through the base station to communicate with the internet. In one example, the transmission device may be a Radio Frequency (RF) module, which is used to communicate with the internet wirelessly.
Display devices may be, for example, touch screen type Liquid Crystal Displays (LCDs) and touch displays (also referred to as "touch screens" or "touch display screens"). The liquid crystal display may enable a user to interact with a user interface of the mobile terminal. In some embodiments, the mobile terminal has a Graphical User Interface (GUI), and the user may interact with the GUI by touching finger contacts and/or gestures on the touch-sensitive surface, where the man-machine interaction functions optionally include the following interactions: executable instructions for performing the above-described human-machine interaction functions, such as creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, sending and receiving electronic mail, talking interfaces, playing digital video, playing digital music, and/or web browsing, are configured/stored in a computer program product or readable storage medium executable by one or more processors.
In this embodiment, a method for processing a test case running on the mobile terminal is provided, and fig. 1 is a flowchart of a method for processing a test case according to one embodiment of the present invention, as shown in fig. 1, where the method includes the following steps:
step S12, acquiring currently acquired test data, wherein the test data comprises: the test method comprises the steps of a plurality of first test cases and test results corresponding to each first test case in the plurality of first test cases, wherein the test results comprise: code blocks of the software to be tested hit by each first test case in the execution process and at least one type of loopholes detected by each first test case in the execution process;
In software engineering, it is necessary to continuously add new software characteristics and gradually iterate out the final software application. For example, 10 development iterations are required in the development of a piece of software, and then the piece of software is also required to undergo 10 tests. And the test difficulty is gradually increased along with the increase of the iteration times. Taking the 10 th development iteration test as an example, the testing process needs to ensure that the 10 th development iteration itself does not have quality problems, and also needs to ensure that after the 10 th development iteration is combined with the 9 first development iterations, the code of the 9 first development iterations does not have quality problems. Then, for the 10 th test, the first test cases used in the first 9 tests and the test results corresponding to each first test case in the first test cases (including the code block of the software to be tested hit by each first test case in the execution process and at least one type of bug detected by each first test case in the execution process) are the test data.
Step S14, calculating actual measurement probability of each first test case based on the test data;
s16, constructing a second test case, and acquiring the prediction probability of the second test case based on the actual measurement probability;
In each development iteration process, the existing first test cases can be used completely to complete the test process, part of the test cases can be selected from the test case database, and part of the test cases are added again to complete the test process together.
And S18, determining whether to add the second test case into the test data by using a comparison result of the prediction probability and the actual measurement probability.
Through the steps, the currently acquired test data can be acquired, the test data comprises a plurality of first test cases and test results corresponding to each first test case in the plurality of first test cases, the test results comprise code blocks of software to be tested hit by each first test case in the execution process and at least one type of loopholes detected by each first test case in the execution process, the actual measurement probability of each first test case is obtained through calculation of the test data, the second test case is constructed, the prediction probability of the second test case is acquired based on the actual measurement probability, whether the second test case is added into the test data or not is determined by utilizing the comparison result of the prediction probability and the actual measurement probability, the purpose that whether the new test case can be added into the test data or not is achieved by utilizing the acquired test data to obtain the prediction probability of the new test case for detecting different types of loopholes is achieved, the purpose that the test case contained in the test data is updated continuously is achieved, the dependence of the new test case on manual test experience is reduced, the problem that the test case is difficult to develop due to the fact that the manual test experience is high in the software iteration characteristic is solved, the problem that the test case is difficult to develop the technology is required to be met continuously by the fact that the test case is required to be updated continuously, and the technology is difficult to develop due to the technology is met.
Optionally, the method may further include the following steps:
and S10, performing block processing on codes in the software to be tested to obtain a plurality of code blocks.
In an alternative embodiment, the code in the software under test may be partitioned. For example: and performing blocking processing according to the function level of the submodule of the software to be tested. Specifically, the software A to be tested is subjected to block processing according to the function level of the submodule of the software to be tested to obtain a code block A1, a code block A2, a code block A3 and a code block A4 … … code block An. In the process of executing a plurality of test cases by software test, the hit condition of the plurality of test cases on each code block can be recorded in real time. For example: hit code block A1 total 10 times, hit code block A2 total 0 times, hit code block A3 total 2 times … … and so on, thereby counting hits of each test case in the plurality of test cases to each code block.
Optionally, in step S14, calculating the measured probability of each first test case based on the test data may include performing the steps of:
step S141, obtaining first hit frequencies of each code block in a plurality of first test cases;
Step S142, obtaining the occurrence frequency of each code block in a plurality of code blocks in a first test case of detecting each type of loopholes in at least one type of loopholes;
step S143, obtaining the number ratio of the first test cases of each type of loopholes in at least one type of loopholes relative to the plurality of first test cases;
step S144, based on the first hit frequency, the occurrence frequency and the number ratio, the conditional probability of each type of bug occurring in each first test case under the condition of hitting each code block is calculated, and the actual measurement probability of each first test case is obtained.
In the process of calculating the actual measurement probability of each first test case based on the test data, first, the first hit frequency of each code block in the plurality of code blocks in the plurality of first test cases can be obtained; secondly, the occurrence frequency of each code block in the plurality of code blocks in the first test case of detecting each type of loopholes in at least one type of loopholes can be obtained; then, the number ratio of the first test cases with each type of loopholes in at least one type of loopholes relative to the plurality of first test cases can be obtained; and finally, calculating the conditional probability of each type of loopholes of each first test case under the condition of hitting each code block based on the first hit frequency, the occurrence frequency and the number ratio, and obtaining the actual measurement probability of each first test case.
Optionally, in step S141, acquiring the first hit frequency of each code block of the plurality of code blocks in the plurality of first test cases may include performing the steps of:
step 1411, obtaining a first hit number of each code block in a plurality of first test cases;
step S1412, obtaining the second hit times of the plurality of code blocks in the plurality of first test cases;
in step S1413, a ratio of the first hit number to the second hit number is calculated to obtain a first hit frequency.
The first hit frequency of each code block Ai in a plurality of test cases is counted and denoted as P (Ai). Firstly, acquiring first hit times of each code block in a plurality of first test cases; secondly, obtaining second hit times (namely the total times of all the test cases hitting each code block) of the code blocks in the first test cases; then, the ratio of the first hit times to the second hit times is calculated to obtain the first hit frequency.
Assuming that there are currently 2 test cases, test case 1 and test case 2, respectively, there are currently 3 code blocks, code block A1, code block A2, and code block A3, respectively. Test case 1 hit code block A1 total 2 times, hit code block A2 total 0 times and hit code block A3 total 1 time; while test case 2 hits code block A1 total times, hit code block A2 total 2 times and hit code block A3 total 3 times, whereby the first hit frequency of each code block Ai in a plurality of test cases can be counted:
P(A1)＝(2+1)/(2+1+1+2+3)＝3/9；
P(A2)＝2/(2+1+1+2+3)＝2/9；
P(A3)＝(1+3)/(2+1+1+2+3)＝4/9。
Optionally, in step S142, obtaining the frequency of occurrence of each code block of the plurality of code blocks in the first test case in which each type of vulnerability of the at least one type of vulnerability is detected may include performing the steps of:
step S1421, first test cases in which each type of vulnerability is detected are respectively screened out from a plurality of first test cases;
step S1422, counting the occurrence frequency of each code block in the first test case obtained by screening.
After detecting a bug (bug), the bug needs to be classified, for example: logical class bugs, render class bugs, user Interface (UI) element class bugs, and the like. FIG. 2 is a schematic diagram of statistics of mapping relationships between multiple test cases and vulnerabilities of different types according to an alternative embodiment of the present invention, as shown in FIG. 2, the logical vulnerabilities may include: character blood deduction errors, gold calculation errors, etc., rendering vulnerabilities may include: character frame errors, article material errors, etc. The character blood deduction error is detected through the test case 1, the article material error is detected through the test case 2, the gold coin calculation error is detected through the test case 3, and the character frame error and the article material error are detected through the test case 5, so that bug types (marked as bugX, bugY, bugZ … …) generated in a plurality of test cases can be counted, and a mapping relation can be established between different bug types and the plurality of test cases. Then, screening out test cases with bug from the test cases, thereby counting the occurrence frequency of the code block Ai in at least one test case with bug detected, and marking the occurrence frequency as P (ai|X) in the test cases; screening test cases with bug from a plurality of test cases, thereby counting the occurrence frequency of code blocks Ai in at least one test case with bug detected, and marking the occurrence frequency as P (ai|Y) in the test cases; and screening test cases with the bug Z from the plurality of test cases, thereby counting the occurrence frequency of the code blocks Ai in at least one test case with the bug Z detected, and recording as P (ai|Z) … and the like until the occurrence frequency of each code block in the first test case obtained by screening is completely counted.
Optionally, in step S143, obtaining the number ratio of the first test cases in which each type of the at least one type of the vulnerabilities occurs to the plurality of first test cases may include performing the steps of:
step S1431, obtaining a first number of first test cases in which each type of loopholes in at least one type of loopholes occurs;
step S1432, obtaining the number of second use cases of the plurality of first test cases;
in step S1433, a ratio of the number of the first use cases to the number of the second use cases is calculated to obtain a number ratio.
In order to obtain the number ratio of the first test cases with respect to the plurality of first test cases, wherein each type of loopholes occurs in at least one type of loopholes, first, the number of the first cases of the first test cases with respect to each type of loopholes occurs in at least one type of loopholes can be obtained; secondly, the number of second use cases of the plurality of first test cases can be obtained; then, the ratio of the number of the first cases to the number of the second cases is calculated to obtain the number ratio.
In an alternative embodiment, the ratio of at least one test case that generates a bug to the plurality of test cases may be denoted as P (X), the ratio of at least one test case that generates a bug to the plurality of test cases may be denoted as P (Y), the ratio of at least one test case that generates a bug Z to the plurality of test cases may be denoted as P (Z) …, and so on. Finally, the conditional probability P (x|ai) =p (ai|x) ×p (X)/P (Ai) of occurrence of bugX under the condition that a plurality of test cases hit the Ai code blocks is calculated according to the bayesian theorem. Similarly, the conditional probabilities P (y|ai) and P (z|ai) when other types of bug such as bug and bug Z occur when other code blocks Ai are hit can be calculated. The conditional probability obtained through the calculation not only can provide conditions for the bug forecast probability of the test case calculated later, but also can improve the processing speed of the follow-up real-time settlement probability through the pre-calculation mode.
Optionally, in step S16, obtaining the predicted probability of the second test case based on the measured probability may include performing the steps of:
step S161, obtaining a second hit frequency of each code block in the plurality of code blocks in a second test case;
step S162, obtaining the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block;
step S163, calculating based on the second hit frequency and the conditional probability to obtain the predicted probability of the second test case for each type of vulnerability.
After constructing the new second test case, first, a second hit frequency of each code block in the plurality of code blocks in the second test case may be obtained; secondly, the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block can be obtained; and then, calculating the prediction probability of the second test case for each type of loopholes based on the second hit frequency and the conditional probability.
FIG. 3 is a schematic diagram of predicting the probability of occurrence of a bug of a specific type according to An alternative embodiment of the present invention, as shown in FIG. 3, the probability that the second test case (i.e., the temporary test case) hits each code block A1, A2 … … An may be obtained in real time, and the probability of occurrence of a bug of a specific type may be determined from the conditional probabilities calculated above. For example: for a new test case, the prediction probability of occurrence of the bug X may be calculated according to the frequency of occurrence of Ai in the test case, where P (A1) P (x|a1) +p (A2) P (x|a2) + … … +p (An) P (x|an). Similarly, the prediction probability when other types of bugs such as bugY, bugZ and the like occur can also be calculated.
Optionally, in step S18, determining whether to add the second test case to the test data using the comparison result of the predicted probability and the measured probability may include the following steps:
step S181, setting a judging condition based on the number of the current cases of the plurality of first test cases;
step S182, when the comparison result meets the judgment condition, determining to add the second test case into the test data;
and step S183, when the comparison result does not meet the judgment condition, performing iterative processing on the second test case until the comparison result meets the judgment condition.
In the process of determining whether to add the second test case into the test data by using the comparison result of the prediction probability and the actual measurement probability, firstly, setting a judging condition based on the number of the current cases of the plurality of first test cases; then, when the comparison result meets the judgment condition, determining to add the second test case into the test data; and when the comparison result does not meet the judgment condition, carrying out iterative processing on the second test case until the comparison result meets the judgment condition.
Specifically, when the number of first test cases existing in the test case database is large, in order to prevent redundancy of the test cases, the following determination conditions may be set: if the second test case finds that the prediction probability of all bug types is higher than the actual measurement probability of the existing first test case, the second test case can be subjected to warehousing operation. If the second test case finds that the prediction probability of part of bug types is lower than the actual measurement probability of the existing first test case, iterative operation is needed to be carried out on the second test case so as to improve the prediction occurrence probability, and the warehousing operation can be executed on the second test case until the prediction probability of all bug types is found to be higher than the actual measurement probability of the existing first test case. In the process of performing iterative operation on the second test case, part of operations performed by the second test case can be modified. For example: the second test case originally comprises an operation a, an operation b and an operation c, and comprises an operation a, an operation b, an operation d and an operation e after modification, so that the prediction probability of the second test case is improved until the prediction probability of all bug types is found to be higher than the actual measurement probability of the existing first test case.
Therefore, when a new second test case is tested, the prediction probability of the bug of the specific type can be obtained and used for judging the prediction value of the second test case, so that a foundation is laid for simplifying and effectively laying the test cases in the follow-up test case database.
From the description of the above embodiments, it will be clear to a person skilled in the art that the method according to the above embodiments may be implemented by means of software plus the necessary general hardware platform, but of course also by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a storage medium (e.g. ROM/RAM, magnetic disk, optical disk) comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to perform the method according to the embodiments of the present invention.
The embodiment also provides a processing device for a test case, which is used for implementing the above embodiment and the preferred implementation manner, and is not described in detail. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. While the means described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.
FIG. 4 is a block diagram of a processing device for test cases according to one embodiment of the present invention, as shown in FIG. 4, the device includes: the acquiring module 10 is configured to acquire currently acquired test data, where the test data includes: the test method comprises the steps of a plurality of first test cases and test results corresponding to each first test case in the plurality of first test cases, wherein the test results comprise: code blocks of the software to be tested hit by each first test case in the execution process and at least one type of loopholes detected by each first test case in the execution process; the calculation module 20 is configured to calculate an actual measurement probability of each first test case based on the test data; the processing module 30 is configured to construct a second test case, and obtain a prediction probability of the second test case based on the actual measurement probability; and the comparison module 40 is configured to determine whether to add the second test case to the test data according to the comparison result of the predicted probability and the measured probability.
Optionally, fig. 5 is a block diagram of a processing apparatus for a test case according to an alternative embodiment of the present invention, and as shown in fig. 5, the apparatus includes, in addition to all the modules shown in fig. 4: the dividing module 50 is configured to perform a block processing on a code in the software to be tested, so as to obtain a plurality of code blocks.
Optionally, the calculating module 20 is configured to obtain a first hit frequency of each code block in the plurality of first test cases; acquiring the occurrence frequency of each code block in the plurality of code blocks in a first test case in which each type of vulnerability in at least one type of vulnerability is detected; acquiring the number ratio of the first test cases with each type of loopholes in at least one type of loopholes relative to a plurality of first test cases; based on the first hit frequency, the occurrence frequency and the number ratio, calculating the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block, and obtaining the actual measurement probability of each first test case.
Optionally, the calculating module 20 is configured to obtain a first hit number of each code block in the plurality of first test cases; acquiring second hit times of the code blocks in the first test cases; and calculating the ratio of the first hit times to the second hit times to obtain the first hit frequency.
Optionally, the computing module 20 is configured to screen out the first test cases that detect each type of vulnerability from the plurality of first test cases respectively; and counting the occurrence frequency of each code block in the first test case obtained by screening.
Optionally, the computing module 20 is configured to obtain a first number of cases of the first test case where each type of the at least one type of vulnerabilities occurs; acquiring the number of second cases of the plurality of first test cases; and calculating the ratio of the number of the first cases to the number of the second cases to obtain the number ratio.
Optionally, the processing module 30 is configured to obtain a second hit frequency of each code block in the plurality of code blocks in the second test case; acquiring the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block; and calculating the prediction probability of the second test case for each type of loopholes based on the second hit frequency and the conditional probability.
Optionally, the comparing module 40 is configured to set a determining condition based on the number of current cases of the plurality of first test cases; when the comparison result meets the judgment condition, determining to add the second test case into the test data; and when the comparison result does not meet the judgment condition, carrying out iterative processing on the second test case until the comparison result meets the judgment condition.
It should be noted that each of the above modules may be implemented by software or hardware, and for the latter, it may be implemented by, but not limited to: the modules are all located in the same processor; alternatively, the above modules may be located in different processors in any combination.
Embodiments of the present invention also provide a non-volatile storage medium having a computer program stored therein, wherein the computer program is arranged to perform the steps of any of the method embodiments described above when run.
Alternatively, in the present embodiment, the above-described nonvolatile storage medium may be configured to store a computer program for performing the steps of:
s1, acquiring currently acquired test data, wherein the test data comprise: the test method comprises the steps of a plurality of first test cases and test results corresponding to each first test case in the plurality of first test cases, wherein the test results comprise: code blocks of the software to be tested hit by each first test case in the execution process and at least one type of loopholes detected by each first test case in the execution process;
s2, calculating actual measurement probability of each first test case based on the test data;
s3, constructing a second test case, and acquiring the prediction probability of the second test case based on the actual measurement probability;
s4, determining whether to add the second test case into the test data by using a comparison result of the prediction probability and the actual measurement probability.
Alternatively, in the present embodiment, the above-described nonvolatile storage medium may include, but is not limited to: a usb disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing a computer program.
An embodiment of the invention also provides an electronic device comprising a memory having stored therein a computer program and a processor arranged to run the computer program to perform the steps of any of the method embodiments described above.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, where the transmission device is connected to the processor, and the input/output device is connected to the processor.
Alternatively, in the present embodiment, the above-described processor may be configured to execute the following steps by a computer program:
s1, acquiring currently acquired test data, wherein the test data comprise: the test method comprises the steps of a plurality of first test cases and test results corresponding to each first test case in the plurality of first test cases, wherein the test results comprise: code blocks of the software to be tested hit by each first test case in the execution process and at least one type of loopholes detected by each first test case in the execution process;
s2, calculating actual measurement probability of each first test case based on the test data;
s3, constructing a second test case, and acquiring the prediction probability of the second test case based on the actual measurement probability;
S4, determining whether to add the second test case into the test data by using a comparison result of the prediction probability and the actual measurement probability.
Alternatively, specific examples in this embodiment may refer to examples described in the foregoing embodiments and optional implementations, and this embodiment is not described herein.
The foregoing embodiment numbers of the present application are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
In the foregoing embodiments of the present application, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.
Claims (10)
1. A method for processing a test case, comprising:
acquiring currently acquired test data, wherein the test data comprises: the test system comprises a plurality of first test cases and test results corresponding to each of the plurality of first test cases, wherein the test results comprise: code blocks of the software to be tested hit by each first test case in the execution process and at least one type of loopholes detected by each first test case in the execution process;
calculating actual measurement probability of each first test case based on the test data;
constructing a second test case, and acquiring the prediction probability of the second test case based on the actual measurement probability;
determining whether to add the second test case into the test data by using a comparison result of the predicted probability and the actually measured probability;
wherein the method further comprises: partitioning the codes in the software to be tested to obtain a plurality of code blocks;
The obtaining the predicted probability of the second test case based on the measured probability includes: acquiring a second hit frequency of each code block in the plurality of code blocks in the second test case; acquiring the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block; and calculating the prediction probability of each type of vulnerability of the second test case based on the second hit frequency and the conditional probability.
2. The processing method of claim 1, wherein calculating the measured probability for each first test case based on the test data comprises:
acquiring first hit frequencies of each code block in a plurality of code blocks in the plurality of first test cases;
acquiring the occurrence frequency of each code block in the plurality of code blocks in a first test case of detecting each type of loopholes in the at least one type of loopholes;
acquiring the number ratio of the first test cases of each type of loopholes in the at least one type of loopholes relative to the plurality of first test cases;
based on the first hit frequency, the occurrence frequency and the number ratio, calculating the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block, and obtaining the actual measurement probability of each first test case.
3. The processing method of claim 2, wherein obtaining the first hit frequency of each code block of the plurality of code blocks in the plurality of first test cases comprises:
acquiring first hit times of each code block in the plurality of code blocks in the plurality of first test cases;
acquiring second hit times of the code blocks in the first test cases;
and calculating the ratio of the first hit times to the second hit times to obtain the first hit frequency.
4. The processing method of claim 2, wherein obtaining the frequency of occurrence of each code block of the plurality of code blocks in the first test case where each type of vulnerability of the at least one type of vulnerability is detected comprises:
screening out the first test cases in which each type of vulnerability is detected from the plurality of first test cases respectively;
and counting the occurrence frequency of each code block in the first test case obtained by screening.
5. The processing method of claim 2, wherein obtaining the count ratio of first test cases for which each type of vulnerability of the at least one type of vulnerability occurs relative to the plurality of first test cases comprises:
Acquiring the number of first cases of first test cases of each type of loopholes in the at least one type of loopholes;
acquiring the number of second cases of the plurality of first test cases;
and calculating the ratio of the number of the first cases to the number of the second cases to obtain the number ratio.
6. The processing method according to claim 1, wherein determining whether to add the second test case to the test data using a result of comparing the predicted probability with the measured probability comprises:
setting a judging condition based on the number of the current cases of the plurality of first test cases;
when the comparison result meets the judging condition, determining to add the second test case into the test data; and when the comparison result does not meet the judgment condition, carrying out iterative processing on the second test case until the comparison result meets the judgment condition.
7. A test case processing apparatus, comprising:
the device comprises an acquisition module, a data processing module and a data processing module, wherein the acquisition module is used for acquiring currently acquired test data, and the test data comprises: the test system comprises a plurality of first test cases and test results corresponding to each of the plurality of first test cases, wherein the test results comprise: code blocks of the software to be tested hit by each first test case in the execution process and at least one type of loopholes detected by each first test case in the execution process;
The calculation module is used for calculating actual measurement probability of each first test case based on the test data;
the processing module is used for constructing a second test case and acquiring the prediction probability of the second test case based on the actual measurement probability;
the comparison module is used for determining whether the second test case is added into the test data or not by utilizing the comparison result of the prediction probability and the actual measurement probability;
wherein the apparatus further comprises: the dividing module is used for carrying out block processing on codes in the software to be detected to obtain a plurality of code blocks;
the processing module is further configured to: acquiring a second hit frequency of each code block in the plurality of code blocks in the second test case; acquiring the conditional probability of each type of vulnerability of each first test case under the condition of hitting each code block; and calculating the prediction probability of each type of vulnerability of the second test case based on the second hit frequency and the conditional probability.
8. A non-volatile storage medium, wherein a computer program is stored in the storage medium, wherein the computer program is arranged to execute the method of processing a test case according to any of the claims 1 to 6 at run-time.
9. A processor, characterized in that the processor is arranged to run a program, wherein the program is arranged to execute the method of processing a test case as claimed in any of the claims 1 to 6 at run-time.
10. An electronic device comprising a memory and a processor, characterized in that the memory has stored therein a computer program, the processor being arranged to run the computer program to perform the method of processing a test case as claimed in any of the claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011247876.8A CN112363928B (en) | 2020-11-10 | 2020-11-10 | Processing method and device of test case, processor and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011247876.8A CN112363928B (en) | 2020-11-10 | 2020-11-10 | Processing method and device of test case, processor and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112363928A CN112363928A (en) | 2021-02-12 |
| CN112363928B true CN112363928B (en) | 2023-08-22 |
Family
ID=74508671
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011247876.8A Active CN112363928B (en) | 2020-11-10 | 2020-11-10 | Processing method and device of test case, processor and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112363928B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103208039A (en) * | 2012-01-13 | 2013-07-17 | 株式会社日立制作所 | Method and device for evaluating software project risks |
| CN105243014A (en) * | 2015-09-18 | 2016-01-13 | 北京金山安全软件有限公司 | Method and device for testing application program |
| CN106021115A (en) * | 2016-06-06 | 2016-10-12 | 重庆大学 | Non-supervision defect prediction method based on probabilities |
| JP2017224185A (en) * | 2016-06-16 | 2017-12-21 | 日本電信電話株式会社 | Bug contamination probability calculation program and bug contamination probability calculation method |
| CN108268373A (en) * | 2017-09-21 | 2018-07-10 | 平安科技（深圳）有限公司 | Automatic test cases management method, device, equipment and storage medium |
| CN109062794A (en) * | 2018-07-24 | 2018-12-21 | 北京奇艺世纪科技有限公司 | A kind of the determination method, apparatus and electronic equipment of software evaluating result |
| CN110297764A (en) * | 2019-05-30 | 2019-10-01 | 北京百度网讯科技有限公司 | Loophole test model training method and device |
| CN110826071A (en) * | 2019-09-24 | 2020-02-21 | 平安科技（深圳）有限公司 | Software vulnerability risk prediction method, device, equipment and storage medium |
| CN111274126A (en) * | 2020-01-14 | 2020-06-12 | 华为技术有限公司 | Test case screening method, device and medium |
| WO2020151344A1 (en) * | 2019-01-22 | 2020-07-30 | 平安科技（深圳）有限公司 | Software development and testing method for sub-iterations and related devices |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140033174A1 (en) * | 2012-07-29 | 2014-01-30 | International Business Machines Corporation | Software bug predicting |
| US10983853B2 (en) * | 2017-03-31 | 2021-04-20 | Microsoft Technology Licensing, Llc | Machine learning for input fuzzing |
| US10514890B2 (en) * | 2017-11-15 | 2019-12-24 | Accenture Global Solutions Limited | Test case and data selection using a sampling methodology |
| US20190243743A1 (en) * | 2018-02-07 | 2019-08-08 | Apple Inc. | Unsupervised anomaly detection |
-
2020
- 2020-11-10 CN CN202011247876.8A patent/CN112363928B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103208039A (en) * | 2012-01-13 | 2013-07-17 | 株式会社日立制作所 | Method and device for evaluating software project risks |
| CN105243014A (en) * | 2015-09-18 | 2016-01-13 | 北京金山安全软件有限公司 | Method and device for testing application program |
| CN106021115A (en) * | 2016-06-06 | 2016-10-12 | 重庆大学 | Non-supervision defect prediction method based on probabilities |
| JP2017224185A (en) * | 2016-06-16 | 2017-12-21 | 日本電信電話株式会社 | Bug contamination probability calculation program and bug contamination probability calculation method |
| CN108268373A (en) * | 2017-09-21 | 2018-07-10 | 平安科技（深圳）有限公司 | Automatic test cases management method, device, equipment and storage medium |
| CN109062794A (en) * | 2018-07-24 | 2018-12-21 | 北京奇艺世纪科技有限公司 | A kind of the determination method, apparatus and electronic equipment of software evaluating result |
| WO2020151344A1 (en) * | 2019-01-22 | 2020-07-30 | 平安科技（深圳）有限公司 | Software development and testing method for sub-iterations and related devices |
| CN110297764A (en) * | 2019-05-30 | 2019-10-01 | 北京百度网讯科技有限公司 | Loophole test model training method and device |
| CN110826071A (en) * | 2019-09-24 | 2020-02-21 | 平安科技（深圳）有限公司 | Software vulnerability risk prediction method, device, equipment and storage medium |
| CN111274126A (en) * | 2020-01-14 | 2020-06-12 | 华为技术有限公司 | Test case screening method, device and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112363928A (en) | 2021-02-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Mahmoud et al. | Pytorchfi: A runtime perturbation tool for dnns | |
| Oliner et al. | Carat: Collaborative energy diagnosis for mobile devices | |
| CN109062809B (en) | Online test case generation method and device and electronic equipment | |
| CN111124926B (en) | Fuzzy test method and device, electronic equipment and storage medium | |
| CN110619210A (en) | Simulator detection method and system | |
| US11146580B2 (en) | Script and command line exploitation detection | |
| CN107145782A (en) | A kind of recognition methods, mobile terminal and the server of abnormal application program | |
| CN109062794A (en) | A kind of the determination method, apparatus and electronic equipment of software evaluating result | |
| CN113268243B (en) | Memory prediction method and device, storage medium and electronic equipment | |
| CN110674047A (en) | Software testing method and device and electronic equipment | |
| CN107391362A (en) | Application testing method, mobile terminal and storage medium | |
| CN111078579A (en) | Monkey testing method and device and terminal equipment | |
| CN111611390B (en) | Data processing method and device | |
| CN110297764B (en) | Vulnerability test model training method and device | |
| CN112148594A (en) | Script testing method and device, electronic equipment and storage medium | |
| CN112363928B (en) | Processing method and device of test case, processor and electronic device | |
| US20240086310A1 (en) | What-if analysis for notebooks | |
| CN106034150B (en) | Application program dynamic pushing method, device and system | |
| CN109086198A (en) | Test method, device and the storage medium of database | |
| Navarro et al. | Using microsoft solver foundation to analyse feature models and configurations | |
| CN112527681B (en) | Program vulnerability detection method and device | |
| CN116149978A (en) | Service interface testing method and device, electronic equipment and storage medium | |
| CN109189827B (en) | Time sequence processing method and device and electronic equipment | |
| US20200250072A1 (en) | Software component verification using random selection on pooled devices | |
| CN113051153A (en) | Application software vulnerability scanning method and related equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |